Yield to iterable

iterator generators

Nick Ribal

Front-end consultant, freelancer and a family man

Part 1: Iterables

Why iterability?
ES2015 iteration protocol interfaces
- Iterable
- Iterator
- IteratorResult
Iterables in ES2015
Implementing iterables

What's wrong here?

Iteration entails coupling and lack of encapsulation:

Know what you're iterating
Based on that, decide how to iterate
Use collection implementation details to iterate

// 1. Array and TypedArray
const array = ['this', 'is', 'aweful']
array.forEach(member => console.log(member))

3 different ways to do the same basic task

Can't (easily) iterate non built-ins

// 3. Newer ES2015 collections
for (member of new Set(array)) console.log(member)

// 2. Array-like: arguments, NodeList, String, etc
(function(){
  for (let i = 0; i < arguments.length; i++) {
    console.log(arguments[i])
  }
}).apply(null, array)

// Some things aren't what they seem
for (let i = 0; i < '😀'.length; i++)
  console.log('😀'[i]) // � � Unicode in JS

Iterables solve all that!

const array = ['😀', '😀']

function getCollections(){
  return [
    '😀',
    array,
    arguments,
    new Set(['😀']),
    new Map([array])
  ]
}

for (const collection of getCollections()) {
  for (const member of collection) {
    console.log(member)
  }
}

Iterate anything!

Implementation is hidden

Object with [Symbol.iterator] property (own or in prototype chain), whose value is a function, which returns an Iterator

Iterable: an object which can be iterated

String, Array, TypedArray, Map, Set, arguments, NodeList (DOM collections), generator object

// In TypeScript notation
interface Iterable {
  [Symbol.iterator](): Iterator;
}

Built-in Iterables

iterator.next() can be called to request a value

next() will return IteratorResult object

Iterator: object with next() method

iterator.return() is called when iteration terminates by break, continue, return or throw

returns { value, done: true }

interface Iterator {
  next(): IteratorResult;
  return?(value?): IteratorResult;
}

Optional return() closes iterator

{ value, done: false }

as long as iterator has values

IteratorResult is returned per iteration

return() or after iterator exhaustion should return

{ done: true }

interface IteratorResult {
  value: any;
  done: boolean;
}

{ done: true }

after last value

➡

Calling next() when iteration ended

A protocol defines interfaces (signatures for methods or functions) and rules for using them

interface Iterable {
  [Symbol.iterator](): {
    // Iterator
    next(): {
      // IteratorResult
      value: any;
      done: boolean;
    };
    return?(value?): {
      // IteratorResult
      value: any;
      done: boolean;
    };
  };
}

Formalizing iteration protocol

Iterable has Iterator, which repeatedly returns IteratorResult(s)

ES2015 💜 Iterables

Map([iterable]), WeakMap([iterable])
Set([iterable]), WeakSet([iterable])
Array.from([iterable]), TypedArray.from([iterable])
Array, TypedArray, Map, Set and Object have entries(), keys() and values() methods, returning iterables
Promise.all([iterable]), Promise.race([iterable])
NodeList, arguments, String, Map, Set

ES2015 Iteration syntax constructs

const iterable = ['😀', '😀😀', '😀😀😀']
// for-of loop
for (const member of iterable)
  console.log(member)

// destructuring assignment
const [ first, second, ] = iterable

// array spread operator
const [ firstAgain, ...secondAndOn] = iterable

// recursive yield in generator
function* generator(){ yield* iterable }
for (const member of generator())
  console.log(member)

A simple iterable

const plan1 = { // Iterable
  [Symbol.iterator](){
    let step = 0
    const steps = [  // IteratorResult, not done
      { done: false, value: '1. Learn iterables!' },
      { done: false, value: '2. Learn generators' },
      { done: false, value: '3. ????????????????' },
      { done: false, value: '4. PROFIT!!!!!!!!!!' }
    ]
    return { // Iterator
      next: () => steps[step++] || { done: true }
    }
  }
}

for (const step of plan1)
  console.log(step)

1. Learn iterables!
2. Learn generators
3. ????????????????
4. PROFIT!!!!!!!!!!

Output

An iterable iterator

const plan2 = (function(step = 0){
  const steps = [ // IteratorResult
    { done: false, value: '1. Learn iterables!' },
    { done: false, value: '2. Learn generators' },
    { done: false, value: '3. ????????????????' },
    { done: false, value: '4. PROFIT!!!!!!!!!!' }
  ]
  return {
    [Symbol.iterator](){ return this }, // Iterable
    next(){ // Iterator
      const isDone = step >= steps.length // exhaustion
      return isDone ? { done: true } : steps[step++]
    }
  }
})()
// a for-of loop receives only the value 
for (const step of plan2) console.log(step)

Interrupt, resume, exhaust

for (const step of plan2) {
  console.log(step)
  break;
}
// > 1. Learn iterables!

for (const step of plan2) {
  console.log(step)
}
// > 2. Learn generators
// > 3. ????????????????
// > 4. PROFIT!!!!!!!!!!

interrupted, has more values

finished, exhausted

Pause/resume means iteration is LAZY

Closing iterators using return()

const plan3 = (function(step = 0){
  const steps = [
    '1. Learn iterables!', '2. Learn generators',
    '3. ????????????????', '4. PROFIT!!!!!!!!!!',
  ], { length: len, } = steps
  return {
    [Symbol.iterator](){ return this },
    next: () => ({ value: steps[step++], done: step > len }),
    return(){
      step = len + 1 // cleanup here
      return { done: true }
    }
  }
})()

for (const step of plan3) {
  console.log(step)
  break // or continue or throw or return
}
// > 1. Learn iterables!
for (const step of plan3) console.log(step)
// return() closed iterator, so no more values

- Iterator

Part 2: Generators

What are generators?
Generator functions, their variations
iterator = generator()
yielding
Bidirectional, yet asymmetrical
Use cases
- Iterators: producing values
- Observers: consuming values
- Coroutines: interchanging values bidirectionally

Generator function

Generators are functions which can be exited and later re-entered. Their variables are saved across re-entrances.

Calling a generator function does not execute it, but returns an iterator for that generator

I'm back - Iterator

variations of function*


function* generatorFnDeclaration(){}

const generatorFnExpression = function*(){}

const objectLiteralWithGeneratorMethod = {
  * genratorMethod(){}
}

class ClassWithGeneratorMethods {
  * instanceGeneratorMethod(){}
  static * classGeneratorMethod(){}
}

Generator > Iterator > IterationResult(s)

const { log, } = console

function* generator(){
  log('before yield')
  yield 'yielded value'
  log('after yield')
}

const iterator = generator()

for (const value of iterator)
  log(value)

'before yield'
'yielded value'
'after yield'

Output

Looks simple, right?

valueFromNext = yield valueToNext

yield causes the generator function to pause and returns the yielded value to the calling iterator's next() as the return value, wrapped in an IteratorResult object

function* generator(){ yield 'foo' }
const iterator = generator()

console.log(iterator.next()) // { value: 'foo', done: false }
console.log(iterator.next()) // { value: undefined, done: true }
console.log(iterator.next()) // { value: undefined, done: true }

A paused generator is resumed by calling iterator's next().

Each next() call, resumes generator execution and runs until nearest yield/throw/return or end of generator function

Under the hood

const { log, } = console

function* generator(){
  log('G: before yield')
  yield 'G: yielded value'
  log('G: after yield')
}

const iterator = generator()

log(iterator.next())
// > 'G: before yield'
// > { value: 'G: yielded value', done: false }

log(iterator.next())
// > 'G: after yield'
// > { value: undefined, done: true }

log(iterator.next())
// > {
// >   value: undefined,
// >   done: true
// > }

log(iterator.next())
// > {
// >   value: undefined,
// >   done: true
// > }

No output cause only the controlling iterator's next() call runs the generator till following yield

generator pauses after yielding

Below the surface: yield returns a value

const { log, } = console

function* generator(){
  log('G: before yield')
  log('G: after yield', yield 'G: yielded value')
}

const iterator = generator()

for (const value of iterator)
  log(value)

'G: before yield'
'G: yielded value'
'G: after yield' undefined

iterator's next()...

...is next(undefined)

...back to next()

yield / next() are bidirectional

const { log, } = console

function* generator(){
  log('G: before yield')
  const valueFromNext = yield 'G: yielded value'
  log('G: after yield', valueFromNext)
}

const iterator = generator()

log(iterator.next('I: 1'))
// > 'G: before yield'
// > { value: 'G: yielded value', done: false }

log(iterator.next('I: 2'))
// > 'G: after yield' 'I: 2'
// > { value: undefined, done: true }

log(iterator.next('I: 3'))
// > {
// >   value: undefined,
// >   done: true
// > }

log(iterator.next('I: 4'))
// > {
// >   value: undefined,
// >   done: true
// > }

No output cause only the controlling iterator's next() call runs the generator till following yield

generator pauses after yielding

Bidirectional, yet asymmetrical

const { log, } = console

function* generator(){
  log('G: before yield')
  const valueFromNext = yield 'G: yielded value'
  log('G: after yield', valueFromNext)
}

const iterator = generator()

log(iterator.next('I: 1'))
// > 'G: before yield'
// > { value: 'G: yielded value', done: false }

log(iterator.next('I: 2'))
// > 'G: after yield' 'I: 2'
// > { value: undefined, done: true }

sent...

...recieved

???

What?!?!?

const { log } = console

function* generator(){
log('G: before yield')
const valueFromNext = yield 'G: yielded value'
log('G: after yield', valueFromNext)
}

const iterator = generator()

log(iterator.next('I: 1'))
// > 'G: before yield'
// > { value: 'G: yielded value', done: false }

log(iterator.next('I: 2'))
// > 'G: after yield' 'I: 2'
// > { value: undefined, done: true }

An infinite generator

function* series(step = 1, start = 0){
  while (true) yield start += step
}

const naturalNumbers = series()
// iterables are lazy, so this is cool
const [one, two, ] = naturalNumbers

// but these cause a stack overflow
const [f, s, ...rest, ] = naturalNumbers
for (const n of naturalNumbers) {
  // without a break/continue/throw
}

A "simple" objectEntries iterable

function objectEntries(obj){
  let index = 0 // Reflect.ownKeys() gets strings/symbols
  const propKeys = Reflect.ownKeys(obj)
  return {
    [Symbol.iterator](){ return this },
    next(){
      if (index < propKeys.length) {
        const key = propKeys[index++]
        return {
          value: [key, obj[key]]
        }
      }
      return { done: true }
    }
  }
}

Initialization

An awkward loop-like terminator monster

Iteration

Exhaustion

Generator based objectEntries

function* objectEntries(obj){
  for (const key of Reflect.ownKeys(obj)) {
    yield [key, obj[key]]
  }
}


function map(obj, fn, result = {}){
  for (const [key, value] of objectEntries(obj)) {
    result[key] = fn(value)
  }
  return result
}

map({ foo: 2, bar: 5 }, x => x**2) // ES2016 FTW!!!
// { foo: 4, bar: 25 }

Generators express the same, using much shorter syntax

Because they reintroduce loop syntax back to iterators!

Recursive yield via yield*

function* recursive(){
  yield 'sequence'
  yield* ['of', 'yielded']
  yield 'values'
}

const series = recursive()
for (const i of series) {
  console.log(i)
}

// > "sequence"
// > "of"
// > "yielded"
// > "values"

function* flat(){
  yield 'sequence'
  yield ['of', 'yielded']
  yield 'values'
}

const series = flat()
for (const i of series) {
  console.log(i)
}

// > "sequence"
// > ["of", "yielded"]
// > "values"

yield* operates on iterables

function* g2(){
  yield 2
  yield* [3, 4]
  yield* '5😀'
}

function* g1(){
  yield '1 start'
  yield* g2()
  yield '7 end'
}

for (const i of g1()) console.log(i)

'1 start'
2
3
4
'5'
'😀'
'7 end'

Output

Generator use cases

Data producers: iterable iterators, infinite and finite - we covered
Data consumers: observers, to which you can 'push' values to be processed lazily
Co-routines: bidirectional communication

Here be dragons

Super advanced and interesting stuff

Sources and references

Exploring ES6 book, chapter 21. Iterables and iterators by Dr Axel Rauschmayer
Exploring ES6 book, chapter 22. Generators by Dr Axel Rauschmayer
Understanding ECMAScript 6 book, chapter Iterators and Generators by Nicholas C. Zakas
Learning ES6: Iterators & iterables by Ben Ilegbodu
No promises: asynchronous JavaScript with only generators by Dr Axel Rauschmayer
The Hidden Power of ES6 Generators: Observable Async Flow Control by Eric Elliott
MDN: Iteration protocols, function*, Generator

Thank you and

stay curious :)

Yield to iterable iterator generators

Iterable, iterator, generator, function*, yield, yield*, spread operator, destructuring, for-of and Symbol.iterable are commonly used by ES2015 built-ins. Learn why and how these related concepts come together to implement ES2015's advanced features - so you too can harness their power!

1,984

Nick Ribal

Front-end veteran, consultant and freelancer, who's armed with a decade of experience solving challenges on the Web platform for large Internet companies.

Yield to iterable

iterator generators

Nick Ribal

elektronik2k5

@elektronik2k5

Front-end consultant, freelancer and a family man

about.me/nickribal

@elektronik

Part 1: Iterables

Why iterability?

ES2015 iteration protocol interfaces

Iterables in ES2015

Implementing iterables

What's wrong here?

Iterables solve all that!

Iterate anything!

Implementation is hidden

Iterable: an object which can be iterated

Built-in Iterables

Iterator: object with next() method

Optional return() closes iterator

IteratorResult is returned per iteration

➡

Calling next() when iteration ended

Formalizing iteration protocol

ES2015 💜 Iterables

ES2015 Iteration syntax constructs

A simple iterable

Output

An iterable iterator

Interrupt, resume, exhaust

interrupted, has more values

finished, exhausted

Pause/resume means iteration is LAZY

Closing iterators using return()

- Iterator

Part 2: Generators

Use cases

Generator function

variations of function*

Generator > Iterator > IterationResult(s)

Output

Looks simple, right?

valueFromNext = yield valueToNext

Under the hood

Below the surface: yield returns a value

...back to next()

yield / next() are bidirectional

Bidirectional, yet asymmetrical

What?!?!?

An infinite generator

A "simple" objectEntries iterable

An awkward loop-like terminator monster

Generator based objectEntries

Generators express the same, using much shorter syntax

Because they reintroduce loop syntax back to iterators!

Recursive yield via yield*

yield* operates on iterables

Output

Generator use cases

Data producers: iterable iterators, infinite and finite - we covered

Data consumers: observers, to which you can 'push' values to be processed lazily

Co-routines: bidirectional communication

Here be dragons

Super advanced and interesting stuff

Sources and references

Thank you and

stay curious :)

about.me/nickribal

Yield to iterable iterator generators

More from Nick Ribal