Iterators & Generators

Collections

  • list
  • tuple
  • set
  • dict

Iterable

  • list
  • tuple
  • set
  • dict
  • str
  • file objects

Object of any class that defines __iter__ is called an iterable. Iterable is capable of returning only one of it's members at a time.

Iterator

  • An iterator is an object that represents a stream of data
  • An iterator is every object in Python that implements the iterator protocol
  • __iter__ + __next__
  • StopIteration

How can we use this?

What is "for"?

# for-loop pseudo code

iterator_obj = iter(iterable)

while True:
    try:
        element = next(iterator_obj)
    except StopIteration:
        break

Custom iterators

class Pandas:
    def __init__(self):
        self.data = [
            {'name': 'Ivo', 'kg': 100},
            {'name': 'Marto', 'kg': 80},
            {'name': 'Pesho', 'kg': 120}
        ]

    def __iter__(self):
        self.index = 0
        return self  # Important!

    def __next__(self):
        index = self.index

        self.index += 1

        try:
            return self.data[index]
        except IndexError:
            raise StopIteration

Using custom iterators

pandas = Pandas()
for panda in pandas:
    print(panda)
    
    
╰─Ѫ py demo.py                                                                                                                                             1 ↵
{'name': 'Ivo', 'kg': 100}
{'name': 'Marto', 'kg': 80}
{'name': 'Pesho', 'kg': 120}
pandas = Pandas()
pandas_iterator = iter(pandas)
print(next(pandas_iterator))


╰─Ѫ py demo.py                                                                                                                                             1 ↵
{'name': 'Ivo', 'kg': 100}

How iter() works?

  1. Tries to call the __iter__ method of the passed argument
  2. If there is no __iter__ method, it consecutively tries to call __getitem__(index), starting with index = 0 and continues until StopIteration is raised
  3. If there is neither __iter__ nor __getitem__, it raises an error that the object is not iterable.

iter() example

class Wallet1:
    def __init__(self, money):
        self.money = money

    def __iter__(self):
        self.index = 0
        return self

    def __next__(self):
        if self.index >= len(self.money):
            raise StopIteration

        element = self.money[self.index]
        self.index += 1
        return element
class Wallet2:
    def __init__(self, money):
        self.money = money

    def __getitem__(self, index):
        if index >= len(self.money):
            raise StopIteration

        return self.money[index]
money = [10, 10, 5, 100, 50, 20]
zipped_wallets = zip(Wallet1(money), Wallet2(money))
print([(x, y) for x, y in zipped_wallets])

╰─Ѫ py demo.py
[(10, 10), (10, 10), (5, 5), (100, 100), (50, 50), (20, 20)]

Infinite iterators

class OddNumbers:
    def __iter__(self):
        self.number = 1
        return self

    def __next__(self):
        number = self.number

        self.number += 2

        return number


odd_numbers = iter(OddNumbers())

print(next(odd_numbers))
print(next(odd_numbers))
print(next(odd_numbers))
print(next(odd_numbers))


╰─Ѫ py demo.py                                                                                                                                             1 ↵
1
3
5
7

Let's get lazy!

What is "lazy"

Laziness in programming usually means that the element is generated only when it's needed/invoked.

numbers = [[1, 2, 3], [1, 2, 3], [1, 2, 3]]

sums = (sum(x) for x in numbers)

numbers[2].append(100)

for x in sums:
    print(x)
    

╰─Ѫ py demo.py
6
6
106

Python builtin lazy objects

  • range
  • map
  • zip
  • filter
  • enumerate
  • etc.

Function Execution

Every function is executed from its first line. Then execution continues until:

An Exception is raised.

A return statement. Return means that the function returns control to the point where it was called.

A yield statement. Yield means that the function is "paused" in its current state. The transfer of control is temporary and the function expects to regain it in future.

Generator

  • Every Python function that contains at least one yield in it is automatically transformed into a generator function.
  • Generator function creates generator iterator a.k.a. generator.
  • We can get the values from a generator by calling next
  • Once a generator has been exhausted it will raise StopIteration
  • You can consume all values from a generator only once

Generator example

def random_gen():
    index = 1

    print(f'Called for {index} time')
    yield randint(1, 10)
    index += 1

    print(f'Called for {index} time')
    yield randint(1, 10)
    index += 1

    print(f'Called for {index} time')
    yield randint(1, 10)
    index += 1
gen = random_gen()

print(next(gen))
print(next(gen))
print(next(gen))

╰─Ѫ py demo.py
Called for 1 time
5
Called for 2 time
6
Called for 3 time
8

for x in random_gen():
    print(x)
    

╰─Ѫ py demo.py
Called for 1 time
7
Called for 2 time
6
Called for 3 time
4

Fibonacci generator

def fib():
    a, b = 1, 1

    while a < 100:
        yield a
        a, b = b, a + b


for f in fib():
    print(f)
    

╰─Ѫ py demo.py
1
1
2
3
5
8
13
21
34
55
89

.send()

def complex_generator():
    items = [1, 2, 3]
    wtf = 1

    for item in items:
        value = (yield item)

        if value is not None:
            wtf += value

        print(wtf, item)

    return 42
  • send allows us to return values back to the generator
  • x.send(None) == next(x)

Python 101 9th Iterators & Generators

By Hack Bulgaria

Made with Slides.com

Python 101 9th Iterators & Generators

  • 984

More from Hack Bulgaria