async python

tuna vargi

software developer @ hipo

async code is more efficient but maybe harder

parallelism

Use multiple processors to make computation faster

concurrency

permit multiple tasks to proceed without waiting for each other

is python supports async code

python threads

• Python uses real system threads
• POSIX threads
• Fully managed by operating system

Global Interpreter Lock

• Parallel execution is blocked
• Only one thread for one interpreter
• Simplifies many low-level details(Memory management etc)

thread execution

• When a thread is running, it holds the GIL
• GIL released on I/O (read,write,send,recv,etc.)

concurrency problem

• Multiple Processes
• Threads
• Green Threads
• Asynchronous programming

asynchronous 
programing

event loop
&
reactor pattern

Python implementations

• Tornado
• Twisted
• AsyncIO

Tornado Example
python2

class GenAsyncHandler(RequestHandler):
    @gen.coroutine
    def get(self):
        http_client = AsyncHTTPClient()
        response = yield http_client.fetch("http://example.com")
        do_something_with_response(response)
        self.render("template.html")

Tornado Periodic Callback

asyncio

PEP 3156

Coroutine

• generator function may receive parameters
• decorated with @couroutine

Future

• promise of a result or an error 

Task

• Future which runs on a coroutine    

which prevents sequential coding

"yield from" keeps the sequential style

Generators calling other generators, generator delegation.

to remove callbacks

sequential query

async query with callback

def get(self):
    def on_result(result):
        self.write(result)
    make_the_query(callback=on_result)

def get(self)
    result = make_the_query()
    self.write(result)

async query with sub-generator

def get(self)
    result = yield from make_the_query()
    self.write(result)

Future object

• Just a placeholder which going to be a real object when it will be ready
• Like JS promise, defer

f.result()

f.exception()

f.done() f.cancel()

Title Text

Subtitle

@asyncio.coroutine
def print_page(url):
    response = yield from aiohttp.request('GET', url)
    body = yield from response.read_and_close(decode=True)
    print(body)

loop.run_until_complete(asyncio.wait([print_page('http://example.com/foo'),
                                      print_page('http://example.com/bar')]))

Parallel Tasks

import asyncio

@asyncio.coroutine
def factorial(name, number):
    f = 1
    for i in range(2, number+1):
        print("Task %s: Compute factorial(%s)..." % (name, i))
        yield from asyncio.sleep(1)
        f *= i
    print("Task %s: factorial(%s) = %s" % (name, number, f))

loop = asyncio.get_event_loop()
tasks = [
    asyncio.ensure_future(factorial("A", 2)),
    asyncio.ensure_future(factorial("B", 3)),
    asyncio.ensure_future(factorial("C", 4))]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()

result

compatibility

semi compatible with tornado and twisted

THANKS