Priyank Trivedi
Backend Engineer at Zenatix
Introducing event driven or asynchronous programming model.
The essential aspect of asynchronus/event-driven systems - you tell x that when y occurs, call z.
Processing not allowed before current execution is done
e.g. blocking IO operations
def sum(list_of_nums):
sum = 0
for (i=0; i<len(list_of_nums); i++)
s += list_of_nums[i]
print "Sum is", sum
Each task is performed one at a time
#!/usr/bin/python
import thread
import time
# Define a function for the thread
def print_time( threadName, delay):
count = 0
while count < 5:
time.sleep(delay)
count += 1
print "%s: %s" % ( threadName, time.ctime(time.time()) )
# Create two threads as follows
try:
thread.start_new_thread( print_time, ("Thread-1", 2, ) )
thread.start_new_thread( print_time, ("Thread-2", 4, ) )
except:
print "Error: unable to start thread"
while 1:
pass
Processing allowed before current execution is done
e.g. non-blocking IO operations
Event Loops
reactor.listenTCP(8080, someFactory)
button.connect('clicked', someCallback)
Once it's inside your callable function, you have to let go
Event
Loop
Your code
Event
Loop
Your code
Need some billion rows data from MySQL
Event
Loop
Your code
Need some billion rows data from MySQL
To give users a better experiences
e.g.
No more "XXX is not responding" dialogue popped up
Show loading indicator
To utilize resources
Here the method call wouldn't return you the values.
But
an object which will tell when your data is.
x = y() doesn't work anymore.
x = y() doesn't work anymore.
def hello():
print 'Hello from the reactor loop!'
print 'I\'m in an infinite loop here.'
from twisted.internet import reactor
reactor.callWhenRunning(hello)
print 'Starting the reactor.'
reactor.run()
class Countdown(object):
counter = 5
def count(self):
if self.counter == 0:
reactor.stop()
else:
print self.counter, '...'
self.counter -= 1
reactor.callLater(1, self.count)
from twisted.internet import reactor
reactor.callWhenRunning(Countdown().count)
print 'Start!'
reactor.run()
print 'Stop!'
from twisted.internet.defer import Deferred
def got_poem(res):
print 'Your poem is served:'
print res
def poem_failed(err):
print 'No poetry for you.'
d = Deferred()
# add a callback/errback pair to the chain
d.addCallbacks(got_poem, poem_failed)
# fire the chain with a normal result
d.callback('This poem is short.')
print "Finished"
from twisted.internet import reactor, defer
def getDummyData(x):
"""
This function is a dummy which simulates a delayed result and
returns a Deferred which will fire with that result. Don't try too
hard to understand this.
"""
d = defer.Deferred()
# simulate a delayed result by asking the reactor to fire the
# Deferred in 2 seconds time with the result x * 3
reactor.callLater(2, d.callback, x * 3)
return d
def printData(d):
"""
Data handling function to be added as a callback: handles the
data by printing the result
"""
print d
d = getDummyData(3)
d.addCallback(printData)
# manually set up the end of the process by asking the reactor to
# stop itself in 4 seconds time
reactor.callLater(4, reactor.stop)
# start up the Twisted reactor (event loop handler) manually
reactor.run()
import sys
from twisted.internet.defer import Deferred
def got_poem(poem):
print poem
def poem_failed(err):
print >>sys.stderr, 'poem download failed'
print >>sys.stderr, 'I am terribly sorry'
print >>sys.stderr, 'try again later?'
def poem_done(_):
from twisted.internet import reactor
reactor.stop()
d = Deferred()
d.addCallbacks(got_poem, poem_failed)
d.addBoth(poem_done)
from twisted.internet import reactor
reactor.callWhenRunning(d.callback, 'Another short poem.')
reactor.run()
Event
Loop
Your code
Need some billion rows data from MySQL
Asynchronous Programming to your rescue
"""
We have expanded our row processor into its own function as
well as adding an error handler to the twisted Deferred.
"""
def gotaRequest(request, response):
df = mySqlConn.query("SELECT * FROM Pydelhi")
df.addCallback(_someData, response)
df.addErrback(_someError, response)
def _someData(rows, response):
response.render(rows)
response.finish()
def _someError(e, response):
response.render("Error! %s" % e)
response.finish()
d = Deferred()
d.addCallbacks(_someData, _someError)
from twisted.internet import reactor
reactor.callWhenRunning(d.callback, request)
reactor.run()
Why Bother ?
As compared to the synchronous model,
There are a large number of tasks so there is likely always at least one task that can make progress.
The tasks perform lots of I/O, causing a synchronous program to waste lots of time blocking when other tasks could be running.
The tasks are largely independent from one another so there is little need for inter-taskcommunication (and thus for one task to wait upon another).