Performance in Python applications
@woakas
Ubidots
Gustavo Angulo
February 2020
woakas
Coffee / 5-7
13 years
2 sons
CTO / Co-founder
Ubidots
Ubidots
+ 40k Developers
+ 30 M per day
The problem
- Slowness
- Downs
- I don't know what's happening
- Bad experience for users
The source
Bad code
Database
Physical
Language
Human
Bad code
from flask import Flask, escape, request
import time
app = Flask(__name__)
@app.route("/add")
def add():
## Adding some time
time.sleep(1)
a = float(request.args.get("a", 0))
b = float(request.args.get("b", 0))
return "{}".format(a + b)
WRK
# lua
request = function()
wrk.method = "GET"
return wrk.format(
"GET", '/add?a=' .. math.random(0, 100) .. '&b=' .. math.random(0, 100)
)
end
# bash
$ wrk -s add_random.lua -c 50 -t 1 -d 10 http://localhost:5000
# Flask
FLASK_APP=slow_code.py flask run --reload --without-threads
Big O notation
https://www.geeksforgeeks.org/analysis-algorithms-big-o-analysis/
Bad code
from flask import Flask, escape, request
app = Flask(__name__)
@app.route("/add/cube")
def add():
t = 0
mn, mx = (0, 100)
a = int(request.args.get("size", 0))
for i in range(a):
for j in range(a):
for k in range(a):
t += i + j
return "{}".format(t)
Database
Database
- Limits
- # queries per second
- Joins
- Request / Queries
Django
devices = Device.objects.filter(owner__username='woakas')
for d in devices:
print(d.owner)
# 66 queries
devices = Device.objects.filter(owner__username='woakas').prefetch_related('owner')
for d in devices:
print(d.owner)
# 2 queries
Redis
➜ ~ redis-benchmark -t get
====== GET ======
100000 requests completed in 1.36 seconds
50 parallel clients
3 bytes payload
keep alive: 1
95.94% <= 1 milliseconds
99.97% <= 2 milliseconds
100.00% <= 2 milliseconds
73583.52 requests per second
Physical
Limits
Limits
fs.file-max=65535
net.core.somaxconn = 65536
net.ipv4.tcp_max_tw_buckets = 1440000
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_max_syn_backlog = 65536
Limits
Network
CPU
RAM
Disc
IO
Language
Sanic / async
from sanic import Sanic
from sanic.response import json
import time
app = Sanic()
@app.route("/")
async def test(request):
# time.sleep(1)
return json({"hello": "world"})
if __name__ == "__main__":
app.run(host="0.0.0.0", port=8000)
Languages
Human
Developer
Thank you
@woakas
woakas@ubidots.com
pycon2020
By Gustavo Angulo
pycon2020
- 810