Debug and Profile your Python code

Lecturer

Rositsa Kotseva

Full Stack Developer at HedgeServ Bulgaria

Python Lecturer at the HackBulgaria Academy

Content

Debug
- What? Why? How?
- Python Debug tools
Code Profiling
- What? Why? How?
- Python Profiling Tools
Profile visualization

Why did I choose this topic?

What is debugging?

The process of finding/detecting and fixing existing and potential bugs is called DEBUGGING

Why do we debug our code?

How to debug?

print()

def sum_number_digits(number):
    print(number)
    digits = []
    for ch in str(number):
        print(ch)
        print(type(ch))
        digits.append(ch)
    print(digits)
    return sum(digits)


print(sum_number_digits(45))
print(sum_number_digits('4O'))

Logging

import logging


logging.basicConfig(
    filename='example.log',
    level=logging.DEBUG,
    format='%(asctime)s:%(levelname)s:%(message)s'
    )


def sum_number_digits(number):
    logging.debug(number)
    digits = []
    for ch in str(number):
        logging.debug(ch)
        logging.debug(type(ch))
        digits.append(ch)
    logging.debug(digits)
    return sum(digits)


print(sum_number_digits(45))
print(sum_number_digits('4O'))

Log result

2021-03-14 22:47:12,091:DEBUG:45
2021-03-14 22:47:12,091:DEBUG:4
2021-03-14 22:47:12,091:DEBUG:<class 'str'>
2021-03-14 22:47:12,091:DEBUG:5
2021-03-14 22:47:12,091:DEBUG:<class 'str'>
2021-03-14 22:47:12,091:DEBUG:['4', '5']

Log & print based libraries

Loguru - https://github.com/Delgan/loguru
structlog - https://pypi.org/project/structlog/
colorlog - https://pypi.org/project/colorlog/
IceCream - https://github.com/gruns/icecream
PySnooper - https://github.com/cool-RR/PySnooper/

logger.add("out.log", backtrace=True, diagnose=True)

def func(a, b):
    return a / b

def nested(c):
    try:
        func(5, c)
    except ZeroDivisionError:
        logger.exception("What?!")

nested(0)

Loguru example

2018-07-17 01:38:43.975 | ERROR    | __main__:nested:10 - What?!
Traceback (most recent call last):

  File "test.py", line 12, in <module>
    nested(0)
    └ <function nested at 0x7f5c755322f0>

> File "test.py", line 8, in nested
    func(5, c)
    │       └ 0
    └ <function func at 0x7f5c79fc2e18>

  File "test.py", line 4, in func
    return a / b
           │   └ 0
           └ 5

ZeroDivisionError: division by zero

Debuggers

Debugger is a program that assists in the detection and correction of errors in computer programs.

Python Debuggers

pdb - https://docs.python.org/3/library/pdb.html

```
pdb++ - https://github.com/pdbpp/pdbpp
```

IPython pdb - https://github.com/gotcha/ipdb

hunter - https://github.com/ionelmc/python-hunter

pdb commands

python -m pdb.py <file_name>.py [args]
import pdb; pdb.set_trace()
breakpoint()
help [command]

What is Profiling?

A set of statistics that describes how often and for how long various parts of the program executed

Why do we profile our code?

How to profile?

Python Profiling tools

time/timeit - https://docs.python.org/3/library/timeit.html
cProfile - https://docs.python.org/3/library/profile.html
PyInstrument - https://github.com/joerick/pyinstrument
PySpy - https://github.com/benfred/py-spy
Yappi - https://github.com/sumerc/yappi/

Profiling Statistics

```
Total number of calls
```
```
primitive calls
```
```
ncalls
```
```
tottime
```
```
percall
```
```
cumtime
```
```
percall #2
```
```
filename:lineno
```

Profiling example

Link to code example and profiling file

Profile Visualization

SnakeViz
pyprof2calltree
gprof2dot

SnakeViz example

Memory Profiler

Official Documentation

Debug and profile Django

Django Debug ToolBar
django-cprofile-middleware
django-profiler
Django Silk

Django Debug Toolbar

Django Silk

Tips & Tricks

& useful links

PDB debugging with examples

PDB Commands Cheatsheet

Profiling example

Profiling in Django

Python Memory Profilers

Debug and Profile your Python code

Lecturer

Rositsa Kotseva

Content

Debug

What? Why? How?

Python Debug tools

Code Profiling

What? Why? How?

Python Profiling Tools

Profile visualization

Why did I choose this topic?

What is debugging?

The process of finding/detecting and fixing existing and potential bugs is called DEBUGGING

Why do we debug our code?

How to debug?

print()

Logging

Log result

Log & print based libraries

Debuggers

Debugger is a program that assists in the detection and correction of errors in computer programs.

Python Debuggers

pdb commands

What is Profiling?

A set of statistics that describes how often and for how long various parts of the program executed

Why do we profile our code?

How to profile?

Python Profiling tools

Profiling Statistics

Profiling example

Profile Visualization

SnakeViz

pyprof2calltree

gprof2dot

Memory Profiler

Debug and profile Django

Django Debug ToolBar

django-cprofile-middleware

django-profiler

Django Silk

Tips & Tricks

Thank you!

Contacts

Serverless WebSockets with AWS Lambda and API Gateway

Next event