Power up your work with compiling and profiling
GitHub repo:
Hello I am Cheuk
-
Open-Source contributor
-
Organisers of community events
-
ex EPS board member, PSF fellow
- Looking for my next role
Outline
- Presentation (30 mins)
- Q&A and Break (15 mins)
- Do Ex.1 and Ex.2 together (60 mins)
- Q&A and Break (15 mins)
- Do Ex. 3.1 and Ex. 3.2 on your own (90 mins)
Who has this problem - the code is too slow?
Profiling and Compling may have you
What is profiling?
Profiling (in software eng.)
- Investigate software behaviour
- When it is being executed
- Dynamic analysis
- the tool that does that is called a profiler
What is compiling?
Compiling (in software eng.)
- convert a program into a machine code or lower-level form
- in which the program can be executed
- it takes time to compile codes
- but lower-level codes are generally faster to execute
How can it help me?
It helps because
- Analyse what makes my program slow
- Avoid slow code
- Or make slow code faster
- Compiling makes repeated code faster
What is Numba
Numba is an open source JIT compiler that translates a subset of Python and NumPy code into fast machine code.
What is JIT complier
- just-in-time compiler
- Python does not compile programs
- It takes time to complie
- If there are functions that got used more than once, the 2nd time running a complied version will be faster
- Perfect for loops
What works with Numba
- Numpy - what it is designed for
- Loopings - things that is done more than once
Different modes
- object mode
- handles all values as Python objects
- usually not faster
- unless loop component can be "jit"
- nopython mode
- does not access the Python C API
- high performamce
Different modes
- @jit
- automatically fall back to object mode
- can compile code that failed in nopython mode
- @njit
- short hand for @jit(nopython=True)
- garentee high performamce
- if not supported it will failed to compile
Support CUDA GPUs
- Numba supports CUDA GPU programming
- Compiling a restricted subset of Python code
- Allow CUDA kernels to have direct access to NumPy arrays
- See PyData Amsterdam 2019 talk recording