Python - GIL
The good the bad and the ugly
woakas
Coffee / 5-7
14 years
2 sons
CTO / Co-founder
Global Interpreter Lock
Byte-code One by One
CPython
Pros
- Prevents race conditions.
- Increased speed of single-threaded programs.
- No deadlocks
- Easy to test
Cons
- Threaded programs run slower
- Threading is not concurrent
The good
How does it work?
David Beazley, http://dabeaz.com/
How does it work?
David Beazley, http://dabeaz.com/
Single thread
import time
def sum():
total = 0
for i in range(50_000_000):
total += 1
return total
def main():
start = time.time()
sum()
end = time.time()
total_time = int((end - start) * 1000)
print(f"Time in milliseconds - {total_time}")
main()
Two threads
import time
from threading import Thread
def fn(n):
total = 0
for i in range(n):
total += 1
return total
def main():
count = 50_000_000
t1 = Thread(target=fn, args=(count // 2,))
t2 = Thread(target=fn, args=(count // 2,))
start = time.time()
t1.start()
t2.start()
t1.join()
t2.join()
end = time.time()
total_time = int((end - start) * 1000)
print(f"Time in milliseconds - {total_time}")
main()
Multiprocessing
import time
import multiprocessing
def fn(n):
total = 0
for i in range(n):
total += 1
return total
def main():
count = 50_000_000
p1 = multiprocessing.Process(target=fn, args=(count // 2,))
p2 = multiprocessing.Process(target=fn, args=(count // 2,))
start = time.time()
p1.start()
p2.start()
p1.join()
p2.join()
end = time.time()
total_time = int((end - start) * 1000)
print(f"Time in milliseconds - {total_time}")
main()
Results
?????
Single thread
Avg: 3.123 ms
Two threads
Avg: 2.945 ms
The bad?????
Single thread
Avg: 3.123 ms
Two threads
Avg: 2.945 ms
Tracing the Python GIL
pip install tabulate per4m viztracer
Tracing the Python GIL
# Single Thread
perf record -e sched:sched_switch -e sched:sched_process_fork -e 'sched:sched_wak*' \
-k CLOCK_MONOTONIC --call-graph dwarf -- viztracer -o viztracer-single_thread_gil.json --ignore_frozen -m single_thread
perf script --no-inline | per4m perf2trace sched -o perf-single_thread.json
viztracer --log_multiprocess --combine perf-single_thread.json viztracer-single_thread_gil.json -o data-single_thread.html
# Two Threads
perf record -e sched:sched_switch -e sched:sched_process_fork -e 'sched:sched_wak*' \
-k CLOCK_MONOTONIC --call-graph dwarf -- viztracer -o viztracer-two_threads-gil.json --ignore_frozen -m two_threads
perf script --no-inline | per4m perf2trace sched -o perf-two_threads.json
viztracer --combine perf-two_threads.json viztracer-two_threads-gil.json -o data-two_threads.html
# Multiprocessing
viztracer --log_multiprocess multi_processing.py -o data-multi_processing.html
Tracing the Python GIL
Tracing the Python GIL
Tracing the Python GIL
Tracing the Python GIL
Tracing the Python GIL
The ugly
The ugly
GIL
Know it, understand it and use it
It's not magic, it's just code
References
@woakas
GIL The good the bad and the ugly
By Gustavo Angulo
GIL The good the bad and the ugly
- 690