PyTorch
What is PyTorch ?
(obviously its something with python)
"Scientific Computing Package"
?
?
?
?
?
?
?
?
?
"GPU Utilisation"
Usual Code Execution
+ Performance
Is it worth it ?
Absolutely !
But why ?
Latency Optimized
Bandwidth Optimized
Deep learning does lots of calculations
512x512 Image
= 262144 Pixel
262144 Pixel with RGB
= 786.432 Values
Layer 1: 2000 Weights
= 1.572.864.000 Calculations
262.144x2000
Input: 512x512 Image
.........
PyTorch is Imperative not Symbolic
a = 1
b = 3
c = a + b
d = c * 3
e = c - 5
print(d)Imperative
"e" is calculated
a = 1
b = 3
c = a + b
d = c * 3
e = c - 5
f = run(d)
print(f)Symbolic
"e" is not calculated
Compution Graphs
Or
Coding using Code
b = w1 * a
c = w2 * a
d = (w3 * b) + (w4 * c)
L = f(d)
Why use Graphs ?
-
Chain Rule
-
Parallelisation
-
Distribution
-
Compilation
-
Portability
Main Selling Points
- Switch between Imperative and Symbolic
- Distributed Training
- Best Python Integration
- Eager Community
- Similarity to NumPy
Neural Style Transfer
not-so-live Demo
MNIST Classification
a-tad-bit-more-live Demo
Benchmark
i7 4720HQ:
i7 8700K:
1080 TI:
960 M:
4 Cores 2.6 GHZ 6MB
6 Cores 3.7 GHZ 12MB
3584 Cores 1.7 GHZ 11GB
640 Cores 1.2 GHZ 4GB
697.9s
382.3s
51.4s
9.15s
PyTorch
By Lukas Kurz
