Introduction
Multilayer Perceptrons
Implementations in Python
Image from MNIST handwritten digit dataset
A zero that is difficult to distinguish from a six algorithmically
We don't know what program to write because we don't know how it's done by our brains
Let's define our model to be a function
A linear perceptron
Feed-forward NNs:
Logistic function / Sigmoid function: 0~1
Hyperbolic tangent (Tanh) function: -1~1
ReLU (rectified linear unit) function
Cost Function:
Gradient Descent:
The gradient descent algorithm is:
repeat until convergence:
There are various optimization techniques that utilize adaptive learning rates to automate the process of selecting learning rates
total number of layers in the network
number of units (exclude the bias unit) in layer l
number of output units/classes
the output
Major limitation:
Python, R, Julia, and Scala
Lua
Python
Python
Java
Python
Select the correct binaries to install
# Ubuntu/Linux 64-bit, CPU only, Python 2.7
$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.11.0-cp27-none-linux_x86_64.whl
# Ubuntu/Linux 64-bit, GPU enabled, Python 2.7
# Requires CUDA toolkit 8.0 and CuDNN v5. For other versions, see "Install from sources" below.
$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.11.0-cp27-none-linux_x86_64.whl
# Mac OS X, CPU only, Python 2.7:
$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-0.11.0-py2-none-any.whl
# Mac OS X, GPU enabled, Python 2.7:
$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow-0.11.0-py2-none-any.whl
Install TensorFlow
# Python 2 $ pip install --upgrade $TF_BINARY_URL
$ pip install keras
$150 credits = ~230 hours usage of g2.2xlarge