Tensorflow.js
Alvin Chan
Full Stack Developer
Data scientist
What's the big deal?
- AI in Browser
- Do it in Javascript \0/
A neuron
input_1
input_2
input_3
output
param_1
param_2
param_3
+
+
=
output
input_1 * param_1 input_2 * param_2 input_3 * param_3
'Rectangle' neuron
length
breadth
brightness
perimeter
x0
x2
x2
length
breadth
Deep learning
input_1
input_2
input_3
output
Types of neural layer
- Dense Layer
- Convolutional Layer
- Recurrent Layer
- Many more..
Dense layers
Dense Layer 1
Dense Layer 2
Convolutional layers
input_1
input_2
input_3
output_1
input_4
input_5
step 1
Convolutional layers
input_1
input_2
input_3
output_1
input_4
input_5
output_2
Step 2
Convolutional layers
input_1
input_2
input_3
input_4
input_5
output_3
step 3
output_1
output_2
Why GPU?
Parallel computations
Deep learning for images
32 px
32 px
Source: http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture6.pdf
32 px
32 px
Convolutional layers
input_1
input_2
input_3
output_1
input_4
input_5
Deep layers
Source: http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture6.pdf
What's Tensorflow?
- Open-source Framework
- Uses CPU or GPU (or TPU)
- Build, Train & Predict with Deep Learning
What about Tensorflow.JS?
- Lower latency
- Uses CPU or GPU (with WebGL)
- Build, Train & Predict with Deep Learning
Demo
https://storage.googleapis.com/tfjs-examples/webcam-transfer-learning/dist/index.html
Stages
-
Building
-
Training
-
Prediction
Building AI model
[224, 224, 3]
[7, 7, 256]
[4]
[12544]
flatten
dense
layer
mobilenet
trained model
Building AI model
import * as tf from '@tensorflow/tfjs';
async function loadMobilenet() {
const mobilenet = await tf.loadModel(
'https://storage.googleapis.com/tfjs-models/tfjs/mobilenet_v1_0.25_224/model.json');
// Return a model that outputs an internal activation.
const layer = mobilenet.getLayer('conv_pw_13_relu');
return tf.model({inputs: mobilenet.inputs, outputs: layer.output});
}
mobilenet = loadMobilenet();
Building AI model
import * as tf from '@tensorflow/tfjs';
model = tf.sequential({
layers: [
tf.layers.flatten({inputShape: [7, 7, 256]}),
// Layer 1
tf.layers.dense({
units: ui.getDenseUnits(),
activation: 'relu',
kernelInitializer: 'varianceScaling',
useBias: true
}),
// Layer 2. The number of units of the last layer should correspond
// to the number of classes we want to predict.
tf.layers.dense({
units: NUM_CLASSES,
kernelInitializer: 'varianceScaling',
useBias: false,
activation: 'softmax'
})
]
});
Training AI
[N, 224, 224, 3]
[N, 7, 7, 256]
[N, 4]
[N, 12544]
flatten
dense
layer
mobilenet
trained model
Training AI
const optimizer = tf.train.adam(ui.getLearningRate());
model.compile({optimizer: optimizer, loss: 'categoricalCrossentropy'});
const batchSize =
Math.floor(controllerDataset.xs.shape[0] * ui.getBatchSizeFraction());
// Train the model!
model.fit(controllerDataset.xs, controllerDataset.ys, {
batchSize,
epochs: ui.getEpochs(),
callbacks: {
onBatchEnd: async (batch, logs) => {
ui.trainStatus('Loss: ' + logs.loss.toFixed(5));
await tf.nextFrame();
}
}
});
Prediction
[224, 224, 3]
[7, 7, 256]
[4]
mobilenet
trained model
argmax
LEFT
Prediction
const predictedClass = tf.tidy(() => {
const img = webcam.capture();
const activation = mobilenet.predict(img);
const predictions = model.predict(activation);
return predictions.as1D().argMax();
});
const classId = (await predictedClass.data())[0];
Resources
- Tensorflow.js tutorial
- https://js.tensorflow.org/tutorials/webcam-transfer-learning.html
- Machine Learning Courses
- https://developers.google.com/machine-learning/crash-course/ml-intro
- https://www.coursera.org/learn/machine-learning
- http://course.fast.ai/
Cheers!
Alvin Chan
https://github.com/cheeseprata/
twitter: @a1vinchan
Slides @
https://slides.com/alvinchan/tensorflow-js
Tensorflow.js
By Alvin Chan
Tensorflow.js
talk.js 20th June 2018
- 966