Tensorflow.js

Alvin Chan

Full Stack Developer

Data scientist

What's the big deal?

  • AI in Browser
  • Do it in Javascript \0/

A neuron

input_1

input_2

input_3

output

param_1

param_2

param_3

+

+

=

output

input_1 * param_1    input_2 * param_2    input_3 * param_3

'Rectangle' neuron

length

breadth

brightness

perimeter

x0

x2

x2

length

breadth

Deep learning

input_1

input_2

input_3

output

Types of neural layer

  • Dense Layer
  • Convolutional Layer
  • Recurrent Layer
  • Many more..

Dense layers

Dense Layer 1

Dense Layer 2

Convolutional layers

input_1

input_2

input_3

output_1

input_4

input_5

step 1

Convolutional layers

input_1

input_2

input_3

output_1

input_4

input_5

output_2

Step 2

Convolutional layers

input_1

input_2

input_3

input_4

input_5

output_3

step 3

output_1

output_2

Why GPU?

Parallel computations

Deep learning for images

32 px

32 px

Source: http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture6.pdf

32 px

32 px

Convolutional layers

input_1

input_2

input_3

output_1

input_4

input_5

Deep layers

Source: http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture6.pdf

What's Tensorflow?

  • Open-source Framework
  • Uses CPU or GPU (or TPU)
  • Build, Train & Predict with Deep Learning

What about Tensorflow.JS?

  • Lower latency
  • Uses CPU or GPU (with WebGL)
  • Build, Train & Predict with Deep Learning

Demo

https://storage.googleapis.com/tfjs-examples/webcam-transfer-learning/dist/index.html

Stages

  1. Building

  2. Training

  3. Prediction

Building AI model

[224, 224, 3]

[7, 7, 256]

[4]

[12544]

flatten

dense

layer

mobilenet

trained model

Building AI model

import * as tf from '@tensorflow/tfjs';

async function loadMobilenet() {
  const mobilenet = await tf.loadModel(
      'https://storage.googleapis.com/tfjs-models/tfjs/mobilenet_v1_0.25_224/model.json');

  // Return a model that outputs an internal activation.
  const layer = mobilenet.getLayer('conv_pw_13_relu');
  return tf.model({inputs: mobilenet.inputs, outputs: layer.output});
}

mobilenet = loadMobilenet();

Building AI model

import * as tf from '@tensorflow/tfjs';

model = tf.sequential({
  layers: [
    tf.layers.flatten({inputShape: [7, 7, 256]}),
    // Layer 1
    tf.layers.dense({
      units: ui.getDenseUnits(),
      activation: 'relu',
      kernelInitializer: 'varianceScaling',
      useBias: true
    }),
    // Layer 2. The number of units of the last layer should correspond
    // to the number of classes we want to predict.
    tf.layers.dense({
      units: NUM_CLASSES,
      kernelInitializer: 'varianceScaling',
      useBias: false,
      activation: 'softmax'
    })
  ]
});

Training AI

[N, 224, 224, 3]

[N, 7, 7, 256]

[N, 4]

[N, 12544]

flatten

dense

layer

mobilenet

trained model

Training AI

const optimizer = tf.train.adam(ui.getLearningRate());

model.compile({optimizer: optimizer, loss: 'categoricalCrossentropy'});

const batchSize =
    Math.floor(controllerDataset.xs.shape[0] * ui.getBatchSizeFraction());

// Train the model!
model.fit(controllerDataset.xs, controllerDataset.ys, {
  batchSize,
  epochs: ui.getEpochs(),
  callbacks: {
    onBatchEnd: async (batch, logs) => {
      ui.trainStatus('Loss: ' + logs.loss.toFixed(5));
      await tf.nextFrame();
    }
  }
});

Prediction

[224, 224, 3]

[7, 7, 256]

[4]

mobilenet

trained model

argmax

LEFT

Prediction

const predictedClass = tf.tidy(() => {

  const img = webcam.capture();

  const activation = mobilenet.predict(img);

  const predictions = model.predict(activation);

  return predictions.as1D().argMax();
});

const classId = (await predictedClass.data())[0];

Resources

  • Tensorflow.js tutorial
    • https://js.tensorflow.org/tutorials/webcam-transfer-learning.html
  • Machine Learning Courses
    • https://developers.google.com/machine-learning/crash-course/ml-intro
    • https://www.coursera.org/learn/machine-learning
    • http://course.fast.ai/

Cheers!

Alvin Chan

https://github.com/cheeseprata/

twitter: @a1vinchan

Slides @

https://slides.com/alvinchan/tensorflow-js

Tensorflow.js

By Alvin Chan

Tensorflow.js

talk.js 20th June 2018

  • 960