Machine Learning

❤ JS

Machine Learning

Gives computers the ability to learn without being explicitly programmed, Arthur Samuel 1959

label - prediction

feature - input

Model

LINEAR REGRESSION

training data

source: https://bit.ly/2RUhH43

Machine learning

in the browser

Why ?

Wider access and distribution
Privacy
Distributed computing
No installation required

Demo

Step 1 : Load and convert the model

from keras.applications.mobilenet import MobileNet

# Load Mobilenet model with imagenet weights
model = MobileNet(weights='imagenet')

# Save model on file system
model.save('mobilenet.h5')

$> tensorflowjs_converter --input_format keras \
path/to/mobilenet.h5 path/to/tfjs-target-dir

 {
    "modelTopology": {
      ...
    },

    "weightsManifest": {
      ...
    }
 }

// Load model in tensorflow.js
const model = await tf.loadModel('model.json');

Backup

Conversion

Loading

Step 2 : Collect the training data

tf.tidy(() => {
   const tensor = tf.fromPixels(stream);
   ... 
   this.classifier.addTrainingData(tensor, label);
});

Step 3 : Create and compile the final model

// Create model
this.model = tf.sequential({
      layers: [
        tf.layers.flatten({ inputShape: [7, 7, 256] }),
        tf.layers.dense({
          units: 100,
          useBias: true,
          activation: 'relu',
          ...
        }),
        tf.layers.dense({
          useBias: true,
          activation: 'softmax',
          units: this.classes,
          ...
        }),
      ],
});

// Compile model
this.model.compile({ optimizer: tf.train.adam(0.0001), loss: 'categoricalCrossentropy' });

Step 4 : Training

// Train model
this.model.fit(dataset.features, dataset.labels, {
   batchSize,
   epochs: 20,
   callbacks: {
     onBatchEnd: async (batch, logs) => {
       console.log('Loss:' + logs.loss.toFixed(5));
       await tf.nextFrame();
     },
   },
});

Step 5 : Prediction

async predict(callback) {
    while (this.isPredicting) {
      const predictions = tf.tidy(() => {
        const image = this.camera.takePicture();
        const activation = this.trainedModel.predict(image);
        const predictions = this.model.predict(activation);
        const tensor = predictions.as1D();

        return tensor.argMax();
      });
      const label = (await predictions.data())[0];
      console.log(`Predicted label: ${label}`);
      predictions.dispose();
      callback(label);
      await tf.nextFrame();
    }
}