Nawfal Tachfine

Data Scientist

Typical Data Science Workflow

Then what?

Enter Industrialization...

The Data Dependency Problem

Design Patterns

• Think about your production use-case as early as possible.

• Your data pre-processing pipelines must be deterministic and reproduce the same conditions as in training.
  • make your transformations robust against abnormal values
  • variable order matters (cf. scikit-learn)
• Log as much as you can.
• Write clean code: your models will evolve (re-training, new features, ... etc)
• Keep things simple: stability trumps complexity.

Building a Prediction API

• Web app framework: Flask
  • ​simple,
  • lightweight,
  • powerful

This approach is  based on serialization and can be used for models trained with librairies other than scikit-learn.

@app.route('/api/v1.0/aballone', methods=['POST'])

def index():
    query = request.get_json()['inputs']
    data = prepare(query)
    output = model.predict(data)
    return jsonify({'outputs': output})

if __name__ == '__main__':
    app.run(host='0.0.0.0')

Toolbox

Building a Prediction API

• I/O: JSON
  • ​universal support
• Virtualization: Docker
  • no more dependency problems,
  • run (almost) anywhere
• Deployment: AWS Elastic Beanstalk
  • ​(very) easy to use,
  • (relatively) cheap,
  • provides auto-scaling and load-balancing.

Toolbox

Demo Time!