Serverless Machine Learning

Jowanza Joseph

@jowanza

Agenda

• Problem
• About Me
• Data Pipelines
• Serverless / Event Driven 
• Apache OpenWhisk
• Project Architecture
• Tips
• Questions

The Problem Space

Ergonomics

The Problem Space

• How do we deploy new machine learning models quickly?
• Have minimal impact on the system?
• Maintain custom environments and runtime?
• Maintain ergonomic simplicity?

About Me

• Software Engineer @ One Click Retail
• Data Engineering
• Scala / Spark
• DC/OS / Kubernetes
• Author

Data Pipelines

Serverless

Benefits

• Cheap
• Quick to implement and isolate
• Event Driven
• Easy to take advantage of

Costs

• Cloud specific implementations
• Hard to test / version
• Language specific

Supported APIs

• Javascript
• Python
• PHP
• Swift
• Java
• DOCKER

More Stuff

• Sequences
• Packages
• Composer

Architecture

• Load Balancing
• That's pretty much it

Controller

• Akka
• Service Mapping
• Job Queuing

• Which code to run
• History of code that has been run
• Logs
• Authentication details

• Service Discovery
• That's it

• Pub/Sub
• Exactly Once Delivery
• Retention
• Distributed

Invoker

• Docker Container
• Isolation
• Control

APIs

• Whisk CLI
• API Gateway
  • Supports Versioning
  • Supports packaging

Whisk CLI

Whisk API Gateway

https://{APIHOST}/api/v1/namespaces/{namespace}/actions
https://{APIHOST}/api/v1/namespaces/{namespace}/triggers
https://{APIHOST}/api/v1/namespaces/{namespace}/rules
https://{APIHOST}/api/v1/namespaces/{namespace}/packages
https://{APIHOST}/api/v1/namespaces/{namespace}/activations

Custom Whisk Actions

How It Works

1. Train Models In Spark
2. Bundle
3. Run via MLeap Runtime
4. Profit

How It Works

Benefits

• Type Safety
• Simplified Execution Context
• Similar APIs to Spark

Really Nice Things

• Logging
• Performance Monitoring
• Shared Actions
• Scaling

$$$ Comparison

Trade Offs

• Excellent Isolation
• Scalability
• Customizability

• OpenWhisk is a little hard
• Some of the semantics are hard to grasp
• MLeap Requires extra effort

A Word On Images

Base OpenWhisk Image

More Info

• Slides are on Sched
• Open sourcing soon
• @jowanza
• www.jowanza.com
• www.apachesparkfieldbook.com
• Thanks