Introduction to Machine Learning on IBM Cloud

Upkar Lidder

IBM Developer

http://bit.ly/svldeveloperday

> ulidder@us.ibm.com
> @lidderupk
> upkar.dev

ML Hype

@lidderupk
IBM Developer

ML on IBM Cloud - Cognitive Services

@lidderupk
IBM Developer

Natural Language Processing

Visual Recognition

ML on IBM Cloud - Watson Assistant

@lidderupk
IBM Developer

ML on IBM Cloud - Watson Assistant

@lidderupk
IBM Developer

ML on IBM Cloud - NLU [Greta Thunberg]

@lidderupk
IBM Developer

ML on IBM Cloud - Sentiment Analysis

@lidderupk
IBM Developer

ML on IBM Cloud - Discovery

@lidderupk
IBM Developer

ML on IBM Cloud - Discovery

@lidderupk
IBM Developer

ML on IBM Cloud - Visual Recognition

@lidderupk
IBM Developer

Watson Studio

ML Lifecycle

@lidderupk
IBM Developer

ML Map

@lidderupk
IBM Developer

Unsupervised Learning

Supervised Learning

  • Classification
  • Regression
  • Clustering
  • Dimensionality Reduction

Watson Studio & Watson Machine Learning

@lidderupk
IBM Developer

IBM Watson Studio 

@lidderupk
IBM Developer

IBM Watson Studio - project based development platform 

@lidderupk
IBM Developer

ML on IBM Cloud - Guided ML

@lidderupk
IBM Developer

To help simplify an AI lifecycle management, AutoAI automates:

  • Data preparation
  • Model development
  • Feature engineering
  • Hyper parameter optimization

AutoAI

@lidderupk
IBM Developer
@lidderupk
IBM Developer

AutoAI - Behind the scenes

Data pre-processing

  • analyze, clean and prepare raw data for ML
  • automatically detects and categorizes features based on data type
  • missing value imputation
  • feature encoding
  • feature scaling

Automated model selection

  • test and rank candidate estimators
  • select the best performing estimator with the ranking choice made by the user

Automated Feature Engineering

  • transform raw data into combination of features that best fit the model

Hyperparameter Optimization

  • refine the best performing model
@lidderupk
IBM Developer

AutoAI - Supported Estimators

@lidderupk
IBM Developer

AutoAI - Example

[Moro et al., 2014] S. Moro, P. Cortez and P. Rita. A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems, Elsevier, 62:22-31, June 2014

@lidderupk
IBM Developer

AutoAI - Data Profiling

[Moro et al., 2014] S. Moro, P. Cortez and P. Rita. A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems, Elsevier, 62:22-31, June 2014

@lidderupk
IBM Developer

AutoAI - Data Visualization

[Moro et al., 2014] S. Moro, P. Cortez and P. Rita. A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems, Elsevier, 62:22-31, June 2014

@lidderupk
IBM Developer

AutoAI - Data Transformation

[Moro et al., 2014] S. Moro, P. Cortez and P. Rita. A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems, Elsevier, 62:22-31, June 2014

@lidderupk
IBM Developer

AutoAI - Modeling [complete dataset]

[Moro et al., 2014] S. Moro, P. Cortez and P. Rita. A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems, Elsevier, 62:22-31, June 2014

@lidderupk
IBM Developer

AutoAI - Modeling [small dataset]

[Moro et al., 2014] S. Moro, P. Cortez and P. Rita. A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems, Elsevier, 62:22-31, June 2014

@lidderupk
IBM Developer

AutoAI - Modeling [experiments]

I want to build my own!

😤

😤

Watson Machine Learning

WML - Supported Frameworks as of 06.21.19

@lidderupk
IBM Developer

IBM Watson Machine Learning 

@lidderupk
IBM Developer

IBM Watson Machine Learning Client 

@lidderupk
IBM Developer

http://wml-api-pyclient.mybluemix.net/index.html

WML - create scikit-learn linear regression model

@lidderupk
IBM Developer
from sklearn.linear_model import LinearRegression
from sklearn.datasets import load_boston
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error, r2_score

boston = load_boston()
X_train, X_test, y_train, y_test = train_test_split(boston.data, boston.target)

# Create a new Linear Regression Model
LR_model = LinearRegression()

# Train the model
LR_model.fit(X_train, y_train)

# store actual and predited data to draw chart
predicted = LR_model.predict(X_test)
actual = y_test


# The coefficients
print('Coefficients: \n', LR_model.coef_)
# The mean squared error
print("Mean squared error: %.2f"
      % mean_squared_error(actual, predicted))
# Explained variance score: 1 is perfect prediction
print('Variance score: %.2f' % r2_score(actual, predicted))
Output

👉🏽

WML - evaluation metrics

@lidderupk
IBM Developer

AutoAI

Notebook

WML - get Machine Learning service credentials

@lidderupk
IBM Developer

WML - get Machine Learning service credentials

@lidderupk
IBM Developer

WML - save scikit-learn linear regression model

@lidderupk
IBM Developer
# we will use WML to work with IBM Machine Learning Service
from watson_machine_learning_client import WatsonMachineLearningAPIClient

# Grab your credentials from the Watson Service section in Watson Studio or IBM Cloud Dashboard
wml_credentials = {
}

# Instantiate WatsonMachineLearningAPIClient
from watson_machine_learning_client import WatsonMachineLearningAPIClient
client = WatsonMachineLearningAPIClient( wml_credentials )

# store the model
published_model = client.repository.store_model(model=LR_model,
                                                meta_props={'name':'upkar-housing-linear-reg'},
                                                training_data=X_train, training_target=y_train)

WML - deploy scikit-learn linear regression model

@lidderupk
IBM Developer
import json

# grab the model from IBM Cloud
published_model_uid = client.repository.get_model_uid(published_model)

# create a new deployment for the model
model_deployed = client.deployments.create(published_model_uid, "Deployment of scikit model")

#get the scoring endpoint
scoring_endpoint = client.deployments.get_scoring_url(model_deployed)
print(scoring_endpoint)

#use the scoring endpoint to predict house median price some test data
scoring_payload = {"values": [list(X_test[0]), list(X_test[1])]}
predictions = client.deployments.score(scoring_endpoint, scoring_payload)
print(json.dumps(predictions, indent=2))

WML - deploy scikit-learn linear regression model

@lidderupk
IBM Developer

IBM Code Patterns

@lidderupk
IBM Developer

Model Asset Exchange

@lidderupk
IBM Developer

Data Asset Exchange

@lidderupk
IBM Developer

Thank you

 

Let's chat !

@lidderupk
IBM Developer

Upkar Lidder, IBM

 

@lidderupk

https://github.com/lidderupk/

ulidder@us.ibm.com

SVL Developer Day

By Upkar Lidder

SVL Developer Day

  • 591