Upkar Lidder
Upkar Lidder is a Full Stack Developer and Data Wrangler with a decade of development experience in a variety of roles. He can be seen speaking at various conferences and participating in local tech groups and meetups.
Introduction to Machine Learning on IBM Cloud
Upkar Lidder
IBM Developer
http://bit.ly/svldeveloperday
> ulidder@us.ibm.com > @lidderupk > upkar.dev
ML Hype
@lidderupk
IBM Developer
ML on IBM Cloud - Cognitive Services
@lidderupk
IBM Developer
Natural Language Processing
Visual Recognition
ML on IBM Cloud - Watson Assistant
@lidderupk
IBM Developer
ML on IBM Cloud - Watson Assistant
@lidderupk
IBM Developer
ML on IBM Cloud - NLU [Greta Thunberg]
@lidderupk
IBM Developer
ML on IBM Cloud - Sentiment Analysis
@lidderupk
IBM Developer
ML on IBM Cloud - Discovery
@lidderupk
IBM Developer
ML on IBM Cloud - Discovery
@lidderupk
IBM Developer
ML on IBM Cloud - Visual Recognition
@lidderupk
IBM Developer
Watson Studio
ML Lifecycle
@lidderupk
IBM Developer
ML Map
@lidderupk
IBM Developer
Unsupervised Learning
Supervised Learning
- Classification
- Regression
- Clustering
- Dimensionality Reduction
Watson Studio & Watson Machine Learning
@lidderupk
IBM Developer
IBM Watson Studio
@lidderupk
IBM Developer
IBM Watson Studio - project based development platform
@lidderupk
IBM Developer
ML on IBM Cloud - Guided ML
@lidderupk
IBM Developer
To help simplify an AI lifecycle management, AutoAI automates:
- Data preparation
- Model development
- Feature engineering
- Hyper parameter optimization
AutoAI
@lidderupk
IBM Developer
@lidderupk
IBM Developer
AutoAI - Behind the scenes
Data pre-processing
- analyze, clean and prepare raw data for ML
- automatically detects and categorizes features based on data type
- missing value imputation
- feature encoding
- feature scaling
Automated model selection
- test and rank candidate estimators
- select the best performing estimator with the ranking choice made by the user
Automated Feature Engineering
- transform raw data into combination of features that best fit the model
Hyperparameter Optimization
- refine the best performing model
@lidderupk
IBM Developer
AutoAI - Supported Estimators
@lidderupk
IBM Developer
AutoAI - Example
[Moro et al., 2014] S. Moro, P. Cortez and P. Rita. A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems, Elsevier, 62:22-31, June 2014
@lidderupk
IBM Developer
AutoAI - Data Profiling
[Moro et al., 2014] S. Moro, P. Cortez and P. Rita. A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems, Elsevier, 62:22-31, June 2014
@lidderupk
IBM Developer
AutoAI - Data Visualization
[Moro et al., 2014] S. Moro, P. Cortez and P. Rita. A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems, Elsevier, 62:22-31, June 2014
@lidderupk
IBM Developer
AutoAI - Data Transformation
[Moro et al., 2014] S. Moro, P. Cortez and P. Rita. A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems, Elsevier, 62:22-31, June 2014
@lidderupk
IBM Developer
AutoAI - Modeling [complete dataset]
[Moro et al., 2014] S. Moro, P. Cortez and P. Rita. A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems, Elsevier, 62:22-31, June 2014
@lidderupk
IBM Developer
AutoAI - Modeling [small dataset]
[Moro et al., 2014] S. Moro, P. Cortez and P. Rita. A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems, Elsevier, 62:22-31, June 2014
@lidderupk
IBM Developer
AutoAI - Modeling [experiments]
I want to build my own!
😤
😤
Watson Machine Learning
WML - Supported Frameworks as of 06.21.19
@lidderupk
IBM Developer
IBM Watson Machine Learning
@lidderupk
IBM Developer
IBM Watson Machine Learning Client
@lidderupk
IBM Developer
http://wml-api-pyclient.mybluemix.net/index.html
WML - create scikit-learn linear regression model
@lidderupk
IBM Developer
from sklearn.linear_model import LinearRegression
from sklearn.datasets import load_boston
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error, r2_score
boston = load_boston()
X_train, X_test, y_train, y_test = train_test_split(boston.data, boston.target)
# Create a new Linear Regression Model
LR_model = LinearRegression()
# Train the model
LR_model.fit(X_train, y_train)
# store actual and predited data to draw chart
predicted = LR_model.predict(X_test)
actual = y_test
# The coefficients
print('Coefficients: \n', LR_model.coef_)
# The mean squared error
print("Mean squared error: %.2f"
% mean_squared_error(actual, predicted))
# Explained variance score: 1 is perfect prediction
print('Variance score: %.2f' % r2_score(actual, predicted))
Output
👉🏽
WML - evaluation metrics
@lidderupk
IBM Developer
AutoAI
Notebook
WML - get Machine Learning service credentials
@lidderupk
IBM Developer
WML - get Machine Learning service credentials
@lidderupk
IBM Developer
WML - save scikit-learn linear regression model
@lidderupk
IBM Developer
# we will use WML to work with IBM Machine Learning Service
from watson_machine_learning_client import WatsonMachineLearningAPIClient
# Grab your credentials from the Watson Service section in Watson Studio or IBM Cloud Dashboard
wml_credentials = {
}
# Instantiate WatsonMachineLearningAPIClient
from watson_machine_learning_client import WatsonMachineLearningAPIClient
client = WatsonMachineLearningAPIClient( wml_credentials )
# store the model
published_model = client.repository.store_model(model=LR_model,
meta_props={'name':'upkar-housing-linear-reg'},
training_data=X_train, training_target=y_train)WML - deploy scikit-learn linear regression model
@lidderupk
IBM Developer
import json
# grab the model from IBM Cloud
published_model_uid = client.repository.get_model_uid(published_model)
# create a new deployment for the model
model_deployed = client.deployments.create(published_model_uid, "Deployment of scikit model")
#get the scoring endpoint
scoring_endpoint = client.deployments.get_scoring_url(model_deployed)
print(scoring_endpoint)
#use the scoring endpoint to predict house median price some test data
scoring_payload = {"values": [list(X_test[0]), list(X_test[1])]}
predictions = client.deployments.score(scoring_endpoint, scoring_payload)
print(json.dumps(predictions, indent=2))WML - deploy scikit-learn linear regression model
@lidderupk
IBM Developer
IBM Code Patterns
@lidderupk
IBM Developer
Model Asset Exchange
@lidderupk
IBM Developer
Data Asset Exchange
@lidderupk
IBM Developer
Thank you
Let's chat !
@lidderupk
IBM Developer
Upkar Lidder, IBM
@lidderupk
https://github.com/lidderupk/
ulidder@us.ibm.com
SVL Developer Day
By Upkar Lidder
