Enhancing the Umbraco experience using machine learning
Steve Temple
Technical Director
About me
Technical Director and co-owner of Gibe
Umbraco MVP
Umbraco Certified Master
Organiser of Umbraco Spark conference
Will be using AI and Machine Learning interchangeably
Approaching Machine Learning as a developer
I am not a data scientist
Have been playing with Azure ML since it came out
First experiments were predicting horse racing results
UmbracoSpark Developer Conference
Bristol
6th March 2020
Point of the talk
No finished out of the box solutions
Intention is to spark ideas and inspire packages
There is a massive learning curve to some of these
But for some solutions the hard work has been done an it's up to us to figure out how to use it
Machine Learning tools
On the shoulders of giants
Services available from amongst others:
Microsoft Azure, AWS,
Google, IBM
What can you use Machine Learning for?
Classification
Prediction
Anomaly detection
How can we use these to solve problems in Umbraco sites?
Classification
Categorising & tagging media or content
Problem to solve
How can we use AI to improve Umbraco's media section?
- No standard location for media
- Doesn't scale to 10,000s of images
- Hard to find the exact image you want
The Media Tree
Does a tree make sense for Media?
Has some downsides:
Needs more information on each image/video/file
Editors would have to tag/categorise all media including existing media
Could we search instead?
Tagging of media
Use Azure Vision API to auto tag/categorise/detect features in images
DEMO
Great demo
But rubbish IRL
Custom Vision API
We can train our own model with our own tags which we want to detect
DEMO
What other APIs
could we use?
Focal point?
Video analysis?
Image cropper?
Visual search?
What about content?
Auto tagging / subject identification
Automated translation
https://our.umbraco.com/packages/backoffice-extensions/translation-manager/
Optical Character Recognition (OCR)
https://our.umbraco.com/packages/backoffice-extensions/ocr-content-app/
Prediction
Recommending other content
Problem to solve
How can we use AI to recommend content on a site?
Azure Machine Learning Studio
Azure Machine Learning Service
Replaces Azure ML Studio
Can use VS Code or Jupyter Notebooks
Azure Databricks
Very powerful
Very complicated!
Works with Azure ML Services
Solutions/Examples
https://gallery.azure.ai
Recommendations Solution
Prebuilt solution template that deploys Azure resources pre-configured for performing recommendations
Needs usage information and catalog information to train the model
https://github.com/Microsoft/Product-Recommendations
Recommendations solution
Needs to be supplied with 2 models to create recommendations
Usage
Models
How users are using the site
Catalog
Models
Product or content information
Configuring and using the API
REST API with Swagger UI
- Create a trained model
- Make requests to the API for recommendations
DEMO
Suggesting media to editors
You might also like / related content
Estimated time to read
Personalisation
...
How about?
Anomaly Detection
Making sites more secure and easier for the users
Problem to solve
Can we use AI to make user/member logins simpler while being more secure
Login
Could we serve different user or member login experiences based on probability of the user being legitimate
Factors we might consider
IP Address
Country
Region
Time
Browser / cookie recognised
Previous logins
Recent failed attempts
Privileges
Score the attempt
> 98%
Auto login?
Remember me, etc
75% +
Regular login
< 75%
Show Recaptcha
Require 2FA
This all sounds super complex
How to get started
So much potential
Get a free azure account & experiment
There are multiple competitions/challenges all come with brilliant data sets to get started with
Steep learning curve, but look at sample solutions and work from there
Resources
https://studio.azureml.net/
https://azure.microsoft.com/en-gb/services/cognitive-services/
https://gallery.azure.ai/
https://github.com/Microsoft/Product-Recommendations
https://docs.microsoft.com/en-us/azure/machine-learning/studio/algorithm-choice