How we can use AI to enhance Umbraco sites?

Steve Temple

Technical Director

About me

Technical Director @ Gibe

Will be using AI and Machine Learning interchangeably

Approaching Machine Learning as a developer

I am not a data scientist

Have been playing with Azure ML since it came out

First experiments were predicting horse racing results

Point of the tale

No finished out of the box solutions

Intention is to spark ideas and inspire packages

There is a massive learning curve to some of these

But for some solutions the hard work has been done an it's up to us to figure out how to use it

Machine Learning tools

On the shoulders of giants


Services available from amongst others:

Microsoft Azure, AWS,

Google, IBM



What can you use Machine Learning for?



Anomaly detection

How can we use these to solve problems in Umbraco sites?


Categorising & tagging media or content

Problem to solve

How can we use AI to improve Umbraco's media section?

  1. No standard location for media
  2. Doesn't scale to 10,000s of images 
  3. Hard to find the exact image you want


The Media Tree

Does a tree make sense for Media?

Has some downsides:

Needs more information on each image/video/file

Editors would have to tag/categorise all media including existing media

Could we search instead?

Tagging of media

Use Azure Vision API to auto tag/categorise/detect features in images



Great demo

But rubbish IRL

Custom Vision API

We can train our own model with our own tags which we want to detect




What other APIs

could we use?

Focal point?

Video analysis?

Image cropper?

Visual search?

What about content?

Auto tagging / subject identification

Automated translation

Optical Character Recognition (OCR)


Recommending other content

Problem to solve

How can we use AI to recommend content on a site?

Azure Machine Learning Studio

Azure Machine Learning Service

Replaces Azure ML Studio

Can use VS Code or Jupyter Notebooks

Azure Databricks

Very powerful

Very complicated!

Works with Azure ML Services


Recommendations Solution

Prebuilt solution template that deploys Azure resources pre-configured for performing recommendations


Needs usage information and catalog information to train the model



Recommendations solution

Needs to be supplied with 2 models to create recommendations



How users are using the site



Product or content information

Configuring and using the API 

REST API with Swagger UI

  1. Create a trained model
  2. Make requests to the API for recommendations



Suggesting media to editors

You might also like / related content

Estimated time to read



How about?

Anomaly Detection

Making sites more secure and easier for the users

Problem to solve

Can we use AI to make user/member logins simpler while being more secure


Could we serve different user or member login experiences based on probability of the user being legitimate

Factors we might consider

IP Address




Browser / cookie recognised

Previous logins

Recent failed attempts


Score the attempt

> 98% 

Auto login?

Remember me, etc

75% +

Regular login

 < 75%

Show Recaptcha

Require 2FA

This all sounds super complex

How to get started

So much potential

Get a free azure account & experiment

There are multiple competitions/challenges all come with brilliant data sets to get started with

Steep learning curve, but look at sample solutions and work from there



Thank you

Enhancing Umbraco with AI

By Steve Temple

Enhancing Umbraco with AI

  • 607
Loading comments...

More from Steve Temple