Andrés Santos

Master Visual Analytics and Big Data

Senior Software Engineer

Empanada Ambassador

Kamikaze Speaker

It takes some engineering to dive into deep learning

Agenda

  • An overview of Artificial Intelligence (AI)
  • What the heck does a Machine Learning Engineer (MLE) do?
  • Essential expertise
  • Roles in AI
  • The first project
  • Takeaways

What is AI?

"the science of making machines do things that would require intelligence if done by men."

Marvin Minsky

What does it mean?

What’s the Difference Between Artificial Intelligence, Machine Learning and Deep Learning? by Michael Copeland. Published here.

Machine Learning

Demystifying AI: A Practical Guide to Key Terminology by Tobias Zwingmann. Published here.

Deep Learning

Demystifying AI: A Practical Guide to Key Terminology by Tobias Zwingmann. Published here.

Artificial Intelligence Taxonomy by Tobias Zwingmann. Originally published with AI For BI Rocks.

In summary

  • Narrow AI / Weak AI*
  • Strong AI
    • Artificial General Intelligence AGI
    • Artificial Super Intelligence ASI**

When will we achieve it?

"I think that AGI, AI that can do anything a human can do is still decades away, maybe 30 to 50 years, maybe even longer."

Andrew Ng

"LLM are smarter than any of us in certain key dimensions, but much dumber than any of us in other dimensions."

What the heck does a MLE do?

SWE

  • Implementation
    • Predictable
  • Coding
  • Unit Test

MLE

  • Experimentation
    • ​Unpredictable
  • Analysis
  • Evaluation
  • Programming skills
    • Python or R
    • SQL
  • Statistical Analysis
  • Fundamentals of ML
    • Train and Evaluate
  • Data visualization skills
    • D3.js
    • Scimago Graphica

Essential expertise

Libraries

Top Python Machine Learning Libraries by Fathima Zajel. Published here.

Tools

MLOps Is a Mess But That's to be Expected by Mihail Eric. Published here.

Roles in AI

Data Engineering by KVRA Tech. Published here.

Diagram by AroscopOfficial. Published here.

Machine Learning Engineer

How to recruit Machine Learning engineers by Yuma Heymans. Published here.

The first project

PoC Architecture. (Sopra Steria, 2021)

Proof of Concept

PoC Data Engineering. (Sopra Steria, 2021)

Data engineering

Preprocessing

Consolidated data

Model to predict resource allocation in Kubernetes

Feature engineering

Selected Features

Long short-term memory

Model training

Train Predictions - FS. (Author, 2023)

Model training

Train Predictions - RAM. (Author, 2023)

Model evaluation

Median Absolute Error for FS and RAM. (Author, 2023)

Model evaluation

Evolution of the Median Absolute Error. (Author, 2023)

Takeways

  • Data is out there, just pick one dataset and start with basic preprocessing tasks.
  • Evaluate your current knowledge and identify the new skills that you need to develop.
  • Define a realistic timeline for your transition, it will usually take more than just a few months.
  • It will be a challenging but also exciting path, so enjoy it.

Resources

Copy of It takes some engineering to dive into deep learning

By Andrés Santos

Copy of It takes some engineering to dive into deep learning

  • 55