Andrés Santos
Master Visual Analytics and Big Data
Senior Software Engineer
Empanada Ambassador
Kamikaze Speaker
It takes some engineering to dive into deep learning
Agenda
- An overview of Artificial Intelligence (AI)
- What the heck does a Machine Learning Engineer (MLE) do?
- Essential expertise
- Roles in AI
- The first project
- Takeaways
What is AI?
"the science of making machines do things that would require intelligence if done by men."
Marvin Minsky
What does it mean?
What’s the Difference Between Artificial Intelligence, Machine Learning and Deep Learning? by Michael Copeland. Published here.
Machine Learning
Demystifying AI: A Practical Guide to Key Terminology by Tobias Zwingmann. Published here.
Deep Learning
Demystifying AI: A Practical Guide to Key Terminology by Tobias Zwingmann. Published here.
Artificial Intelligence Taxonomy by Tobias Zwingmann. Originally published with AI For BI Rocks.
In summary
- Narrow AI / Weak AI*
- Strong AI
- Artificial General Intelligence AGI
- Artificial Super Intelligence ASI**
When will we achieve it?
"I think that AGI, AI that can do anything a human can do is still decades away, maybe 30 to 50 years, maybe even longer."
Andrew Ng
"LLM are smarter than any of us in certain key dimensions, but much dumber than any of us in other dimensions."
What the heck does a MLE do?
SWE
- Implementation
- Predictable
- Coding
- Unit Test
MLE
-
Experimentation
- Unpredictable
- Analysis
- Evaluation
- Programming skills
- Python or R
- SQL
- Statistical Analysis
- Fundamentals of ML
- Train and Evaluate
- Data visualization skills
- D3.js
- Scimago Graphica
Essential expertise
Libraries
Top Python Machine Learning Libraries by Fathima Zajel. Published here.
Tools
MLOps Is a Mess But That's to be Expected by Mihail Eric. Published here.
Roles in AI
Data Engineering by KVRA Tech. Published here.
Diagram by AroscopOfficial. Published here.
Machine Learning Engineer
How to recruit Machine Learning engineers by Yuma Heymans. Published here.
The first project
PoC Architecture. (Sopra Steria, 2021)
Proof of Concept
PoC Data Engineering. (Sopra Steria, 2021)
Data engineering
Preprocessing
Consolidated data
Model to predict resource allocation in Kubernetes
Feature engineering
Selected Features
Long short-term memory
Model training
Train Predictions - FS. (Author, 2023)
Model training
Train Predictions - RAM. (Author, 2023)
Model evaluation
Median Absolute Error for FS and RAM. (Author, 2023)
Model evaluation
Evolution of the Median Absolute Error. (Author, 2023)
Takeways
- Data is out there, just pick one dataset and start with basic preprocessing tasks.
- Evaluate your current knowledge and identify the new skills that you need to develop.
- Define a realistic timeline for your transition, it will usually take more than just a few months.
- It will be a challenging but also exciting path, so enjoy it.