Renato Cordeiro Ferreira
Scientific Programmer @ JADS | PhD Candidate @ USP | Co-founder & Coordinator @CodeLab
A Metrics-Oriented Architectural Model
to Characterize Complexity on
Machine Learning-Enabled Systems
https://renatocf.xyz/phd-quali-live
2024
Renato Cordeiro Ferreira
Supervisor: Prof. Dr. Alfredo Goldman
Co-Supervisor: Prof. Dr. Damian Tamburri
IME-USP
Our goal is to
use metrics to identify
where complexity emerges
in the software architecture
of ML-enabled systems
Research Questions
What are the measurable dimensions of complexity
in the architecture of ML-enabled systems?
How can complexity metrics be operationalized
over the architecture of ML-enabled systems?
RQ1
RQ2
RQ3
How can complexity metrics be used to aid
the development, operation, and evolution
of real-world ML-enabled systems?
Research Questions
How can complexity metrics be used to choose between architecture proposals for an ML-enabled system?
How can complexity metrics be used to identify refactoring opportunities in an ML-enabled systems?
RQ3.1
RQ3.2
RQ3
How can complexity metrics be used to aid
the development, operation, and evolution
of real-world ML-enabled systems?
Reference Architecture
for ML-Enabled Systems
The SPIRA
ML-Enabled System
Data Collection App
Scientific Initiation 2021
Francisco Wernke
Streaming
Prediction Server
+ Client API / App
Capstone Project 2022
Vitor Tamae
Highly Availability
with Kubernetes
Capstone Project 2023
Vitor Guidi
Redesign Continuous Training Subsystem
Capstone Project 2023
Daniel Lawand
CI/CD/CD4ML on
Training Pipeline
Capstone Project 2024
Lucas Quaresma
+ Roberto Bolgheroni
Research Methodology
State of the art
about metrics
regarding
ML-Enabled
Systems
Industry- and academic-based case study on complexity metrics for
ML-Enabled Systems
Mixed-method approach to assess the impact of complexity in development tasks for
ML-Enabled Systems
Threats to Validity
Construct Validity
The study can measure what it proposed to measure
Internal Validity
The study can produce the results it reported
External Validity
The study can be generalized to other contexts
Conclusion
The study can be replicated by other researchers
C
I
E
R
I
C
E
R
E
C
Data from
knowledge bases
Researcher
Ontology
Design
Choice of
Case Studies
Selection of
Metrics
Constructed
Examples
Sampling
Population of
Practitioners
I
C
E
R
E
C
Established
Publication
Databases
Guidelines
from
Literature
Inclusion
Criteria for
Case Studies
Exploratory +
Confirmatory
Case Studies
Sampling
Population of
Practitioners
Constructed
Examples
Work Plan
A Metrics-Oriented Architectural Model
to Characterize Complexity on
Machine Learning-Enabled Systems
https://renatocf.xyz/phd-quali-live
2024
Renato Cordeiro Ferreira
Supervisor: Prof. Dr. Alfredo Goldman
Co-Supervisor: Prof. Dr. Damian Tamburri
IME-USP
Committee
is on
Evaluation
[PhD Qualifying Exam] A Metrics-Oriented Architectural Model to Characterize Complexity on ML-Enabled Systems
By Renato Cordeiro Ferreira
