*Flow Pipelines
| Needs ? | Airflow | MLFlow |
|---|---|---|
| Features | Robust scalable scheduler | ML artifacts / models / runs tracking |
| Easy Infrastructure | Fully managed on Astronomer | Stateless Standalone Service (Plugs in RDS / S3) |
| Easy Contributions | Describing Pipelines as python actions | Writing experiments as python functions / Web UI |
| Easy Usage | Web UI showing logs / failed tasks / click and run | Simple but powerful API allowing better collaborations |
+
[1] Data preparation
[2] Data annotation
[3] Run jobs
[4] Record runs & store models / artifacts
[5] Notification