Data Pipeline Automation by integrating             Django Signals with Celery

 

Discussion about the problem statement

 

  •  The Type of Data and the workflow i worked upon ?
  • What other problems can be solved using this          architecture.

How to write State wise Django Signals ?

 

  • Write State wise signal function in a better fashion
  • Integrations it with Celery to make it asynchronous.

Diving into the Code

 

- Using Python's getattr() function to call related django Function.

Airflow Introduction

Advs. & Disadvs.

of Airflow

 

  • How Airflow is overkill for our problem statement.
  • Where does it fits the most.

Thank you!!

Questions?

aynsc-workflow-automation

By Mannu Gupta

aynsc-workflow-automation

Presentation for Talk proposal in Pycon India 2019

  • 545