History and future of Tapan

 

Tapan v1

  • Database and Reports layers
  • Lot's of SQL queries, queries replica, consistency :(
  • Very few typesafety
  • Report results are cached, date range is a key cache
  • No persistence, no zero downtime restarts
  • Exposes CSVs
1

Tapan v2 (now)

  • Database, View, Reports and API
  • No SQL (almost), queries replica, consistency :(
  • Database layer is mixed with Views :(
  • Typesafety, let is crash
  • Reports are built based on both Database and Views layers
  • Only final Reports are cached, no intermediate caching, but filters and aggregations work (not everywhere)
  • No persistence, no zero downtimes / upgrades
  • No way to query from outside (ad-hoc, tableau)
1

Tapan v3

 
  • Separate Views and Database
  • Typesafety, skip and report
  • Do ETL
  • Cache and persist intermediate steps
  • Open access to Views and Reports to outside
1

Tapan v3. Database

 
  • Database, Views, Reports and API
  • Database layer is flat and has simple types (CSV)
  • No database connection, database is exported
    • export-20160223-0100/platform/tasks.csv
    • export-20160223-0100/analytics/ga.csv
  • Export can be consistent
  • Expected to be faster
    • needs to be measured
 

Tapan v3. Views

 
  • Views are built by other Views and Database
  • Typesafety through validation
    • don't crash, just log and skip
  • Consistency! Because export is consistent
  • Cacheable!
  • Queriable/Exportable to/from Tableau, R, whatever
  • Use typed data frames for aggregations and joins
 

Tapan v3. Reports

 
  • Built by other Reports and Views
  • Cacheable as before
  • Fault-tolerant
    • skip and report inconsistent things
    • return last built report if it doesn't work today
  • Consistent
    • cross-checks should always match
  • Queriable/Exportable to/from Tableau, R, whatever
 

Tapan v3. API

 
  • Built by other Reports and Views
  • Cacheable as before
  • Fault-tolerant
    • skip and report inconsistent things
    • return last built report if it doesn't work today
  • Queriable/Exportable to/from Tableau, R, whatever
 

Tapan v3. Deployment

 
  • Zero-downtime upgrade/downgrade/restart
  • Build views and reports, and then switch
    • v7.0.0/views/companies
    • v7.0.0/reports/daily_revenue
    • v7.1.0/views/companies
    • v7.1.0/reports/daily_revenue
  • Don't upgrade if we failed to build new reports
 

Tapan v3. Productivity

 
  • Don't recompute things between restarts
  • Reports and views are stored locally as CSV
    • grep
    • diff (endpoint-compare)
    • ...
  • Computations can be executed remotely
    • Don't spend your CPU and battery power
  • Further yolo-mode enchancements
 

tapan

By Gleb Kanterov

tapan

  • 150
Loading comments...

More from Gleb Kanterov