History and future of Tapan
Tapan v1
Database and Reports layers
Lot's of SQL queries, queries replica, consistency :(
Very few
typesafety
Report results are cached, date range is a key cache
No persistence, no zero downtime restarts
Exposes CSVs
1
Tapan v2 (now)
Database, View, Reports and API
No SQL (almost), queries replica, consistency :(
Database layer is mixed with Views :(
Typesafety, let is crash
Reports are built based on both Database and Views layers
Only final Reports are cached, no intermediate caching, but filters and aggregations work (not everywhere)
No persistence, no zero downtimes / upgrades
No way to query from outside (ad-hoc, tableau)
1
Tapan v3
Separate Views and Database
Typesafety, skip and report
Do ETL
Cache and persist intermediate steps
Open access to Views and Reports to outside
1
Tapan v3. Database
Database, Views, Reports and API
Database layer is flat and has simple types (CSV)
No database connection, database is exported
export-20160223-0100/platform/tasks.csv
export-20160223-0100/analytics/ga.csv
Export can be consistent
Expected to be faster
needs to be measured
Tapan v3. Views
Views are built by other Views and Database
Typesafety through validation
don't crash, just log and skip
Consistency! Because export is consistent
Cacheable!
Queriable/Exportable to/from Tableau, R, whatever
Use typed data frames for aggregations and joins
Tapan v3. Reports
Built by other Reports and Views
Cacheable as before
Fault-tolerant
skip and report inconsistent things
return last built report if it doesn't work today
Consistent
cross-checks should always match
Queriable/Exportable to/from Tableau, R, whatever
Tapan v3. API
Built by other Reports and Views
Cacheable as before
Fault-tolerant
skip and report inconsistent things
return last built report if it doesn't work today
Queriable/Exportable to/from Tableau, R, whatever
Tapan v3. Deployment
Zero-downtime upgrade/downgrade/restart
Build views and reports, and then switch
v7.0.0/views/companies
v7.0.0/reports/daily_revenue
v7.1.0/views/companies
v7.1.0/reports/daily_revenue
Don't upgrade if we failed to build new reports
Tapan v3. Productivity
Don't recompute things between restarts
Reports and views are stored locally as CSV
grep
diff (endpoint-compare)
...
Computations can be executed remotely
Don't spend your CPU and battery power
Further yolo-mode enchancements
Made with Slides.com