Doing the things you should never do
Dr. Ólafur Helgason
VP of Engineering @ OZ
UTMessan 2015
6 feb 2015
What is OZ?
Video channel platform for creators
Upload and manage videos
Create live schedule
VOD and geo-awareness
Subscription and billing
Support major clients
First back-end incarnation
- Micro-services
- Many (very) small services
- Well defined interfaces - REST/MQ
- node.js + a little Erlang
- NoSQL - CouchDB
- RabbitMQ backbone
- IaaS hosting
Frustration grows
- Continuous deployment fragile
- Lack of automation and staging env
- IaaS means DYI
- Chef a headache
- Major dev, minor ops
Lesson: Automate everything from beginning
Lesson: Use Service Discovery
Lesson: Don't use IaaS unless you have OPS
Separating relational data
- Data split between services
- How is your data queried?
- Multiple service queries per req
- Solution:
- Precompute in-memory indexes
- Lot's of caching
- Premature sharding
Lesson: Select service boundaries carefully
Lesson: Cache consistency is hard!
CouchDB
- Document/json store
- Quick and easy - no ORM
- Limited querying
- Views index documents by json properties
- Expensive to update - downtime!
- De-normalizing data
- Lost momentum
- Limited support & toolset
Lesson: Use relational DBs for relational data
node.js
- Early adopters
- Javascript runtime using Google V8
- Asynchronous non-blocking I/O
- Excellent performance
- Active community and good package system
- Asynchronous programming challenging
- Callback hell
- Promises make life better
Lesson: node.js kicks ass
Sense of ownership
- Original architects left
- Lack of team ownership
- Important for startup
Lesson: team must have sense of ownership
Major requirements change
- Re-focused business model
- Simplified many things
- Dropping features
- This is the chance!
Second incarnation
- SOA but not micro
- node.js, PostgreSQL, RabbitMQ
- All-in on PaaS
- Heroku
- Significantly simplifies operations
- Optimize for speed
- SaaS fanatics
- Focus on OZ!
Lesson: Use PaaS if you can afford it
(and if it makes sense for you business)
Conclusions
Did it take more time than planned?
yes
Is it released?
No
Is rewriting the biggest mistake a software company can do?
Ask me in april
Thanks!
Pssst - we are hiring
UTMessan 2015
By olafur
UTMessan 2015
- 1,416