Doing the things you should never do
Dr. Ólafur Helgason
VP of Engineering @ OZ
UTMessan 2015
6 feb 2015
What is OZ?
Video channel platform for creators
Upload and manage videos
Create live schedule
VOD and geo-awareness
Subscription and billing
Support major clients
First back-end incarnation
- Micro-services
- Many (very) small services
- Well defined interfaces - REST/MQ
- node.js + a little Erlang
- NoSQL - CouchDB
- RabbitMQ backbone
- IaaS hosting
Frustration grows
- Continuous deployment fragile
- Lack of automation and staging env
- IaaS means DYI
- Chef a headache
- Major dev, minor ops
Lesson: Automate everything from beginning
Lesson: Use Service Discovery
Lesson: Don't use IaaS unless you have OPS
Separating relational data
- Data split between services
- How is your data queried?
- Multiple service queries per req
- Solution:
- Precompute in-memory indexes
- Lot's of caching
- Premature sharding
Lesson: Select service boundaries carefully
Lesson: Cache consistency is hard!
CouchDB
- Document/json store
- Quick and easy - no ORM
- Limited querying
- Views index documents by json properties
- Expensive to update - downtime!
- De-normalizing data
- Lost momentum
- Limited support & toolset
Lesson: Use relational DBs for relational data
node.js
- Early adopters
- Javascript runtime using Google V8
- Asynchronous non-blocking I/O
- Excellent performance
- Active community and good package system
- Asynchronous programming challenging
- Callback hell
- Promises make life better
Lesson: node.js kicks ass
Sense of ownership
- Original architects left
- Lack of team ownership
- Important for startup
Lesson: team must have sense of ownership
Major requirements change
- Re-focused business model
- Simplified many things
- Dropping features
- This is the chance!
Second incarnation
- SOA but not micro
- node.js, PostgreSQL, RabbitMQ
- All-in on PaaS
- Heroku
- Significantly simplifies operations
- Optimize for speed
- SaaS fanatics
- Focus on OZ!
Lesson: Use PaaS if you can afford it
(and if it makes sense for you business)