Doing the things you should never do

Dr. Ólafur Helgason

VP of Engineering @ OZ


UTMessan 2015

6 feb 2015

What is OZ?

Video channel platform for creators

Upload and manage videos

Create live schedule

VOD and geo-awareness

Subscription and billing

Support major clients

First back-end incarnation

  • Micro-services
    • Many (very) small services
    • Well defined interfaces - REST/MQ
  • node.js + a little Erlang
  • NoSQL - CouchDB
  • RabbitMQ backbone
  • IaaS hosting

Frustration grows

  • Continuous deployment fragile
  • Lack of automation and staging env
  • IaaS means DYI
  • Chef a headache
  • Major dev, minor ops

Lesson: Automate everything from beginning

Lesson: Use Service Discovery

Lesson: Don't use IaaS unless you have OPS

Separating relational data

  • Data split between services
  • How is your data queried?
  • Multiple service queries per req
  • Solution:
    • Precompute in-memory indexes
    • Lot's of caching
    • Premature sharding

Lesson: Select service boundaries carefully

Lesson: Cache consistency is hard!


  • Document/json store
  • Quick and easy - no ORM
  • Limited querying
    • Views index documents by json properties
    • Expensive to update - downtime!
  • De-normalizing data
  • Lost momentum
  • Limited support & toolset

Lesson: Use relational DBs for relational data


  • Early adopters
  • Javascript runtime using Google V8
  • Asynchronous non-blocking I/O
    • Excellent performance
  • Active community and good package system
  • Asynchronous programming challenging
    • Callback hell
    • Promises make life better

Lesson: node.js kicks ass

Sense of ownership

  • Original architects left
  • Lack of team ownership
  • Important for startup

Lesson: team must have sense of ownership

Major requirements change

  • Re-focused business model
  • Simplified many things
  • Dropping features
  • This is the chance!

Second incarnation

  • SOA but not micro
  • node.js, PostgreSQL, RabbitMQ
  • All-in on PaaS
    • Heroku
    • Significantly simplifies operations
    • Optimize for speed
  • SaaS fanatics
  • Focus on OZ!

Lesson: Use PaaS if you can afford it

(and if it makes sense for you business)


Did it take more time than planned?


Is it released?


Is rewriting the biggest mistake a software company can do?

Ask me in april


Pssst - we are hiring

UTMessan 2015

By olafur

UTMessan 2015

  • 1,334