DevOpsCon 

Berlin, 1-3 Jun 2015

@jurisicmarko

DevOps

Developers + Operations

Microservices

James Lewis  (Thoughtworks)

Motivation

  • We are building systems that are too big
  • 90% of TCO (total cost of ownership) post launch
  • We have to rewrite entire ecosystems every few years → does not make CFOs happy
  • Tightly coupled, golden hammer, single point of failure systems
  • Systems should be organized along organizational boundaries (Conway's law)

Why microservices

  • Subdividing systems speeds up releases of new services
  • Different options for scalability, different technologies
  • Different tools and languages for components

http://martinfowler.com/articles/microservices.html

Why now?

  • We understand more about building reliable distributed systems
  • Clooud computing and programmable infrastructure has matured
  • Organizations need to adapt and change quickly to survive
  • We spend too much money on building monoliths
  • Plus - it's cool 

What is a microservice

  • Small unit
  • Should be replacable in 2 weeks (a few days)
  • Netflix - one microservice for every REST verb

Pitfalls

  • Additional effort with integration → solution to test everything in production (blue/green deployment in combination with good reporting tools)
  • Scaling brings more network traffic - not a problem unless you are Google
  • Rippling effect of failure after API change - communication, contract based testing and development
  • We can not avoid complexity, we just push it around
  • We are driven by big companies solving problems that we do not have

Prerequisites

  • Moving away from architecture - big systems handed from ivory tower to developers
  • Now "we just build stuff"
  • Just enough thinking ahead, just enough architecture

Architecture freedom

  • Architecture as a city planning, not designing a house
    • like in SimCity - definition of zones and within zones implementors have freedom
    • common infrastructure available
    • have a rough idea about what you want and defer decisions until you know more
    • decentralized governance, trust our people

Bounded contexts

  • Good service = high cohesion + loose coupling
  • "A specific responsibility enforced by explicit boundaries"
  • "Objects should not be bigger than my head"
  • Each domain/level of abstraction should fit in my head

Service evolution

Testing

  • Balancing test pyramid between testing and rapid remediation
  • Deploy one thing at a time
  • Consumer driven contracts
  • Image based deployment

Monitoring

  • Microservices push the accidental complexity into the infrastructure
  • Response time tracking
  • Log aggregation
  • Track health (Dropwizard)
  • Invariant monitoring (graphite, prometheus, logstash, splunk, kibana, hystrix)

Recovery mechanisms

  • Correlation ids to track requests through the system
  • "Every socket, pipe, transaction ... wil hang" Nygard
  • Circuit breakers - if a service fails, stop calling it for a while
  • Fallback behavior in case of failure
  • Failfast - healthcheck before every request

 

Conclusions

 

  • much easier to reason about
  • much easier to provision
  • fewer side effects

 

  • cost + management overhead
  • organizational problems

Amazon Web services

  • Delivery - make a "change" to production

 

  • things will break
  • embrace and plan for failure
  • divide and conquer
  • limit the blast radius
  • isolate and compartmentalize

Deployment scenarios

  • One box deployment - analyze transactions, CPU, memory
  • Single availability zone
  • Single region / continent
  • Always change your fleet // update one at a time
  • Immutable servers
  • Blue / green deployment - old + new in parallel
  • Load balancer for switching
  • Backup & Restore all the time - one time/week fallback is to slow for tests
  • Commit →CI → Staging → Pre-prod → 1 Box → AZ1
  • You build it →you run it (W. Vogels, Amazon CTO)
  • Chaos monkey
  • Failure will happen, accept it:
    • Limit the impact
    • Practice and anticipate corrective actions
    • A trusted test suite
    • Monitoring and measuring is a basis for everything
    • Auto scaling // experiment

Innovation dank DevOps - Wooga Game Studio

  • 5 mil users / day
  • Devops = a big picture mindset
  • Deliver and maintain something
  • Reality feedback

Innovation dank DevOps - Wooga Game Studio

  • Crossteam communication
  • Rotating jobs
  • No finger pointing
  • Independant teams (team per game)
  • Knowledge exchange
  • Trust

Zalando radical agility

  • Doing it yourself is not the most sensible thing
  • 3 principles
    • Purpose
    • Autonomy
    • Mastery
  • Great judgment comes from experience and experience comes from poor judgment

Keynote - Conway's law

  • 1968 Conway - "Organizations produce systems that mimic the communication structure of those organizations"
  • Loosely coupled organizations make moudlar software

Business design

  • There is nothing so useless as doing effectively something that which should not be done at all
  • OODA Loop : observe - orient - decide - act
  • If a requirement leaves the team it moves 12x slower

DevOps

  • Business design - development- customer support - ops
  • Developers rewarded for change
  • Ops rewarded for stability

 

 

 

DevOps - not only for unicorns

  • IT Revolution Manifesto – 95% of all capital projects depend on IT to get done
  • CALMS: culture, automation, lean, measurement, sharing
  • From idea to production
  • High performing organizations are still deploying 30x more frequently 
  • Enterprise DevOps isn‘t mandatory but neither is survival

DevOps a sharders tale

@ Etsy

  • Lack of surprise
  • Trust
  • Blameless post-mortems (open source morgue tool)
    • push to production on the first day of the job
  • Communication, interdisciplinary teams
  • Automation and tools, CI
  • "If you give an engineer 50 lines of code he will find 10 bugs, if you give him 500 he will say „OK, push it“

Keynote 2

Conclusion

  • There is no silver bullet
  • DevOps and microservices need great organizational effort and multiple iterations:
  • "Even with the best tools, DevOps is just another buzzword if you don't have the right culture."
  • When successfully implemented can lead to great results

References

  • Slides from conference
  • Images from devopsreactions.tumblr.com

Further reading

Thank your for your time

Questions?

Made with Slides.com