The DevOps handbook
What is devops ?
DevOps is Just "Infrastructure as Code" or Automation.
Myths
Myths
DevOps isn't about automation, just as astronomy isn't about telescopes
Christopher Little
DevOps is incompatible with Information Security & Compliance
Myths
DevOps means Eliminating IT Operations, or "NoOps"
Myths
What is devops then ?
The problem
The wall of Confusion - Andrew Clay Shafer
The problems
Move
Stable
Late
then retry
The problems
High time to market
Deployment Rare & Risky
Increased Complexity & technical dept
More firefighting
The problems
Deployment Rare & Risky
Bad customer service
The problems
Technical dept
More failure
Compensate
Less time for initial project
The downward spiral
For who ?
Every Company is a technological company, regardless of what business they think they are. A bank is just an IT Company with a banking license
Christopher Little
For who ?
In 2013, HSBC had more software developers than Google
For who ?
Dr Vernon Richardson
184 Companies
/w material weakness linked to IT
/w material weakness without link to IT
"Clean firms"
x8
x4
For who ?
It is virtually impossible to make any business decision that doesn't result in at least one IT change
How?
Technical dept
More failure
Compensate
Less time for initial project
The downward spiral
How?
Ship quickly & securly in production like environment
Fast feedback loop
Continual assurance
Discover issues early on (less technical dept)
Every person involved at each step of the development
How?
Monitor everything
How?
Unitary changes
Easier to find & fix issues
Release controlled, predictable & low stress
How?
Keep teams intact instead of changing at each feature
Improve work togerther
More productive
How?
Less waiting time
Feeling more productive
Frequent value delivering to customers
Buisness value of devops
- 30 times more deployments
- Less deployment lead time (200 times faster)
- More reliability
- 60 times higer chance of success rate for a production deployment
- 168 times faster to restore services
- Better organisation performance
- 2 times more likelly to exeed productivity, market share & profitability goals
- Better security outcomes
Puppet lalabs
State of devops report
2013-2016
(25 000 technology profesionals)
Buisness value of devops
- Higer employee job satisfaction
- Lower burnout rate
- 22 times more likely to recommend their company as great place to work
Puppet lalabs
State of devops report
2013-2016
(25 000 technology profesionals)
Buisness value of devops
Amazon
2011 - 7 000 deployments / day
2015 - 130 000 deployments / day
The three ways
First Way: The principle of Flow
The three ways
Second Way: The principle of Feedback
The three ways
Third Way: The principle of Continual learning and experimentation
The first way
The principle of flow
The first way
Typical manifacturing value stream :
Order received
Raw materials are released
Processing
Delivering
The first way
Typical IT value stream :
Business hypothesis
Accept work
Development process
Deployment
Validation
The first way
The first way
%C/A
Percent complete and accurate
The first way
The first way
- Make work visible
- Reduce batch size
- Increase qualituy defects beeing passed downstream
The first way
Increase flow
Reduce lead time for customer & internal requests
increase quality
More agile & able to out-experiment the competition
The first way
Work in progress is visible
Movement is visible
Work in Progress is invisible
Movement is done by a click
1. Make work visible
The first way
1. Make work visible
The first way
1. Make work visible
Work is not done when development completes the implementation of a feature, rather it is done when our application is running successfully on production, deliering value to the customer
The first way
2. Limit work in progress
Daily work is dictated by a production schedule
Disruption is highly visible & expensive
Work is dynamic
Disruption is frequent
The first way
2. Limit work in progress
The first way
2. Limit work in progress
All work must be visible!
The first way
2. Limit work in progress
Controlling queue size (WIP) is an extremly powerful managment tool, as it is one of the few leading indicators of lead time - with most work item, we don't know how long it will take until it is actually completed
Dominic DeGondis
The first way
2. Limit work in progress
Stop starting. Start finishing
David J. Anderson
The first way
3. Reduce batch size
The first way
4. Reduce the number of handsoff
The first way
4. Reduce the number of handsoff
The goal is to eliminate the maximum handsoff by automation or reorganization to allow teams to deliver value themselves which reduce wait time, thus improve flow
The first way
5. Continually identify and elevate our constraint
The first way
5. Continually identify and elevate our constraint
In any value stream, there is always a direction of flow, and there is always one and only one constraint. Any improvement not made at that constraint is an illusion
Dr Goldratt - Beyond the Goal
The first way
5. Continually identify and elevate our constraint
Five Focus Steps:
- Identify the system's constraint
- Decide how to exploit the constraint
- Subordinate everything else to the above decision
- Elevate the system's constraint
- If in the previous steps a constraint has been broken, go back to step 1, but do not allow inertia to cause a system constraint
The first way
5. Continually identify and elevate our constraint
Usuall devops transformation :
- Environment creation
- Code deployment
- Test and setup run
- Overly thight architecture
The first way
6. Elimitate hardship and waste in the value stream
The first way
6. Elimitate hardship and waste in the value stream
- Partially done work
- Extra process
- Extra feature
- Task Switching
- Waiting
- Motion
- Defects
- Non standard manual work
- Heroics
- ...
The second way
The principle of feedback
The second way
The second way
- Create fast, frequent, high-quality information flow throughout our value stream and organisation
- Create feedback & feedforward loops
- Learn from failure instead of punishment and blame
The second way
What is a complex system ?
The second way
Conditions to make a safer work system:
- Complexe work is managed so that problems in design and operations are revealed
- Problems are swarmed and solved, resulting in quick construction of knowledge
- New local knowledge is exploited globally throughout the organization
- Leaders create other leaders who continually grow theses types of capabilities
The second way
1. see problems as they occur
The second way
1. see problems as they occur
The second way
2. swarm and solve problems to build new knowledge
In doing so, we build ever-deeper knowledge about how to manage the system for doing our work, converting inevitable up-front ignorance into knowledge
Dr Spear
The second way
2. swarm and solve problems to build new knowledge
The second way
3. Keep pushing quality closer to the source
The second way
3. Keep pushing quality closer to the source
- Require another team to complete tedious, error-prone and manual tasks that could be easily automated
- Require approval from busy people who are distant from the work (make descision without adequate knowledge of the work and its implications)
- Create large volume of documentation with questionable details that become obsolete shortly after they are written
- Pushing large batch of work to teams and special committees for approval and processing and then waiting for responses
The second way
3. Keep pushing quality closer to the source
It is impossible for a developper to learn anything when someone yells at them for something they broke six months ago - that's why we need to provide feedback to everyone as quickly as possible, in minutes, not months
Gary Gruver
The second way
4. Enable optimizing foir downstream work center
The second way
4. Enable optimizing foir downstream work center
The second way
The principle of feedback
The third way
The principle of continual learning and experimentation
The third way
The third way
1.enabling organizational learning and a safety culture
Incidents will happen. How should we handle them ?
The third way
1.enabling organizational learning and a safety culture
Responses to incidents and accidents that are seen as unjust can impede safety investigation, promote fear rather than mindfulness in people who do safety-critical work, make organizations more bureaucratic rather than more careful, and cultivate professional secrecy, evasion, and self-protection
Dr Sydney Dekker
The third way
1.enabling organizational learning and a safety culture
- Pathological organization
- Bureaucratic organizations
- Generative organizations
The third way
1.enabling organizational learning and a safety culture
The third way
1.enabling organizational learning and a safety culture
By removing blame, you remove fear; by removing fear, you enable honesty; and honesty enables prevention
Bethany Marcri
The third way
2. Institutionalize the improvement of daily work
The third way
2. Institutionalize the improvement of daily work
In the absence of improvement, processes don't stay the same - due to chaos and entropy, processes actually degrade over time
Mike Rother - Toyota Kata
The third way
2. Institutionalize the improvement of daily work
Even more than daily work is the improvement of daily work
Mike Orzen - Lean IT
The third way
2. Institutionalize the improvement of daily work
The third way
2. Institutionalize the improvement of daily work
The third way
2. Institutionalize the improvement of daily work
The third way
2. Institutionalize the improvement of daily work
Paul O'Neill
The third way
2. Institutionalize the improvement of daily work
95%
lower injury rate over ten years
The third way
2. Institutionalize the improvement of daily work
Alcoa gradually stopped working around the difficulties, inconveniences, and impediments they experienced. Coping, fire fighting, and making do were gradually throughout the organization by a dynamic of identifying opportunities for process and product improvement. As those opportunities were identified and the problems were investigated, the pockets of ignorance that they reflected were converted into nuggets of knowledge
Dr Spear
The third way
3. Transform local discoveries into global improvements
The third way
3. Transform local discoveries into global improvements
The third way
3. Transform local discoveries into global improvements
The third way
4. Inject resilience patterns into our daily work
The third way
4. Inject resilience patterns into our daily work
The third way
4. Inject resilience patterns into our daily work
The third way
4. Inject resilience patterns into our daily work
The third way
5. Leaders reinforce a learning culture
Leaders lead by "making all the right decisions"
The third way
5. Leaders reinforce a learning culture
Creating greatness requires both leaders and workers, each of whom are mutually dependent on each other
The third way
5. Leaders reinforce a learning culture
Coaching Kata
Mike Rotcher
The third way
5. Leaders reinforce a learning culture
- What was your last step and what happend?
- What did you lean?
- What is your condition now?
- What is your next target condition?
- What obstacle are your working on now?
- What is your next step?
- What is your expected outcome?
- When can we check?
The third way
The principle of continual learning and experimentation
and That's just the BEGINING!
and That's just the BEGINING!
