Transitioning Towards Continuous Delivery
A brief overview of how Continuous Delivery was adopted at Instagram and how it is/can be used at Bloomberg
Spencer Carver - October 14, 2016
Pretext
The information from Instagram in these slides is summarized from was Michael Gorven's talk "Continuous Deployment to Millions of Users 40 Times A Day" at Velocity NY on September 21, 2016.
If anything differs from the way it was originally presented, it is the fault of the current presenter's memory.
Instagram's Delivery Process
v1
1. Developer opens a PR
1
2. Code Review
2
4. Staging Verification
3
5. Canary Prod Verification
4
6. Staged Rollout (Cluster -> Region -> Fleet)
5
6
3. Test Suite
Manual
Automated
Instagram's Delivery Process
v2
1. Developer opens a PR
1
2. Code Review
2
4. Staging Verification
3
5. Canary Prod Verification
4
6. Staged Rollout (Cluster -> Region -> Fleet)
5
6
3. Test Suite
Manual
Automated
Instagram's Delivery Process
v2.5
1. Developer opens a PR
1
2. Code Review
2
4. Staging Verification
3
5. Canary Prod Verification
4
6. Staged Rollout (Cluster -> Region -> Fleet)
5
6
3. Test Suite
Manual
Automated
Instagram's Delivery Process
v3
1. Developer opens a PR
2
2. Code Review
1
4. Staging Verification
3
5. Canary Prod Verification
4
6. Staged Rollout (Cluster -> Region -> Fleet)
5
6
3. Test Suite
Manual
Automated
Instagram's Delivery Process
v3 (Pipelining)
Failure
Success
New deploy becomes base, grab next PR in queue
Old deploy still base, mark error and grab next PR in queue
Bloomberg Javelin Delivery Process
1. Developer opens a PR
1
2. Code Review
2
4. Staging Verification
3
5. Canary Prod Verification
4
6. Production Rollout
6
3. Test Suite
Manual
Automated
Build This
Stage This
Deploy This
Comparing Instagram v2, v3, and Javelin
1
2
3
4
5
6
2
1
3
4
5
6
1
2
3
4
6
Javelin
Instagram v2 (powered by human)
Instagram v3 (powered by Sauron)
My Takeaways
- Canary Prod
- Better Heathchecks / Automated Scan
- (Potentially) Change our workflow so that the tools we have are more fully able to pipeline deployments
- Most Importantly, remember this takes time. Instagram adopted it slowly over 2 years until all developers were comfortable with each step.