Stop Automating Deployments...

Automate the World!!!

Lansing DevOps Meetup

April 7th, 2015

Brendon Thiede

Eric Fusciardi

Release Engineers

...How many environments?

...How many servers?

...How many variations?

scenario 1:

java certs update

Complications

• Multiple JVMs
• Scheduled job
• Distributed queue
• Downstream systems

Symptoms

• Inconsistent
• Partially processed

Diagnosis

• Customer data
• Bugs in the code
• Inconsistent cacert found

Scenario 2:

Anti-Virus Install

Complications

• Different file structures
• Load balancer

Symptoms

• Inconsistent
• Sometimes works

Diagnosis

• Bugs in code
• Magic files
• Wrong file permissions found

Scenario 3:

Error in the Spec

Complications

• Inconsistent format
• Different configuration methods
• Divergent naming conventions

Symptoms

• Application is "slow"
• Everything still works
• Logs are full of "noise"

Diagnosis

• VM is "bad"
• Poor load balancing scheme
• Increased customer volume
• Found queue was not distributed

Scenario 4:

Upgrade host OS version

...you get the idea

But what about peer review???

The Big Payoff:

The Future

Chef

Infrastructure as Code

Adding a new service

default['microsvcs']['manifest'] = [
  {name: 'a-service', user: 'derpy', port: '8000', topology: 'backrail'},
  {name: 'b-service', user: 'bigmac', port: '8010', topology: 'dmz'},
  {name: 'c-service', user: 'rarity', port: '8020', topology: 'backrail'}
]

default['microsvcs']['manifest'] = [
  {name: 'a-service', user: 'derpy', port: '8000', topology: 'backrail'},
  {name: 'b-service', user: 'bigmac', port: '8010', topology: 'dmz'},
  {name: 'c-service', user: 'rarity', port: '8020', topology: 'backrail'},
  {name: 'd-service', user: 'applejack', port: '8030', topology: 'dmz'}
]

Change this:

To this:

Chef is:

• Versionable
• Testable
• Repeatable
• Reusable

Bamboo

Build once

• SCM metadata captured
• Artifact flows through environments

*Panda adds promotion criteria

Bamboo Handles:

• Continuous integration
• Automated testing
• Scheduling
• Auditing*
• Deployments*

Pillar and Liquibase

DML/DDL are Code

• Same SCM
• Versioned
• True rollback
• Continuous integration
• Always ship everything
• Only apply what's needed

Are We There Yet?

End Game

• Lower environments from scratch
• Prod-like setup
• Prod-like deployment
• Realistic data
• Continuous improvement