Testing infra:
CI, CD, and other unicorns
Continuous Integration
Methodology
- Rebase frequently
- Test even more frequently
- Don't merge if tests don't pass
Rebase frequently
Avoid conflict hell
Test more frequently
Don't carry over broken code (implicit conflict hell)
Don't merge if tests broken
Spare conflict hell to everyone else
Unless you are Chuck Norris
Continuous Delivery
Methodology
- Build often
- Keep every merged commit releasable
- Automate environment deploys
- Creation of environments on demand
- Big Red Deploy Button
That's even less reasonable than CI...
Build often
Avoid build-time surprises:
- Missing libraries
- Library upgrades
- Bugs in build process/scripts
Every merged commit releasable
- Encourages smaller changes.
- Easy to deploy ANY change.
- That also means easy to deploy a revert (no need for rollback).
- Forces to keep all the build/release scripts or code up to date and working.
Automate environment deploys
- Easy to recreate envs from scratch
- Easy to create testing envs.
- Easy to create demo envs.
- Full provisioning process tested (and improved) all the time.
Big red deploy button
- Makes deployment independent of anybody.
- Value gets to production way faster, often, and safer.
Deployment becomes ubiquitous
Changes can be deployed any time
Changes can be deployed without special knowledge
DevOps
It's not
- A tool
- A set of tools
- A role (the devops guy)
- A team
- Operations/infrastructure
- Continuous integration/delivery/deployment
We all are DevOps
Not really...
Software development process
Might include:
- Agile development
- Continuous Integration
- Continuous Delivery
- Canary deployments
- A/B testing
- Dailies
- Infrastructure as code
- Cloud deployments
- ...
Chess vs Go
Copyright Alan Light - CC BY-SA 3.0
Copyright Donarreiskoffer - CC BY-SA 3.0
Three basic principles
Value flow
Remember the candy test!
Feedback
Continuous learning and experimentation
3 types of tests (kind of)
Quick tests
Quick tests
Locally run tests
$ docker-compose -f docker-compose.deps.yml run --rm pip
$ docker-compose -f docker-compose.deps.yml run --rm assets
$ docker-compose -f docker-compose.tests.yml run --rm unit
$ docker-compose -f docker-compose.tests.yml run --rm integration
$ docker-compose -f docker-compose.tests.yml run --rm workflows
$ docker-compose -f docker-compose.tests.yml run --rm acceptance
Quick tests
Travis
Normal tests
Normal tests
Migration tests (nightly)
Normal tests
Acceptance tests
Slow tests
Slow tests
Manual/end to end tests:
QA
Inspire infra overview
All envs
CI
CI
CI
QA
QA
...
Production
CI/CD stream
Current status
Jenkins setup
- Openshift hosted master
- Kubernetes connected slaves
- Docker image creation + repository
- Triggering on master merge (well, kinda)
- Run of acceptance tests
- User-triggered dummy deploy job
- Automate QA deployment
- Run of the migration tests
- Full pipeline
We have:
We lack:
Kubernetes setup
- Magnum generated cluster
- Easy add/remove nodes
- Monitoring (performance + status)
We have:
We lack:
QA setup
- Easy manual deploy steps (for non-breaking)
- Non-manual deployments
- Automated full reprovisioning
- Compatibility-breaking changes deployment
We have:
We lack:
Questions
Testing Infrastructure: CI, CD and other unicorns
By David Caro
Testing Infrastructure: CI, CD and other unicorns
- 793