New Service Deployment

Considerations

David Fetterman

for Everactive

2020/03/23

Q: Who is this person?

A: Dave Fetterman

  • Local (to C'ville)
  • Family
  • Working (mostly) in technology since 1995
    • Many roles:
      • Developer - Java, Perl, Bash, Ruby, .NET
      • Database Admin - MySQL, Postgres, Oracle, MSSQL
      • Unofficial server monkey
      • Tech Support
  • Linux/Unix geek
  • Lots of other jobs along the way...

(too many?) Driving Principles

  • Don't Panic (I always have a towel with me)
  • Be honest
  • Test everything, collect everything, automate everything - in that order
  • Pragmatism over Dogma
  • YAGNI, DRY, and other Agile TLA's
  • Teach people to fish and how to react to others being hit by buses.
  • Don't reimplement the wheel, especially if your business is windshields
  • Always have a plan B, and if possible, plans C..n

"Everactive is launching a new microservice as a part of a new feature in our product and would like to use Infrastructure as Code to control the deployment. Describe how you would set up the infrastructure in AWS to support running this microservice. Assume the microservice is available as a docker container already."

Task:

I was going to draw diagrams and show some code,

but I made a bunch of bulletpoints instead.

Assumptions:

  • The AWS Account is already configured and basic infrastructure exists (Default VPC, Route53, etc.)
  • The Docker container provided has passed all unit, integration, and acceptance tests.
  • The new microservice is internally facing.
  • Authentication/Authorization handled by the application.

Risks:

  • Amazon outages
  • Misconfigured environment
  • "Bad artifact"
  • Partial deployment failure
  • Unexpected performance impact under load
  • So many others...

 Infrastructure

  • Primary Setup/Configuration via Terraform
  • Environment - one time setup, occasionally updated
    • VPC(s)
    • ECR
    • AWS AppMesh (maybe in the group below?)
  • Application - every deployment
    • ECS
    • Route53
    • Database(s) - migrations, etc.
  • Support
    • Datadog - Metrics & Logging
    • Consul - Service Discovery & Configuration
    • AWS Secrets Manager / Vault - Secret Management

Things to consider:

  • AWS Roles and Permissions

  • Upstream build pipeline

  • Environments, e.g. test, stage, prod...

  • Identifying new versions and targeted environments for container artifacts.

  • AppMesh:
    • Deployment strategies:
      • Full or incremental cut-overs
      • Rollback capabilities?
      • Triggering deployments
      • Failures

More to Consider

  • ECS:
    • Fargate or EC2?
    • Scaling
    • Failure detection and responses
  • Terraform scripts:
    • in VCS
    • Code review
    • Generated?
    • Testing

Considering Some More

  • DataDog, and other loggers/reporters
    • CloudWatch
    • AWS Metrics
    • Log ingestion
    • APM
  • Alerting
    • Signal/Noise ratio
    • Notification methods
  • Secrets management
  • Configuration
  • Circuit breakers
  • Service discovery
Made with Slides.com