Is Terraform worth the hype?

26 April 2017

 

Stoyan Stoyanov

Michael Munsch

 

welt.de

Who are we?

Who are we?

Structure

  • ~60 ppl, 5 Teams
  • Cross-functional teams
  • Every team has its own AWS Account

Tech Jungle

Our Team

  • Owns ~30 services
    • Payments
    • Premium features
    • Comment system
    • Personalization

Most used services

AWS Setup a year ago

How the infrastructure code evolved

  • From AWS CloudFormation to HashiCorp Terraform
    • No JSON
    • Plan
    • No more glue code for sharing resource ids
    • Basic templating
    • A lot of build-in functions
    • Written in Go and open source
variable "aws_region" {
  default = "eu-central-1"
}

provider "aws" {
  region = "${var.aws_region}"
}

data "terraform_remote_state" "vpc" {
  backend = "s3"

  config {
    bucket = "tf-foobar-production"
    key    = "vpc/terraform.tfstate"
    region = "eu-central-1"
  }
}

resource "aws_security_group" "clair_allow_outbound" {
  description = "Allow all outbound"
  vpc_id      = "${data.terraform_remote_state.vpc.vpc_id}"
}

resource "aws_security_group_rule" "allow_all" {
  type        = "egress"
  from_port   = 0
  to_port     = 65535
  protocol    = "tcp"
  cidr_blocks = ["0.0.0.0/0"]

  security_group_id = "${aws_security_group.clair_allow_outbound.id}"
}

output "clair_sg_outbound" {
  value = "${aws_security_group.clair_allow_outbound.id}"
}

Workflow

  • read docs, code, fmt, plan, apply, (destroy)
  • read docs, code, fmt, plan, push to git, ci plan, ci apply

The State

  • Snapshot of the infrastructure
  • Can be local or remote with different backends
    • S3, consul, etcd ..
  • Can be encrypted depending on backend
  • Locking is also possible depending on backend
{
    "version": 3,
    "terraform_version": "0.9.2"
    .....
    "aws_security_group_rule.allow_all": {
        "depends_on": [
            "aws_security_group.clair_allow_outbound"
        ],
        "deposed": [],
        "primary": {
            "attributes": {
                "cidr_blocks.#": "1",
                "cidr_blocks.0": "0.0.0.0/0",
                "from_port": "0",
                "id": "sgrule-1926861517",
                "prefix_list_ids.#": "0",
                "protocol": "tcp",
                "security_group_id": "sg-e16e938a",
                "self": "false",
                "to_port": "65535",
                "type": "egress"
            },
            "id": "sgrule-1926861517",
            "meta": {
                "schema_version": "2"
            },
            "tainted": false
        },
        "provider": "",
        "type": "aws_security_group_rule"
    }
}

Demo 1

  • Go binary in a scratch Docker image ~ 2 mb
  • ECR
  • ECS Cluster with 2 nodes
  • Autoscaling, ALB, SGs, IAM

The State

  • It was the main advantage over AWS CloudFormation
  • No more aws cloudformation describe-stacks
    • Outputs can be shared between "stacks" via remote states

Complexity

Fighting complexity

  • Use modules as abstractions
  • Centralized applying
    • Plan the core infrastructure every hour
  • Wrapper script for the order of the plans and applies of core components
    • Hosted Zones, VPCs, Subnets, ECS clusters
  • The whole teams should own the infrastructure code

Modules

  • Sharing code between teams
    • Sources from relative path, github, S3, bitbucket .. 
  • Examples:
    • private_nets, public_nets, ecs, ecs_app ..
    • grafana, chef_server, clair, consul .. 
  • Versioning is really important !

Advanced TF

  • Integrating Configuration Management
    • Complexity is skyrocketing (AWS, Chef, Ruby, TF)
    • Makes sense for databases and really complex configurations
    • Why not using just user data?
  • Plugins

Demo 2

Drawbacks

  • Very frequent tf updates
  • Forgetting to pull the remote state
  • Not exporting module outputs twice (newbie mistake #1)
  • Plan != Test

Tips'n'Tricks

  • Make small changes and integrate them immediately
    • "game over" without CI
  • Create some kind of conventions
    • Inputs, outputs, remote states in different files
  • Don't use deep hierarchies !

Tips'n'Tricks

  • Stable software is always versioned
    • Modules
    • States
  • If for some reason there is no tf resource for the job
    • plugin
    • local-exec + awscli
  • Test creating your whole infrastructure every day in different region

Learnings

  • Best tool for the job if you are using AWS
    • Production-ready
    • Smooth learning curve
    • Saves us a lot of time
    • Makes infrastructure management a solved problem

Is it worth the hype?

btw we are hiring :)

https://goo.gl/mA06k3

Is Terraform worth the hype?

By Stoyan Stoyanov

Is Terraform worth the hype?

  • 1,672