Automation 101

Automation 101

Automation without frustration

$whoami

Diana Rodríguez 

Google Developer Expert

Auth0 Ambassador

Microsoft MVP 

🐍 Developer Advocate @ Nexmo

GDG Durham Organiser @ gdgdurham

🦊  https://gdgdurham.org

🐦  @cotufa82

 

@cotufa82

$whoami-not

@cotufa82

Effects of DevOps
 

The adoption of DevOps culture, tools and agile engineering practices has, among other things, the nice effect of increasing the collaboration between the roles of development and operations. One of the main problems of the past (but also today in some realities) is that the dev team tended to be uninterested in the operation and maintenance of a system once it was handed over to the ops team, while the latter tended to be not really aware of the system’s business goals and, therefore, reluctant in satisfying the operational needs of the system (also referred to as “whims of developers”).

Vue.js Docs

📜

@cotufa82

@cotufa82

  • Automation is a term that refers to the processes and tools an organisation uses to reduce the manual efforts associated with provisioning and managing infrastructure.
  • Infrastructure as Code (IaC) is the management of infrastructure (networks, virtual machines, load balancers, and connection topology) in a descriptive model, using the same versioning as DevOps team uses for source code. ... Infrastructure as Code evolved to solve the problem of environment drift in the release pipeline

@cotufa82

@cotufa82

Frustration Factors

  • Learning Curve / Complexity of tools
  • Documentation / Community
  • Open Source vs Paid
  • Adaptability (specific needs)
  • Provider compatibility
  • Restrictions (lack of control)
  • Migration issues

@cotufa82

@cotufa82

WHY?!

  • Error prone errors. Where there are manual tasks there is always a huge chance of 

  • Make the "computer" or "the cloud" do the work for you... not the other way around

  • Make your life easier

  • Reuse code. 

  • You don't want to repeat the same tasks over and over again?  Script them 

💩

🤪

@cotufa82

 

  • Configuration orchestration tools, which include Terraform and AWS CloudFormation, are designed to automate the deployment of servers and other infrastructure.

 

  • Configuration management tools like Chef, Puppet, etc to help configure the software and systems on this infrastructure that has already been provisioned.

 

  • Configuration orchestration tools do some level of configuration management, and configuration management tools do some level of orchestration. Companies can and many times use both types of tools together.

Configuration Orchestration vs. Configuration Management

@cotufa82

Manage

bootstrapping and initialising resources.

Deploy

infrastructure deployments

⇄

ARM

GCP

@cotufa82

@cotufa82

What we had...

@cotufa82

Allows you to describe and provision all the infrastructure resources in your cloud environment. [...] allows you to use a simple text file to model and provision, in an automated and secure manner [...] This file serves as the single source of truth for your cloud environment.

@cotufa82

Eliminating frustration factors

@cotufa82

We migrated our cloudformation stacks to terraform because the learning curve and templating restrictions were too harsh for our needs. We didn't have effective ways to handle deployment errors or roll backs! 

 

Before this migration predecessors used configuration management tools like chef without good results. (Mainly lack of tool knowledge) So we resorted into creating our own boot scripts, and surely the process can be improved as it is ongoing. 

@cotufa82

Terraform is the first multi-cloud immutable infrastructure tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.  

  •  You can define infrastructure in config/code and rebuild/change and track changes to infrastructure.
  • Source control & Testing
  • Platform Agnostic
  • Strong community & open source
  • Relatively easy learning curve
  • Speed: Reuse code!!
  • Plan and review before applying changes!

@cotufa82

provider "aws" {
  access_key = "ACCESS_KEY_HERE"
  secret_key = "SECRET_KEY_HERE"
  region     = "us-east-1"
}

resource "aws_instance" "example" {
  ami           = "ami-2757f631"
  instance_type = "t2.micro"
}
$ terraform init
Initializing the backend...

Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "aws" (terraform-providers/aws) 2.10.0...

To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.

* provider.aws: version = "~> 2.10"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
$ terraform apply
# ...

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_instance.example will be created
  + resource "aws_instance" "example" # Details ommitted on purpose
    }

Plan: 1 to add, 0 to change, 0 to destroy.
# ...
aws_instance.example: Creating...
aws_instance.example: Still creating... [10s elapsed]
aws_instance.example: Creation complete after 1m50s [id=i-0bbf06244e44211d1]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

@cotufa82

  • 5 VPC
  • 4 Internet Gateways
  • 3 peering connections
  • 5 Route tables  with multiple destinations
  • All availability zones in our region
  • ~ 10 subnets per availability zone
  • 4 Nat Gateways
  • 18 prod, 7 Staging, 28 dev stacks
  • IAM Roles...
  • Other manually created resources that should be preserved

Small scale...

@cotufa82

  • Repository - Deployment flow using tags for releases

  • S3 bucket to store the .tfstate files (remote backend)

  • MIgrating current resources and referencing our standard templates

  • Communication!!

@cotufa82

😱

✅

@cotufa82

@cotufa82

  • Improved Cloudwatch alarms with more freedom
  • Isolation policies for deployments to affect the selected resource or working space (check plans too)
  • Reducing deployment time
  • Usage of `prevent_destroy` for critical resources
  • Usage of lambda functions to check health of resources not attached to an ELB
  • Slack hooks to notify of changes in a #terraform channel
  • Developed wrapper scripts to allow operations with a single command that includes a process of:
  • committing code
  • planning and locking plans refreshing remote state
  • notify of upcoming changes with specific tags and display                  plans on slack for review
  • when discussed then, the person responsible of applying, can              do so and still review the plan and confirm applying changes.

@cotufa82

@cotufa82

Made with Slides.com