テスト駆動型インフラストラクチャ
コードとしてのコンプライアンス
by
Joaquín Menchaca
ホアキーン • メンチャカ
私についての情報
About ME
ロケット弁護士
職業：フルタイム忍者
実際の写真
Test Driven
InfraStructure
Compliance as Code
by
Joaquín Menchaca
À propos de moi
About ME
Rocket LAWYER
SENIOR DEVOPS BUILD/RELEASE ENGINEER
aka
Linux Ninja Pants Automation Engineering Mutant
actual photo
AGENDA
- Setup
- Context
- Chef + InSpec
- Ansible + InSpec
- Bonus
SETUP
ThE CODE
git clone \
https://github.com/darkn3rd/lisa18_test_driven_infra
cd lisa18_test_driven_infra
# Using Virtual Workstation
vagrant up
vagrant ssh
cd lisa18_test_driven_infra
# Using Host (MacOS X or Linux)
#### Install Requirements
# Using Host (Windows)
#### Install Rrequiremnts
#### Warning: Might not work, had success w/ ChefCODE
SETUP
The HoST
must be able to run
Docker
or
Vagrant
Minimum ReqUIRED
ChefDK - bundles ruby, test kitchen, inspec
Docker - fastest way to run stuff (virtual virtual machines)
DOCKER
Easiest Path is Docker Desktop
choco install docker-for-windowsbrew cask install docker
https://chocolatey.org/
https://brew.sh/
https://www.docker.com/products/docker-desktop
DirecT DOWNLOAD
Package Managers
DOCKER
sudo apt-get update -qq
sudo apt-get install -y apt-transport-https \
curl ca-certificates software-properties-common
DOCKER_REPO="https://download.docker.com/linux/ubuntu"
curl -fsSL ${DOCKER_REPO}/gpg | \
sudo apt-key add -
sudo add-apt-repository \
"deb [arch=amd64] ${DOCKER_REPO} \
$(lsb_release -cs) \
stable"
sudo apt-get update -qq
sudo apt-get install -y docker-ce
sudo usermod -aG docker $USER
Debian Package on Ubuntu
DOCKER
CHEFDK
Easiest Way to Get Test Kitchen and InSpec
choco install chefdk
chef gem install kitchen-ansible
chef gem install kitchen-dockerbrew tap chef/chef
brew cask install chefdk
chef gem install kitchen-ansible
chef gem install kitchen-dockerhttps://chocolatey.org/
https://brew.sh/
https://downloads.chef.io/chefdk/
DirecT DOWNLOAD
Package Managers
CHEFDK
VER=3.2.30
PKG=chefdk_${VER}-1_amd64.deb
PREFIX=https://packages.chef.io/files/stable/chefdk/
# Fetch and Install
wget --quiet ${PREFIX}/${VER}/ubuntu/16.04/${PKG}
sudo dpkg -i ${PKG}
# Local ChefDK Ruby Gems
chef gem install kitchen-ansible
chef gem install kitchen-dockerDebian Package on Ubuntu
CHEFDK
CONTEXT
WHY?
WHY?
We Code
SYSTEMS
We Code
PLATFORMS
We Code INFRASTRUCTURE
WHY?
WHY?
- Disaster Recovery, Rollback
- Migrate/Replicate New Environments
- Swap out components, Experiment
- Rapid (Like Seriously) Development
Destroy and Recreate with Ease
Because Tests…
WHEN?
WHEN
You have
an
Artifact?
What?
WHAT TO TEST?
HOW?
ABOUT THE
TOOLS
TEST KITCHEN
Create (Vagrant)→Converge(Chef)→Verify(InSpec)
TEST KITCHEN: Vagrant
---
driver:
name: vagrant
provider: hyperv
provisioner:
name: chef_zero
verifier:
name: inspec
platforms:
- name: ubuntu-16.04
suites:
- name: default
Create (Docker)→Converge(Ansible)→Verify(InSpec)
TEST KITCHEN
---
driver:
name: docker
provisioner:
name: ansible_playbook
verifier:
name: inspec
platforms:
- name: ubuntu-16.04
suites:
- name: default
TEST KITCHEN: DOCKER
control 'ez_apache-contract-01' do
describe port(80) do
it { should be_listening }
end
end
control 'ez_mysql-security-conf-01' do
describe file(mysql_data_path) do
it { should be_directory }
it { should be_owned_by 'mysql' }
it { should be_grouped_into 'mysql' }
end
endINSPEC
package 'apache2'
service 'apache2' do
action %i(enable start)
supports(
status: true,
restart: true,
reload: true
)
end
cookbook_file "#{node['docroot']}/index.html" do
source 'index.html'
action :create
endCHEF
- name: "Install Web Service"
package:
name: apache2
state: present
- name: "Start Web Service"
service:
name: apache2
state: started
enabled: yes
- name: "Copy Content"
copy:
src: "{{ role_path }}/files/index.html"
dest: "{{ docroot }}/index.html"
ANSIBLE
InSPEC
CHEF
CookBOOK STRUCTURE
./chef/cookbooks/ez_mysql
├── attributes
│ └── default.rb
├── Berksfile
├── kitchen.docker.yml
├── kitchen.hyperv.yml
├── kitchen.vbox.yml
├── metadata.rb
├── README.md
├── recipes
│ ├── client.rb
│ ├── database.rb
│ ├── default.rb
│ ├── harden.rb
│ ├── install.rb
│ └── service.rb
├── templates
│ └── security.cnf.erb
└── test -> ../../../inspec/ez_mysql/testTEST STRUCTURE
./inspec/ez_mysql
└── test
└── integration
└── default
├── ansible.cfg
├── conform_test.rb
├── contract_test.rb
├── default.yml
└── security_test.rbShared Test for ANY change configuration, e.g.
CAPS (Chef, Ansible, Puppet, Salt Stack), or other automation
THE STEPS
# Setup (tell kitchen to use docker)
export KITCHEN_YAML=kitchen.docker.yml
# Create Environment
kitchen create
# (optional) Login into environment
kitchen login
# Converge to Desired State
kitchen converge
# Verify Environment
kitchen verifyWORKED FINE?
THE STEPS
include_recipe 'ez_mysql::install'
include_recipe 'ez_mysql::service'
# include_recipe 'ez_mysql::database'
include_recipe 'ez_mysql::harden'
THE STEPS
include_recipe 'ez_mysql::install'
include_recipe 'ez_mysql::service'
include_recipe 'ez_mysql::database'
include_recipe 'ez_mysql::harden'
WHEN CODE WORKS
ANSIBLE
InSPEC
ROLE STRUCTURE
./ansible/roles/ez_mysql
├── defaults
│ └── main.yml
├── handlers
│ └── main.yml
├── kitchen.docker.yml
├── kitchen.hyperv.yml
├── kitchen.vbox.yml
├── meta
│ └── main.yml
├── tasks
│ ├── client.yml
│ ├── database.yml
│ ├── harden.yml
│ ├── install.yml
│ ├── main.yml
│ └── service.yml
├── templates
│ └── security.cnf.j2
└─── test -> ../../../inspec/ez_mysql/testTEST STRUCTURE
./inspec/ez_mysql
└── test
└── integration
└── default
├── ansible.cfg
├── conform_test.rb
├── contract_test.rb
├── default.yml
└── security_test.rbShared Test for ANY change configuration, e.g.
CAPS (Chef, Ansible, Puppet, Salt Stack), or other automation
THE STEPS
# Setup (tell kitchen to use docker)
export KITCHEN_YAML=kitchen.docker.yml
# Create Environment
kitchen create
# (optional) Login into environment
kitchen login
# Converge to Desired State
kitchen converge
# Verify Environment
kitchen verifyWORKED FINE?
THE STEPS
- include: install.yml
- include: config.yml
- include: service.yml
# - include: harden.ymlTHE STEPS
- include: install.yml
- include: config.yml
- include: service.yml
- include: harden.ymlWHEN CODE WORKS
EXTRA STUFF
INRAstructure
variable "username" {
default = "admin"
}
variable "password" {}
variable "project" {}
variable "region" {}
module "gke" {
source = "./gke"
project = "${var.project}"
region = "${var.region}"
username = "${var.username}"
password = "${var.password}"
}
INRAstructure
module "k8s" {
source = "./k8s"
host = "${module.gke.host}"
username = "${var.username}"
password = "${var.password}"
client_certificate = "${module.gke.client_certificate}"
client_key = "${module.gke.client_key}"
cluster_ca_certificate = "${module.gke.cluster_ca_certificate}"
}INRAstructure
password = "$3kr3tl!k3$er!ou$1Y"
project = "lisa18-test-driven-infra"
region = "us-east4"
terraform plan \
-var-file="kluster.tfvars"
terraform apply \
-var-file="kluster.tfvars"kluster.tfvars
console
INRAstructure
name: gcp-profile
title: InSpec Profile
maintainer: Joaquín Menchaca
copyright: Joaquín Menchaca
license: MIT
summary: InSpec GCP Demo
version: 0.1.0
depends:
- name: gcp-resources
url: https://github.com/inspec/inspec-gcp/archive/master.tar.gz
supports:
- platform: gcpguestbook-profile/inspec.yml
INRAstructure
title 'GCP Guestbook Cluster'
control 'gcp-1' do
impact 0.7
title 'Check Guestbook Cluster'
describe google_container_cluster(
project: PROJECT_NAME,
zone: CLUSTER_ZONE,
name: CLUSTER_NAME
) do
it { should exist }
its('name') { should eq CLUSTER_NAME }
its('status') { should eq 'RUNNING' }
its('master_auth.username')
{ should eq 'admin' }
its('network') { should eq "default" }
its('subnetwork') { should eq "default"}
its('initial_node_count') { should eq 3 }
end
end
guestbook-profile/control/cluster.rb
INRAstructure
project_name: lisa18-test-driven-infra
cluster_zone: us-east4-c
cluster_name: guestbook
inspec exec guestbook-profile \
-t gcp:// --attrs attributes.yml
attributes.yml
console
FINAL NOTES
(for reference)
TESTED PLATFORMS
Mac OS HOST
LINUX GUEST
WINDOWS HOST
☑ Mac/Vagrant (Virtualbox) / TestKitchen (Docker) / Chef
☑ Mac/Vagrant (Virtualbox)/ TestKitchen (Docker) / Ansible
☑ Win/Vagrant (HyperV) / TestKitchen (Docker) / Chef
☐ Win/Vagrant (HyperV)/ TestKitchen (Docker) / Ansible
☑ TestKitchen (HyperV) / Chef
☒ TestKitchen (HyperV) / Ansible
☐ TestKitchen (Docker) / Chef
☐ TestKitchen (Docker) / Ansible
☑ TestKitchen (Virtualbox) / Chef
☑ TestKitchen (Virtualbox) / Ansible
☑ TestKitchen (Docker) / Chef
☑ TestKitchen (Docker) / Ansible
FURTHER READING
TEST KITCHEN RESROUCES
-
Test Kitchen: https://kitchen.ci/
-
Ansible Provisioner: https://github.com/neillturner/kitchen-ansible
-
Vagrant Driver: https://github.com/test-kitchen/kitchen-vagrant
-
Docker Driver: https://github.com/test-kitchen/kitchen-docker
-
InSpec Verifier: https://github.com/inspec/kitchen-inspec
-
GCP Driver: https://github.com/test-kitchen/kitchen-google
-
AWS Driver: https://github.com/test-kitchen/kitchen-ec2
-
Kubernetes Driver: https://github.com/coderanger/kitchen-kubernetes
Vagrant
-
HashiCorp Vagrant: https://www.vagrantup.com/
DOCKER
-
Docker: https://www.docker.com/
FURTHER READING
INSPEC
-
InSpec: https://www.inspec.io/
-
inspec-gcp: https://github.com/inspec/inspec-gcp
-
inspec-gcp blog: https://blog.chef.io/2018/06/19/inspec-gcp-deep-dive/
KUBERNETES
-
Sonobuoy: https://github.com/heptio/sonobuoy
TERRAFORM
-
Terratest: https://github.com/gruntwork-io/terratest
-
kitchen-terraform: https://github.com/newcontext-oss/kitchen-terraform
FURTHER READING
INSPEC SCRIPTS
DevSec Hardening Framework: https://dev-sec.io/index.html
Try InSpec Tutorial: https://learn.chef.io/modules/try-inspec#/
Create InSpec Profile fro Compliance Documentation: https://learn.chef.io/modules/create-profile-from-doc#/
Testing Servers with InSpec and Test Kitchen: https://mkdev.me/en/posts/testing-servers-with-inspec-and-test-kitchen
FURTHER READING
OTHER QUESTIONS
THE END