DevOps cult
Chef Style DevOps Kung Fu
- On Github: http://github.com/chef/devops-kungfu
- Join by PR
- We work together to develop our principles, forms, and applications
- Start your own school by forking, removing the names, and changing the content
Roadmap
- Materialize the vision
- Align early feedback
- Balance innovation with real needs
- Design themes, with an outcome for each
- Distill those into features, and validate
Vision:
Deploy and operate all infrastructures by code, as applications are code too.
As traditional IT becomes "truck" and cloud native is a "car". Practice DevOps and apply cloud architectures to acquire expertise, tools and operational skills to drive.
Purpose
- every team needs an back-end systems
- every solution has an infrastructure
- every project require repeatable deployment
- recognized approach cross teams
Believes
- brings velocity to drive robust solutions
- self document infrastructure as a code
- none laptop has enough memory
- dirty hands touching ssh is pain in the ass
- fix, batch, deploy, re-use, automate
Choosed languages and tools that fit the job
- Kitchen CI
- Drone.io
- ServerSpec, InSpec
- Fog cloud libs
- Virtualbox
This means that Chef can:
- Build Deterministic Servers
- Deploy and Configure Applications
- Deploy Infrastructures
- Update Applications
- Run audits and notifications
Chef is a configuration management
and automation platform
Overview
How it works
chef-provisioning
recipes
knife-softlayer
knife-ec2
knife-google
Ecosystem
Chef Development Kit (ChefDK)
$ chef generate app cookbook_name
$ chef generate attribute
$ chef generate recipe
$ chef generate lwrp
$ chef-apply name_of_recipe.rb
- chef 12.0.3
- chefspec 4.2.0
- chef-provisioning 0.18
- chef-provisioning-aws 0.2.1
- chef-provisioning-fog 0.12
- chef-vault 2.4.0
- knife-spork 1.5.0
- rubocop 0.28.0
Ecosystem
$ knife search node "platform:centos AND languages_ruby_version:1.8*"
$ knife search node "chef_environment:production AND platform:ubuntu"
$ knife ssh 'name:* AND NOT name:chef*' 'sudo postmap -p /etc/postfix/transport'
$ knife bootstrap --environment dev --node-name catalog.lab.dev 10.10.50.15 \
--ssh-password passw0rd --run-list "role[base]" --server-url "https://10.10.50.2" \
--json-attributes '{ "hostname": "catalog","name": "catalog.lab.dev", \
"system":{ "short_hostname":"catalog", "domain_name":"lab.dev" } }' \
--secret-file .chef/encrypted_data_bag_secret
$ knife node run_list add ora.kb.dev "role[git], recipe[backup::gitlab]"
$ knife role from file roles/*.rb
$ knife environment from file staging.rb
knife
** KNIFE COMMANDS **
bootstrap, client, configure, cookbook, cookbook site, data bag, delete, deps,
diff, download, edit, environment, exec, list, node, raw, recipe_list, role,
search, serve, show, ssh, ssl check, ssl fetch, status, tag, upload, user
** NODE COMMANDS **
knife node bulk delete REGEX (options)
knife node create NODE (options)
knife node delete NODE (options)
knife node edit NODE (options)
knife node environment set NODE ENVIRONMENT
knife node from file FILE (options)
knife node list (options)
knife node run_list add [NODE] [ENTRY[,ENTRY]] (options)
knife node run_list remove [NODE] [ENTRY[,ENTRY]] (options)
knife node run_list set NODE ENTRIES (options)
knife node show NODE (options)
Ecosystem
** SOFTLAYER COMMANDS **
knife softlayer datacenter list (options)
knife softlayer datacenter show DATACENTER
knife softlayer flavor list (options)
knife softlayer global ip list (options)
knife softlayer image list
knife softlayer key pair create
knife softlayer key pair list
knife softlayer server create (options)
knife softlayer server destroy (options)
knife softlayer server list (options)
knife softlayer server relaunch <NODE NAME> [<NODE NAME>]
knife softlayer vlan create
knife softlayer vlan list (options)
knife softlayer vlan show ID (options)
- knife-softlayer
- knife-ec2
- knife-azure
- knife-windows
- knife-bluebox
- knife-openstack
- knife-rackspace
- community plugins
knife plugins
Ecosystem
chef supermarket
Ecosystem
include_recipe 'apache2'
user 'userdoc' do
supports manage_home: true
comment 'userdoc (bind user)'
gid 'users'
home '/home/userdoc'
password '$1$9p26dAeZ$qZ7P4XW.ASDG4DSD3234vK0'
end
directory '/var/www/userdoc' do
mode 00775
owner 'apache2'
group 'users'
recursive true
end
web_app 'userdoc' do
server_name "udoc.#{node['domain']}"
server_aliases ["userdoc.#{node['system']['domain_name']}"
docroot '/var/www/userdoc'
cookbook 'apache2'
directory_options 'Indexes FollowSymLinks'
end
cookbooks, recipes
cookbook
|
├── .kitchen.yml
├── Berksfile
├── metadata.rb
├── attributes
│ └── default.rb
├── recipes
│ └── default.rb
└── templates
└── default
└── index.html.erb
# metadata.md
name 'gitzone'
maintainer 'Petr Michalec'
maintainer_email 'epcim@apealive.net'
license 'Apache 2.0'
description 'Installs/Configures gitzone managed zone files for Bind'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '1.0.15'
recipe 'gitzone::default', 'Install and fully configure gitzone using the install and configure recipes'
recipe 'gitzone::install', 'Install gitzone scripts'
recipe 'gitzone::configure', 'Configure gitzone (including BIND and default zones)'
recipe 'gitzone::configure_zonefile', 'Deploy zone files'
supports 'ubuntu'
supports 'centos'
depends 'bind'
depends 'git'
depends 'sudo'
depends 'system'
depends 'build-essential'
depends 'simple_iptables'
Ecosystem
require 'chef/provisioning'
with_driver 'fog:OpenStack'
...
# PRE ALLOCATE
machine_batch do
%w(chef web repo ci).each do |m|
machine "#{m}.lab.ci" do
action :allocate
add_machine_options bootstrap_options: PER_MACHINE_BOOTSTRAP_OPTIONS[m] || {}
retries 1
end
end
end
# BOOTSTRAP/CONVERGE NODE
machine 'chef.lab.ci' do
ohai_hints 'openstack' => '{}'
attribute %w(apt compile_time_update), true
attribute %w(resolver nameservers), search(:node, 'role:dns').map \
{ |x| x.automatic.ipaddress } unless search(:node, 'role:dns').empty?
recipe 'lab_base::default'
role 'chef'
add_machine_options bootstrap_options: PER_MACHINE_BOOTSTRAP_OPTIONS['chef'] || {}
files('/etc/chef/encrypted_data_bag_secret' => "#{dtbgsec}")
action [:ready, :converge]
converge true
end
Chef-Zero
Simple, easy-run, fast-start in-memory Chef server for testing and solo purposes
chef-provisioning, chef-zero
Chef-Provisioning
A collection of resources that enable the creation of machines and machine infrastructures using the chef-client.
Ecosystem
Analytics & audits
# in audit recipe
control_group 'ssh' do
control 'password authentication' do
it 'is disabled' do
expect(file('/etc/ssh/sshd_config')).to_not \
match(/^PasswordAuthentication\s+yes/i)
end
end
end
# in analytics rule
rules 'failed-audit'
rule on run_control_group
when
status != 'success'
then
notify('slack', "{
'username': 'Audit Alarm', 'text': "{{message.name}} (cookbook {{message.cookbook_name}})\n
had '{{message.number_failed}}' failed audit test(s) on node '{{message.run.node_name}}'"
}")
end
end
Chef WebUI
Hands On
Chef workflow
"simple" web app
PartA, develpop cookbooks/repo/recipes
PartB - execute serverspec/inspec audit
lab_ cookbooks
- lab_gitlab
- lab_audit
- lab_backup
- lab_base
- lab_openvpn
- lab_chef-server
- lab_bind
- lab_drone
- lab_nagios
- lab_firewall
- lab_trac
- lab_redmine
- lab_openldap
- lab_artifactory
- lab_maven
- lab_docker
- lab_subversion
- lab_postfix
- lab_jenkins
- lab_sensu
- lab_oracle
- lab_websphere
- lab_owncloud
- ...
wrappers
- percona ( MySQL XtraDB Percona cluster )
- sensu ( Sensu monitoring + Graphite, Grafana )
- gitzone ( git managed BIND zones )
applications
chef-repo-#{ENV}
- Chef cookbook as many others
- Contain definition for roles, environments, data bags
-
Contain chef provisioning recipes
- for applications, environments, etc ...
- Contain chef audit recipes
- ChefSpec, ServerSpec/Specinfra V2 tests
- KitchenCI integration tests
- Rakefile, scripts, core configuration
chef-repo
workflow
rake -T
rake berks # Berks install/update/vendor
rake bootstrap:chefonly # Boostrap CHEF ONLY with SSH
rake bootstrap:ci # Boostrap CI against OpenStack env
rake bootstrap:gtshub # Boostrap gtshub using SSH
rake bootstrap:setup # Bootstrap local cfg for chef-provisioning setup
rake bootstrap:test # Boostrap TEST against OpenStack env
rake bootstrap:vagrant # Bootstrap Vagrant env
rake chef:hosts # Create a hosts.yml file based on knife information
rake chef:hosts_local # Create a hosts.yml file based on local ChefZero information
rake chef:nodes_local # Print ip + fqdn from nodes records
rake convert # Convert ruby classes to json
rake convert:env_to_json # Convert ruby environments from ruby to json, creating/overwriting json files
rake convert:metadata_to_json # Convert all metadata from ruby to json
rake convert:role_to_json # Convert ruby roles from ruby to json, creating/overwriting json files
rake destroy:ci # Destroy CI environment
rake dev:bootstrap # Run task bootstrap:ci in local DEV environment
rake dev:cls # Clean up local chef-zero instances
rake dev:setup # Setup local development ENV variables
rake foodcritic # Foodcritic
rake knife:bootstrap # Bootstrap new node using knife
rake lint # Lint & syntax checks
rake provision:chefonly # Provision CHEF server with SSH
rake provision:ci # Provision CI environment against Openstack
rake provision:gtshub # Provision gtshub environment with SSH
rake provision:test # Provision TEST environment with Openstack
rake rubocop # Run RuboCop style and lint checks
rake rubocop:auto_correct # Auto-correct RuboCop offenses
rake spec # Run serverspec to all hosts
rake vagrant:create # Create the Vagrant machine
rake vagrant:destroy # Stop and delete the Vagrant machine
rake vagrant:login # Log into the Vagrant machine via SSH
rake vagrant:provision # Provision the Vagrant machine with Chef
knife-zero-repo
- For one-time / simple deployments
- Runs Chef-Zero in memory
- Filesystem as persistent storage
- Commands
-
- bootstrap
- converge
- chef_client
- diagnose
Chef cookbook testing
Continuous Integration test Continuous Deployment cycle
- Cookbooks
- Unit test (generated by ChefDK using ChefSpec)
- Integration tests (KitchenCI, ServerSpec)
- Functional tests (ServerSpec, InSpec)
-
Chef-Repo
- Functional tests (Rake, ServerSpec, InfraSpec, InSpec)
- Smoke tests (Integration tests, Audit recipes)
CI Overview
.kitchen.yml
---
# https://docs.chef.io/config_yml_kitchen.html
driver:
name: vagrant
use_sudo: false
provisioner:
name: chef_zero
cookbook_path: ['.','cookbooks', 'test/cookbooks']
platforms:
- name: centos-7.1
run_list:
- recipe[selinux::disabled]
- name: ubuntu-14.04
suites:
- name: default
run_list:
- recipe[lab_base]
- recipe[lab_gitlab::db]
- recipe[lab_gitlab::default]
attributes:
apt:
compile_time_update: true
mysql:
server_root_password: test
gitlab:
database:
host: 127.0.0.1
password: test
# vim: sw=2 ts=2 sts=2
Drone.io
.drone.yml
image: epcim/drone-chef-ci:dind
env:
- CI=drone
- LABREPO=https://gitlab.lab.local/chef-cookbooks
script:
- source /root/.bash_profile
- eval "$(chef shell-init bash)"
- export LANG="en_US.UTF-8"
- export SSL_CERT_FILE=/opt/chefdk/embedded/ssl/certs/cacert.pem
- chef exec berks install
- chef gem install chef-rewind
- wrapdocker
- chef exec rspec spec
- chef exec kitchen test
- chef exec foodcritic "-f any -f ~FC014 -f ~FC015 -f ~FC024 -f ~FC043" .
- chef exec rubocop -a || chef exec rubocop .
notify:
slack:
webhook_url: 'https:#hooks.slack.com/services/T0SDDASD43/BSDFD84QY/buISDF436dsfjsdlEoo57qO'
username: 'drone.io'
channel: '#lab-cookboks'
on_started: false
on_success: true
on_failure: false
Drone.io addons
Monitoring & analytics
- Chef Analytics rules
- Chef Audit recipes
- Nagios / Sensu
- ELK stack, Grafana, ...
Only alert on what is actionable
- Get the attention of the right humans
- As few alerts as possible
- Routed to the people who can take action
- Start with the is it up alert
Chef Audit
# it's just a "serverspec"...
control_group 'SSH Service compliance' do
control 'ssh service' do
it 'should be listening on port 22' do
expect(port(22)).to be_listening
end
end
control 'ssh configuration' do
it 'ssh disables root logins' do
expect(file('/etc/ssh/sshd_config')).to contain('PermitRootLogin no')
end
it 'is disabled' do
expect(file('/etc/ssh/sshd_config')).to_not \
match(/^PasswordAuthentication\s+yes/i)
end
end
end
Chef InSpec
# use basic tests
describe package('nginx') do
it { should be_installed }
end
# extend tests with metadata
control '01' do
impact 0.7
title 'Verify nginx service'
desc 'Ensures nginx service is up and running'
describe service('nginx') do
it { should be_enabled }
it { should be_installed }
it { should be_running }
end
end
# implement os dependent tests
web_user = 'www-data'
web_user = 'nginx' if os[:family] == 'centos'
describe user(web_user) do
it { should exist }
Sensu
Grafana dashboards
Let's rock
Demo
Questions
Feedback
Demo
Questions
Feedback
Demo
Questions
Feedback
Backup slides
Dockerfile
FROM jpetazzo/dind
MAINTAINER Petr Michalec <epcim@apealive.net>
RUN apt-get update
RUN apt-get install -qqy curl \
sudo \
git \
mercurial \
subversion \
ca-certificates \
locales \
jq
RUN echo 'en_US.UTF-8 UTF-8'>>/etc/locale.gen
RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV DEBIAN_FRONTEND noninteractive
## CHEF DK ###########################
RUN curl -L https://www.opscode.com/chef/install.sh | sudo bash -s -- -P chefdk
ENV PATH /opt/chefdk/bin:/.chefdk/gem/ruby/2.1.0/bin:/opt/chefdk/embedded/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# Make Chef DK the primary Ruby/Chef development environment.
RUN echo 'eval "$(chef shell-init bash)"' >> ~/.bash_profile
RUN eval "$(chef shell-init bash)"
RUN chef gem install kitchen-docker
RUN chef gem install kitchen-openstack
RUN chef gem install chef-sugar
RUN chef gem install chef-rewind
RUN chef gem install serverspec
RUN chef gem install infratester
# berks pre-fetch some common soup of cookbooks
RUN mkdir /tmp/fake_cookbook; cd $_
RUN echo "name 'fake_cookbook'\nmaintainer 'fake_cookbook'\nlicense 'fake_cookbook'\ndescription 'fake_cookbook'\nversion '0.0.1'" > metadata.rb
RUN echo "source 'https://supermarket.chef.io'\nmetadata\n\n" > Berksfile
RUN echo "cookbook '7-zip'\ncookbook 'apache2'\ncookbook 'apt'\ncookbook 'ark'\ncookbook 'bluepill'\ncookbook 'build-essential'\ncookbook 'certificate'\ncookbook 'chef-client'\ncookbook 'chef_handler'\ncookbook 'chef_ruby'\ncookbook 'chef-sugar'\ncookbook 'chef-vault'\ncookbook 'cron'\ncookbook 'database'\ncookbook 'device-mapper'\ncookbook 'git'\ncookbook 'minitest-handler'\ncookbook 'modules'\ncookbook 'ncurses'\ncookbook 'nginx'\ncookbook 'ntp'\ncookbook 'ohai'\ncookbook 'openssh'\ncookbook 'openssl'\ncookbook 'packagecloud'\ncookbook 'pacman'\ncookbook 'perl'\ncookbook 'rbenv'\ncookbook 'readline'\ncookbook 'resolver'\ncookbook 'resource-control'\ncookbook 'rsyslog'\ncookbook 'ruby'\ncookbook 'ruby_build'\ncookbook 'runit'\ncookbook 'subversion'\ncookbook 'sudo'\ncookbook 'sysctl'\ncookbook 'system'\ncookbook 'ulimit'\ncookbook 'users'\ncookbook 'windows'\ncookbook 'xml'\ncookbook 'yum'\ncookbook 'yum-epel'\ncookbook 'zlib'\n" >> Berksfile
RUN chef exec berks install
RUN cd -
## FIX UP'S ##########################
RUN chmod -R 0440 /etc/sudoers
RUN chmod -R 0440 /etc/sudoers.d
# workaround (drone.io has no way yet to modify this image before git clone happens)
RUN git config --global http.sslverify false
VOLUME /var/lib/docker
CMD ["wrapdocker"]
Paleo man DevOps cult
By Petr Michalec
Paleo man DevOps cult
@epcim's applied practices of Chef Style DevOps Kung-Fu, while building lab, dev, PoC infrastructures by code.
- 3,012