Vision:
Deploy and operate all infrastructures by code, as applications are code too.
As traditional IT becomes "truck" and cloud native is a "car". Practice DevOps and apply cloud architectures to acquire expertise, tools and operational skills to drive.
This means that Chef can:
Chef is a configuration management
and automation platform
chef-provisioning
recipes
knife-softlayer
knife-ec2
knife-google
Chef Development Kit (ChefDK)
$ chef generate app cookbook_name
$ chef generate attribute
$ chef generate recipe
$ chef generate lwrp
$ chef-apply name_of_recipe.rb
$ knife search node "platform:centos AND languages_ruby_version:1.8*"
$ knife search node "chef_environment:production AND platform:ubuntu"
$ knife ssh 'name:* AND NOT name:chef*' 'sudo postmap -p /etc/postfix/transport'
$ knife bootstrap --environment dev --node-name catalog.lab.dev 10.10.50.15 \
--ssh-password passw0rd --run-list "role[base]" --server-url "https://10.10.50.2" \
--json-attributes '{ "hostname": "catalog","name": "catalog.lab.dev", \
"system":{ "short_hostname":"catalog", "domain_name":"lab.dev" } }' \
--secret-file .chef/encrypted_data_bag_secret
$ knife node run_list add ora.kb.dev "role[git], recipe[backup::gitlab]"
$ knife role from file roles/*.rb
$ knife environment from file staging.rb
knife
** KNIFE COMMANDS **
bootstrap, client, configure, cookbook, cookbook site, data bag, delete, deps,
diff, download, edit, environment, exec, list, node, raw, recipe_list, role,
search, serve, show, ssh, ssl check, ssl fetch, status, tag, upload, user
** NODE COMMANDS **
knife node bulk delete REGEX (options)
knife node create NODE (options)
knife node delete NODE (options)
knife node edit NODE (options)
knife node environment set NODE ENVIRONMENT
knife node from file FILE (options)
knife node list (options)
knife node run_list add [NODE] [ENTRY[,ENTRY]] (options)
knife node run_list remove [NODE] [ENTRY[,ENTRY]] (options)
knife node run_list set NODE ENTRIES (options)
knife node show NODE (options)
** SOFTLAYER COMMANDS **
knife softlayer datacenter list (options)
knife softlayer datacenter show DATACENTER
knife softlayer flavor list (options)
knife softlayer global ip list (options)
knife softlayer image list
knife softlayer key pair create
knife softlayer key pair list
knife softlayer server create (options)
knife softlayer server destroy (options)
knife softlayer server list (options)
knife softlayer server relaunch <NODE NAME> [<NODE NAME>]
knife softlayer vlan create
knife softlayer vlan list (options)
knife softlayer vlan show ID (options)
knife plugins
chef supermarket
include_recipe 'apache2'
user 'userdoc' do
supports manage_home: true
comment 'userdoc (bind user)'
gid 'users'
home '/home/userdoc'
password '$1$9p26dAeZ$qZ7P4XW.ASDG4DSD3234vK0'
end
directory '/var/www/userdoc' do
mode 00775
owner 'apache2'
group 'users'
recursive true
end
web_app 'userdoc' do
server_name "udoc.#{node['domain']}"
server_aliases ["userdoc.#{node['system']['domain_name']}"
docroot '/var/www/userdoc'
cookbook 'apache2'
directory_options 'Indexes FollowSymLinks'
end
cookbooks, recipes
cookbook
|
├── .kitchen.yml
├── Berksfile
├── metadata.rb
├── attributes
│ └── default.rb
├── recipes
│ └── default.rb
└── templates
└── default
└── index.html.erb
# metadata.md
name 'gitzone'
maintainer 'Petr Michalec'
maintainer_email 'epcim@apealive.net'
license 'Apache 2.0'
description 'Installs/Configures gitzone managed zone files for Bind'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '1.0.15'
recipe 'gitzone::default', 'Install and fully configure gitzone using the install and configure recipes'
recipe 'gitzone::install', 'Install gitzone scripts'
recipe 'gitzone::configure', 'Configure gitzone (including BIND and default zones)'
recipe 'gitzone::configure_zonefile', 'Deploy zone files'
supports 'ubuntu'
supports 'centos'
depends 'bind'
depends 'git'
depends 'sudo'
depends 'system'
depends 'build-essential'
depends 'simple_iptables'
require 'chef/provisioning'
with_driver 'fog:OpenStack'
...
# PRE ALLOCATE
machine_batch do
%w(chef web repo ci).each do |m|
machine "#{m}.lab.ci" do
action :allocate
add_machine_options bootstrap_options: PER_MACHINE_BOOTSTRAP_OPTIONS[m] || {}
retries 1
end
end
end
# BOOTSTRAP/CONVERGE NODE
machine 'chef.lab.ci' do
ohai_hints 'openstack' => '{}'
attribute %w(apt compile_time_update), true
attribute %w(resolver nameservers), search(:node, 'role:dns').map \
{ |x| x.automatic.ipaddress } unless search(:node, 'role:dns').empty?
recipe 'lab_base::default'
role 'chef'
add_machine_options bootstrap_options: PER_MACHINE_BOOTSTRAP_OPTIONS['chef'] || {}
files('/etc/chef/encrypted_data_bag_secret' => "#{dtbgsec}")
action [:ready, :converge]
converge true
end
Chef-Zero
Simple, easy-run, fast-start in-memory Chef server for testing and solo purposes
chef-provisioning, chef-zero
Chef-Provisioning
A collection of resources that enable the creation of machines and machine infrastructures using the chef-client.
Analytics & audits
# in audit recipe
control_group 'ssh' do
control 'password authentication' do
it 'is disabled' do
expect(file('/etc/ssh/sshd_config')).to_not \
match(/^PasswordAuthentication\s+yes/i)
end
end
end
# in analytics rule
rules 'failed-audit'
rule on run_control_group
when
status != 'success'
then
notify('slack', "{
'username': 'Audit Alarm', 'text': "{{message.name}} (cookbook {{message.cookbook_name}})\n
had '{{message.number_failed}}' failed audit test(s) on node '{{message.run.node_name}}'"
}")
end
end
PartA, develpop cookbooks/repo/recipes
PartB - execute serverspec/inspec audit
rake berks # Berks install/update/vendor
rake bootstrap:chefonly # Boostrap CHEF ONLY with SSH
rake bootstrap:ci # Boostrap CI against OpenStack env
rake bootstrap:gtshub # Boostrap gtshub using SSH
rake bootstrap:setup # Bootstrap local cfg for chef-provisioning setup
rake bootstrap:test # Boostrap TEST against OpenStack env
rake bootstrap:vagrant # Bootstrap Vagrant env
rake chef:hosts # Create a hosts.yml file based on knife information
rake chef:hosts_local # Create a hosts.yml file based on local ChefZero information
rake chef:nodes_local # Print ip + fqdn from nodes records
rake convert # Convert ruby classes to json
rake convert:env_to_json # Convert ruby environments from ruby to json, creating/overwriting json files
rake convert:metadata_to_json # Convert all metadata from ruby to json
rake convert:role_to_json # Convert ruby roles from ruby to json, creating/overwriting json files
rake destroy:ci # Destroy CI environment
rake dev:bootstrap # Run task bootstrap:ci in local DEV environment
rake dev:cls # Clean up local chef-zero instances
rake dev:setup # Setup local development ENV variables
rake foodcritic # Foodcritic
rake knife:bootstrap # Bootstrap new node using knife
rake lint # Lint & syntax checks
rake provision:chefonly # Provision CHEF server with SSH
rake provision:ci # Provision CI environment against Openstack
rake provision:gtshub # Provision gtshub environment with SSH
rake provision:test # Provision TEST environment with Openstack
rake rubocop # Run RuboCop style and lint checks
rake rubocop:auto_correct # Auto-correct RuboCop offenses
rake spec # Run serverspec to all hosts
rake vagrant:create # Create the Vagrant machine
rake vagrant:destroy # Stop and delete the Vagrant machine
rake vagrant:login # Log into the Vagrant machine via SSH
rake vagrant:provision # Provision the Vagrant machine with Chef
Continuous Integration test Continuous Deployment cycle
---
# https://docs.chef.io/config_yml_kitchen.html
driver:
name: vagrant
use_sudo: false
provisioner:
name: chef_zero
cookbook_path: ['.','cookbooks', 'test/cookbooks']
platforms:
- name: centos-7.1
run_list:
- recipe[selinux::disabled]
- name: ubuntu-14.04
suites:
- name: default
run_list:
- recipe[lab_base]
- recipe[lab_gitlab::db]
- recipe[lab_gitlab::default]
attributes:
apt:
compile_time_update: true
mysql:
server_root_password: test
gitlab:
database:
host: 127.0.0.1
password: test
# vim: sw=2 ts=2 sts=2
image: epcim/drone-chef-ci:dind
env:
- CI=drone
- LABREPO=https://gitlab.lab.local/chef-cookbooks
script:
- source /root/.bash_profile
- eval "$(chef shell-init bash)"
- export LANG="en_US.UTF-8"
- export SSL_CERT_FILE=/opt/chefdk/embedded/ssl/certs/cacert.pem
- chef exec berks install
- chef gem install chef-rewind
- wrapdocker
- chef exec rspec spec
- chef exec kitchen test
- chef exec foodcritic "-f any -f ~FC014 -f ~FC015 -f ~FC024 -f ~FC043" .
- chef exec rubocop -a || chef exec rubocop .
notify:
slack:
webhook_url: 'https:#hooks.slack.com/services/T0SDDASD43/BSDFD84QY/buISDF436dsfjsdlEoo57qO'
username: 'drone.io'
channel: '#lab-cookboks'
on_started: false
on_success: true
on_failure: false
# it's just a "serverspec"...
control_group 'SSH Service compliance' do
control 'ssh service' do
it 'should be listening on port 22' do
expect(port(22)).to be_listening
end
end
control 'ssh configuration' do
it 'ssh disables root logins' do
expect(file('/etc/ssh/sshd_config')).to contain('PermitRootLogin no')
end
it 'is disabled' do
expect(file('/etc/ssh/sshd_config')).to_not \
match(/^PasswordAuthentication\s+yes/i)
end
end
end
# use basic tests
describe package('nginx') do
it { should be_installed }
end
# extend tests with metadata
control '01' do
impact 0.7
title 'Verify nginx service'
desc 'Ensures nginx service is up and running'
describe service('nginx') do
it { should be_enabled }
it { should be_installed }
it { should be_running }
end
end
# implement os dependent tests
web_user = 'www-data'
web_user = 'nginx' if os[:family] == 'centos'
describe user(web_user) do
it { should exist }
FROM jpetazzo/dind
MAINTAINER Petr Michalec <epcim@apealive.net>
RUN apt-get update
RUN apt-get install -qqy curl \
sudo \
git \
mercurial \
subversion \
ca-certificates \
locales \
jq
RUN echo 'en_US.UTF-8 UTF-8'>>/etc/locale.gen
RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV DEBIAN_FRONTEND noninteractive
## CHEF DK ###########################
RUN curl -L https://www.opscode.com/chef/install.sh | sudo bash -s -- -P chefdk
ENV PATH /opt/chefdk/bin:/.chefdk/gem/ruby/2.1.0/bin:/opt/chefdk/embedded/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# Make Chef DK the primary Ruby/Chef development environment.
RUN echo 'eval "$(chef shell-init bash)"' >> ~/.bash_profile
RUN eval "$(chef shell-init bash)"
RUN chef gem install kitchen-docker
RUN chef gem install kitchen-openstack
RUN chef gem install chef-sugar
RUN chef gem install chef-rewind
RUN chef gem install serverspec
RUN chef gem install infratester
# berks pre-fetch some common soup of cookbooks
RUN mkdir /tmp/fake_cookbook; cd $_
RUN echo "name 'fake_cookbook'\nmaintainer 'fake_cookbook'\nlicense 'fake_cookbook'\ndescription 'fake_cookbook'\nversion '0.0.1'" > metadata.rb
RUN echo "source 'https://supermarket.chef.io'\nmetadata\n\n" > Berksfile
RUN echo "cookbook '7-zip'\ncookbook 'apache2'\ncookbook 'apt'\ncookbook 'ark'\ncookbook 'bluepill'\ncookbook 'build-essential'\ncookbook 'certificate'\ncookbook 'chef-client'\ncookbook 'chef_handler'\ncookbook 'chef_ruby'\ncookbook 'chef-sugar'\ncookbook 'chef-vault'\ncookbook 'cron'\ncookbook 'database'\ncookbook 'device-mapper'\ncookbook 'git'\ncookbook 'minitest-handler'\ncookbook 'modules'\ncookbook 'ncurses'\ncookbook 'nginx'\ncookbook 'ntp'\ncookbook 'ohai'\ncookbook 'openssh'\ncookbook 'openssl'\ncookbook 'packagecloud'\ncookbook 'pacman'\ncookbook 'perl'\ncookbook 'rbenv'\ncookbook 'readline'\ncookbook 'resolver'\ncookbook 'resource-control'\ncookbook 'rsyslog'\ncookbook 'ruby'\ncookbook 'ruby_build'\ncookbook 'runit'\ncookbook 'subversion'\ncookbook 'sudo'\ncookbook 'sysctl'\ncookbook 'system'\ncookbook 'ulimit'\ncookbook 'users'\ncookbook 'windows'\ncookbook 'xml'\ncookbook 'yum'\ncookbook 'yum-epel'\ncookbook 'zlib'\n" >> Berksfile
RUN chef exec berks install
RUN cd -
## FIX UP'S ##########################
RUN chmod -R 0440 /etc/sudoers
RUN chmod -R 0440 /etc/sudoers.d
# workaround (drone.io has no way yet to modify this image before git clone happens)
RUN git config --global http.sslverify false
VOLUME /var/lib/docker
CMD ["wrapdocker"]