Chef workshop

LEVEL DEVil

Petr Michalec

Developer. IBMer. Vi(m) lover. DevOps kid. Runner. Performing cloud infrastructure and application architecture with passion for the edge thing.

 

Reach me on Twitter as @epcim

 

Something about you

Background | Practice | Expectations

Workshop agenda

Morning Intro
DevOps intro (J.Svoboda)
DevOps kung-fu (P.Michalec)
Review day 1
InSpec, Compliance tests
Chef Lab II.
Chef intro
Prerequisites / Lab 0.
Chef workflow, CI/CD
Chef Lab III.
Afternoon Cookbook development
​​Chef Lab I.
Chef Delivery, Analytics
Chef Lab IV.
 
TDD, Advanced practices
Chef Lab II.

Experiments ...
T Day 1 Day 2

Day 1

Buzzword

By Definition

You can't buy/hire/adopt it

IBM

DevOps is an enterprise capability for continuous software delivery that enables clients to seize market opportunities and reduce time to customer feedback

Pick #1

Chef.io

A cultural and professional movement, focused on how we build and operate high velocity organizations, born from the experiences of its practitioners. 

Pick #2

Three ways

The principles that all of the DevOps patterns can be derived from

Pick #3

Forget DEV to OPS !

Practices

  • Automation
  • Continuous Delivery
  • Collaboration across teams
  • Extended visibility
  • Amplified feedback loops, rapid feedback

Feedback loop

  • Do something
  • Test, Measure
  • Analyze
  • Correct
  • Repeat

Business value

  • Enhanced customer experience
  • Increased capacity to innovate    
  • Faster time to value

What did the Waterfall, Agile, Scrum done wrong?

Buzzwords

IBM DevOps RA

  • Reference Architecture
  • DevOps Capabilities
  • Adoption Patterns
  • Product assigment
  • Solutions for "inovative edge" architerctures

Links and presenatation in workshop materials.

IBM DevOps by 

Jan Svoboda

Break

(~ 5 min)

Prerequisites check

Lab 0. 

Prerequisites check

Lab 0. 

What are hot trends?

What are hot trends?

  • From traditional IT to microservices

  • API loosely coupled architectures and micro services

  • Technology shift on frontends

  • Design for speed and simplicity

  • Cross platform requirements (Go, mobile platforms)

  • Measure for feedback, data analytics, visualization

  • Stream data

  • Dynamic configuration

What do we need then?

Deploy and operate all infrastructures by code, as applications are code too.

As traditional IT becomes "truck" and cloud native is a "car". Practice DevOps and apply cloud architectures to acquire expertise, tools and operational skills to drive.

My Vision

Personal picks

  • DevOps approach
  • Life-cycle in mind
  • Meet deadlines
  • Independent dependability
  • One artifact for all env.
  • Same deployment process
  • Four eyes rule; visibility
  • Team is not isolated unit
  • Share

Applied           Style DevOps Kung Fu

Originaly by  Adam Jacob / CTO of Chef / @adamhjk

 

Chef Style DevOps Kung Fu

Creative Commons License

Break

(~ 5 min)

Conf. mgmt. tools

Conf. mgmt. tools

       Orchestration

 

 

 

Infrastructure as code

OS/Application configuration

Angle of view matters

  • Point & shoot or long-term manage?
  • Traditional IT apps & infrastrures vs. cloud?
  • Local or distributed (hybrid) ?
  • Platform specific?
    • PaaS as (Heroku/CF/Bluemix)
    • Containers (Docker, rkt, k8s )
  • Dynamic configuration and service discovery
  • Level of orchestration
  • . . .
  • . . .

Recognized classic tools

  • Chef
  • Puppet
  • Ansible
  • SaltStack

 

w/Agent

Platforms supproted

Language

DSL

Encryption

Config store

ACL

Frequency

Conf. mgmt. tools

Comparison

Chef | Puppet | Ansible | SaltStack | uDeploy| GRTE | CloudFactory | Lotus Automator | . . .

 

 To be considered

  • Whether it's agent less  (Ansible) or need an agent at endpoint (Chef, puppet)

  • The level of abstraction Fabric vs. Chef|Salt|Puppet

  • Whether the aim is to install or install & maintain

  • Level of customized / automated configuration you need to apply vs. how advanced and complex are available community resources (cookbooks etc)

  • Integration to in house tools (CI,...)

  • Platforms and environments involved

  • Configure vs. deploy

Backup slides

Platforms

Traditional IT

IaaS - OpenStack

IBM Cloud Orchestrator, RedHat CloudForms, HP Helion, ...

PaaS - Cloud Foundry

Heroku, IBM Bluemix

Patterns

Heat patterns, IBM PureApps

Orchestration

  • containers provide the unit of execution for applications in PaaS
  • Container orchestration (Kubernetes, Swarm) vs. Distributed configuration (etcd, consul, ...)

Backup slide

Containers

  • Focused on infrastructure -OpenStack
  • Focused on application - UrbanCode, Chef Delivery, ...

Deployment tools

Chef intro

This means that Chef can:

  • Build  Deterministic Servers 
  • Deploy and Configure Applications
  • Deploy Infrastructures
  • Update Applications
  • Run audits and notifications

Chef is a configuration management

and automation platform

Overview

How it works

chef-provisioning

recipes

knife-softlayer

knife-ec2

knife-google

Ecosystem

Chef Development Kit (ChefDK)

$ chef generate app cookbook_name
$ chef generate attribute
$ chef generate recipe
$ chef generate lwrp

$ chef-apply name_of_recipe.rb
  • chef
  • chefspec
  • chef-provisioning
  • chef-provisioning-aws
  • chef-provisioning-fog
  • Inspec 
  • chef-vault
  • knife-spork
  • rubocop
  • foodcritic

Ecosystem

include_recipe 'apache2'

user 'userdoc' do
  supports manage_home: true
  comment 'userdoc (bind user)'
  gid 'users'
  home '/home/userdoc'
  password '$1$9p26dAeZ$qZ7P4XW.ASDG4DSD3234vK0'
end

directory '/var/www/userdoc' do
  mode 00775
  owner 'apache2'
  group 'users'
  recursive true
end

web_app 'userdoc' do
  server_name "udoc.#{node['domain']}"
  server_aliases ["userdoc.#{node['system']['domain_name']}"
  docroot '/var/www/userdoc'
  cookbook 'apache2'
  directory_options 'Indexes FollowSymLinks'
end

cookbooks, recipes

cookbook
|
├── .kitchen.yml
├── Berksfile
├── metadata.rb
├── attributes
│   └── default.rb
├── recipes
│   └── default.rb
└── templates
    └── default
        └── index.html.erb
# metadata.md

name 'gitzone'
maintainer 'Petr Michalec'
maintainer_email 'epcim@apealive.net'
license 'Apache 2.0'
description 'Installs/Configures gitzone managed zone files for Bind'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '1.0.15'

recipe 'gitzone::default',   'Install and fully configure gitzone using the install and configure recipes'
recipe 'gitzone::install',   'Install gitzone scripts'
recipe 'gitzone::configure', 'Configure gitzone (including BIND and default zones)'
recipe 'gitzone::configure_zonefile', 'Deploy zone files'

supports 'ubuntu'
supports 'centos'

depends 'bind'
depends 'git'
depends 'sudo'
depends 'system'
depends 'build-essential'
depends 'simple_iptables'

Ecosystem

chef supermarket

Ecosystem

$ knife search node "platform:centos AND languages_ruby_version:1.8*"

$ knife search node "chef_environment:production AND platform:ubuntu"

$ knife ssh 'name:* AND NOT name:chef*' 'sudo postmap -p /etc/postfix/transport'

$ knife bootstrap --environment dev --node-name catalog.lab.dev 10.10.50.15 \
    --ssh-password passw0rd --run-list "role[base]" --server-url "https://10.10.50.2" \
    --json-attributes '{ "hostname": "catalog","name": "catalog.lab.dev", \
    "system":{ "short_hostname":"catalog", "domain_name":"lab.dev" } }' \
    --secret-file .chef/encrypted_data_bag_secret


$ knife node run_list add ora.kb.dev "role[git], recipe[backup::gitlab]"

$ knife role from file roles/*.rb

$ knife environment from file staging.rb

knife

** KNIFE COMMANDS **

bootstrap, client, configure, cookbook, cookbook site, data bag, delete, deps,
diff, download, edit, environment, exec, list, node, raw, recipe_list, role,
search, serve, show, ssh, ssl check, ssl fetch, status, tag, upload, user

** NODE COMMANDS **
knife node bulk delete REGEX (options)
knife node create NODE (options)
knife node delete NODE (options)
knife node edit NODE (options)
knife node environment set NODE ENVIRONMENT
knife node from file FILE (options)
knife node list (options)
knife node run_list add [NODE] [ENTRY[,ENTRY]] (options)
knife node run_list remove [NODE] [ENTRY[,ENTRY]] (options)
knife node run_list set NODE ENTRIES (options)
knife node show NODE (options)

Ecosystem

** SOFTLAYER COMMANDS **

knife softlayer datacenter list (options)
knife softlayer datacenter show DATACENTER
knife softlayer flavor list (options)
knife softlayer global ip list (options)
knife softlayer image list
knife softlayer key pair create
knife softlayer key pair list
knife softlayer server create (options)
knife softlayer server destroy (options)
knife softlayer server list (options)
knife softlayer server relaunch <NODE NAME> [<NODE NAME>]
knife softlayer vlan create
knife softlayer vlan list (options)
knife softlayer vlan show ID (options)
  • knife-softlayer
  • knife-ec2
  • knife-azure
  • knife-windows

knife plugins

Ecosystem

require 'chef/provisioning'
with_driver 'fog:OpenStack'

...

# PRE ALLOCATE
machine_batch do
  %w(chef web repo ci).each do |m|
    machine "#{m}.lab.ci" do
      action :allocate
      add_machine_options bootstrap_options: PER_MACHINE_BOOTSTRAP_OPTIONS[m] || {}
      retries 1
    end
  end
end

# BOOTSTRAP/CONVERGE NODE
machine 'chef.lab.ci' do
  ohai_hints 'openstack' => '{}'
  attribute %w(apt compile_time_update), true
  attribute %w(resolver nameservers), search(:node, 'role:dns').map \
    { |x| x.automatic.ipaddress } unless search(:node, 'role:dns').empty?
  recipe 'lab_base::default'
  role 'chef'
  add_machine_options bootstrap_options: PER_MACHINE_BOOTSTRAP_OPTIONS['chef'] || {}
  files('/etc/chef/encrypted_data_bag_secret' => "#{dtbgsec}")
  action [:ready, :converge]
  converge true
end

​Chef-Zero

Simple, easy-run, fast-start in-memory Chef server for testing and solo purposes

chef-provisioning, chef-zero

​Chef-Provisioning

A collection of resources that enable the creation of machines and machine infrastructures using the chef-client.

Ecosystem

Analytics & audits

# in audit recipe
control_group 'ssh' do
  control 'password authentication' do
    it 'is disabled' do
      expect(file('/etc/ssh/sshd_config')).to_not \
        match(/^PasswordAuthentication\s+yes/i)
    end
  end
end

# in analytics rule
rules 'failed-audit'
  rule on run_control_group
  when
    status != 'success'
  then
    notify('slack', "{
      'username': 'Audit Alarm', 'text': "{{message.name}} (cookbook {{message.cookbook_name}})\n
      had '{{message.number_failed}}' failed audit test(s) on node '{{message.run.node_name}}'"
    }")
  end
end

Chef WebUI

Tools

Preffered languages and tools that fit the job

  • Consul
  • ELK stack
  • Grafana
  • Swarm, K8s
  • Bluemix

To be considered ...

  • Orchestration
    • IBM Urban Code *
    • Heat, TOSCA, ...
  • Log, Metrics
    • Collection
    • Analytics
  • Container platforms
  • Continuous Delivery
    • ​CI/CD
  • Infrastructure at scale
    • ​Storage, Networking
  • Service
    • ​Discovery
    • Registration

Backup slides

Vagrant

Kibana (ELK stack)

Grafana

Consul.io

Consul.io

Break

(~ 5 min)

Cookbooks & deployment

Cookbook

  • attributes
  • definitions
  • files
  • libraries
  • recipes
  • resources
  • templates
  • metadata
    • name, version, desc.
    • dependencies
  • tests
    • unit tests
    • integration tests
  • Policyfile

Chef repository

  • .chef
    • keys, certs, ...
    • knife.rb
  • data bags
  • environments
  • roles
  • Policyfiles
  • Deployment "scripts"
    • Rakefile
    • *.sh
  • May contain
    • another cookbooks
    • recipes, attr., files, ...
    • Berksfile
    • Gemfile

Cookbook types

  • base, environment
  • application
  • library
  • wrapper

Chef repo types

  • Monolitic, chef repository with all cookbooks in "cookbooks" directory
  • Berkshelf, cookbooks has own repos. Berkshelf solve dependency, download cookbooks and manage Berksfile.

 

Attribute types

  • default
  • force_default
  • normal
  • override
  • force_override
  • automatic (ohai)

Attribute source

  • environment
  • role
  • cookbook
  • automatic
  • node
    • attributes (current state)

Env. + Roles

  • to define
    • attributes
    • run_list
  • takes advantage of attribute precedence
  • example roles: base, web, database, mail, ...
  • example environments
    • dev, test, staging, prod
    • per application

Policyfiles

  • new approach
  • out of beta, approaching completion of the "1.0" feature set
  • classic env + roles will remain
  • policy
    • group
    • name
  • named run_lists
  • multiple sources

Deployment

  • bootstrap node
    • install chef-client
    • (register w/chef-server)
  • chef-client
    • apply run_list
    • synchronise cookboks
    • merge attributes
    • run converge
    • update chef-server

run_list

  • role
    • run_list
    • env_run_list
  • policy
    • run_list
  • node
    • run_list
    • expanded_run_list

Cookbook lifecycle

DEV workflow

  • Develop cookbook  (recipes, ...)
  • Lint
  • Syntax
  • Verify locally
  • Upload code to feature branch in repository
  • If passing CI, merge with master / tag
  • Consume by another cookbook ( or repo )

Chef workflow

"simple" web app

PartA, develpop cookbooks/repo/recipes

https://asciinema.org/a/31362

 

PartB - execute serverspec/inspec audit

https://asciinema.org/a/31360

Chef Lab I.

Break

(as needed)

Cookbook TDD

Training, docs

Cookbook testing

  • ​Cookbooks
    • Unit test (generated by ChefDK using ChefSpec)
    • Integration tests (KitchenCI, ServerSpec, InSpec)
    • Functional tests (Bats, ServerSpec, InSpec)
  • Chef-Repo
    • Functional tests (Rake,ServerSpec,InfraSpec,InSpec, ...)
    • Smoke tests (Integration tests, Audit profiles)

You are what you eat

Foodcritic.io

http://www.foodcritic.io

 

Chef Style Guide

https://docs.chef.io/ruby.html

Integration tests

Continuous Integration test Continuous Deployment cycle

Test Kitchen

Test Kitchen

.kitchen.yml

---
# https://docs.chef.io/config_yml_kitchen.html

driver:
    name: vagrant
    use_sudo: false

provisioner:
  name: chef_zero
  cookbook_path: ['.','cookbooks', 'test/cookbooks']

platforms:
    - name: centos-7.1
      run_list:
        - recipe[selinux::disabled]
    - name: ubuntu-14.04

suites:
  - name: default
    run_list:
        - recipe[lab_base]
        - recipe[lab_gitlab::db]
        - recipe[lab_gitlab::default]
    attributes:
        apt:
            compile_time_update: true
        mysql:
          server_root_password: test
        gitlab:
          database:
            host: 127.0.0.1
            password: test

# vim: sw=2 ts=2 sts=2

Test dir. structure

$ cd <cookbook>

$ tree

/test
├── chef
│   ├── data_bags
│   │   └── users
│   |       └── testuser.json
│   └── encrypted_data_bag_secret
└── integration
    ├── default
    │   ├── serverspec
    │   │   └── default_spec.rb
    │   └── ssh_spec.rb
    └── helpers
        └── serverspec
            └── spec_helper.rb

8 directories, 4 files

ServerSpec

# Serverspec
$ cat default/serverspec/default_spec.rb 

require 'spec_helper'

describe 'base-linux::default' do
  # Serverspec examples can be found at
  # http://serverspec.org/resource_types.html

  describe port(80) do
    it { should be_listening }
  end

  describe command('curl http://localhost') do
    its(:stdout) { should match /Hello, world!/ }
  end

  describe file('/etc/services') do
    its(:selinux_label) { should eq 'system_u:object_r:etc_t:s0' } 
  end 

  describe host('serverspec.org') do
    it { should be_resolvable.by('dns') }
  end

end

InSpec

# InSpec
$ cat test/integration/default/ssh_spec.rb 

only_if do
  command('sshd').exist?
end

describe service('ssh') do
  it { should be_enabled }
  it { should be_running }
end

control 'sshd-11' do
  impact 1.0
  title 'Server: Set protocol version to SSHv2'
  desc 'Set the SSH protocol version to 2'
  describe sshd_config do
    its('Protocol') { should eq('2') }
  end
end

control 'sshd-7' do
  impact 1.0
  title 'Server: Do not permit root-based login with password.'
  desc "To reduce the potential to gain full privileges, do not allow login as root with password"
  describe sshd_config do
    its('PermitRootLogin') { should match(/no|without-password/) }
  end
end

Resources

Chef Lab II.

Dont forget DevOps?

Cooperation

https://******.slack.com/messages/devops-workshop/

 

 

  • Login with your email
  • Promote your progress, ask others, share code

Unit tests

Test cookbook recipes and provided metadata

Spec dir. structure

$ cd <cookbook>

$ tree

/spec
├── spec_helper.rb
└── unit
    └── recipes
        ├── default_spec.rb
        ├── sysctl_spec.rb
        └── users_spec.rb

2 directories, 4 files

Chef-Spec

require 'chefspec'

describe 'cookbook_name::install' do
  let(:chef_run) { ChefSpec::Runner.new.converge(described_recipe) }

  it 'creates a template with attributes' do
    expect(chef_run).to create_template('/tmp/with_attributes').with(
      user: 'user',
      group: 'group',
      backup: false,
    )
  end

  it 'includes resource that have guards that evalute to true' do
    expect(chef_run).to start_service('true_guard')
  end

  it 'includes the `other` recipe' do
    expect(chef_run).to include_recipe('include_recipe::other')
  end

  it 'removes a package with an explicit action' do
    expect(chef_run).to remove_package('explicit_action')
  end

end

Chef-Spec

# Cookbook Name:: base-linux
# Spec:: default
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

require 'spec_helper'

describe 'base-linux::default' do                                           
  context 'When all attributes are default, on an unspecified platform' do
    before do
      stub_command('which sudo').and_return('/usr/bin/sudo')
    end

    let(:chef_run) do
      runner = ChefSpec::ServerRunner.new
      runner.node.set['virtualization']['system'] = 'vmware'
      runner.converge(described_recipe)
    end

    it 'converges successfully' do
      expect { chef_run }.to_not raise_error
    end
  end
end

Resources

Chef workflow

"simple" web app

PartA, develpop cookbooks/repo/recipes

https://asciinema.org/a/31362

 

PartB - execute serverspec/inspec audit

https://asciinema.org/a/31360

Break

(~ 5 min)

Chef Lab III.

Review


Questions


Feedback

Day 2

Recap

  • DevOps is "buzzword"
  • Create a cookbook (chef generate cookbook NAME)
  • update medatada.rb (dependencies)
  • if anything from ./path or git:/ update Berksfile
  • write a recipe, add some attributes
  • write a .kitchen.yml (test with "kitchen test")
  • write unit tests in "spec" folder
  • write integration tests in InSpec lang. in "test" folder

Tools mentioned

  • packer.io to build images
  • Vagrant (an virtualization layer abstraction)
  • KitchenCI is your local, friendly,  CI engine
  • InSpec/ServerSpec libraries to check infrastructure
  • ChefSpec library for cookbook unit tests

Remember

  • Presentation and Lab material are
    • written as refrence with links to proper place if you want to lear more and go deep
    • there are chef webinar available for cookbook essential stuff, TDD, node compliance (InSpec), etc..

If shit happens

  • RTFM
  • Open Browser against Github and source code
    • (forget time and read)
  • IRC, Gitter, email author

Orace cookboos

  • 15 cookbooks on supermarket

WAS cookboos

wlp_server "myInstance" do
  config ({
            "featureManager" => {
              "feature" => [ "jsp-2.2", "jaxws-2.1" ]
            },
            "httpEndpoint" => {
              "id" => "defaultHttpEndpoint",
              "host" => "*",
              "httpPort" => "9080",
              "httpsPort" => "9443"
            },
            "application" => {
              "id" => "example",
              "name" => "example",
              "type" => "war",
              "location" => "/apps/example.war"
            }
          })
  jvmOptions [ "-Djava.net.ipv4=true" ]
  serverEnv "JAVA_HOME" => "/usr/lib/j2sdk1.7-ibm/"
  bootstrapProperties "default.http.port" => "9080", "default.https.port" => "9443"
  action :create
end

wlp_download_feature "mongodb" do
  name "mongodb-2.0"
  directory "/opt/ibm/wlp/features"
  accept_license true
end

WAS cookboos

ci.chef.wlp
Chef cookbook for installing and managing WebSphere Application Server Liberty Profile #devops
Updated 21 days ago

ci.chef.iim
Chef Cookbook to install, configure and run IBM Installation Manager
Updated on Nov 3, 2015

ci.chef.ihs
Chef cookbook for installing and managing IBM HTTP Server #devops
Updated on Jun 18, 2015

ci.chef.wxs
Chef Cookbook to install and configure WebSphere eXtreme Scale
Updated on Sep 12, 2014

ci.chef.was
Chef cookbook for installing and managing WebSphere Application Server with wsadmin #devops
Updated on Sep 12, 2014

ci.chef.ihs.config
Chef cookbook for configuring IHS WebSphere Plugin
Updated on Sep 12, 2014

ci.chef.wlp.application
Chef cookbook for deploying applications to the WebSphere Application Server Liberty Profile #devops
Updated on Sep 12, 2014

ci.chef.wlp.samples

UCD plugins

Debug RSPEC

# cd <repo>/cookbooks/base-linux

# cat spec/spec_helper.rb

require 'chefspec'
require 'chefspec/berkshelf'

RSpec.configure do |config|

  # Specify the Chef log_level (default: :warn)
  config.log_level = :debug

end

Debug RSPEC

# rspec spec |wc -l

348945

Debug RSPEC

System cookbook in place, still passing UNIT tests

Chef DK versions

# chef exec chef -v

Chef Development Kit Version: 0.10.0
chef-client version: 12.6.0
berks version: 4.0.1
kitchen version: 1.5.0

# berks

Resolving cookbook dependencies...
Fetching 'base-linux' from source at .
Using apparmor (2.0.1)
Using apt (3.0.0)
Using base-linux (0.1.0) from source at .
Using chef-sugar (3.3.0)
Using chef_handler (1.3.0)
Using compat_resource (12.7.3)
Using cron (1.7.5)
Using firewall (2.4.0)
Using hostsfile (2.4.5)
Using motd (0.6.3)
Using ntp (1.10.1)
Using ohai (2.1.0)
Using selinux (0.9.0)
Using sudo (2.9.0)
Using sysctl (0.7.0) from https://github.com/svanzoest-cookbooks/sysctl (at master)
Using system (0.10.1)
Using users (2.0.3)
Using windows (1.39.2)
Using yum (3.10.0)
Using yum-epel (0.6.6)

Where we failed in LABs

  • cooperation?
  • bug visibility?
  • team work?

Get * R * DONE

  • upload your git code to repo
  • compare with my repo
  • make my repo "upstream" and merge ?
  • fork from somebody who did it right?
  • fix issues, make a PR to upstream repo?
  • git init
  • git add .
  • git commit -m "layz boy on comments"
  • #go to https://github.rtp.raleigh.ibm.com/groups/devops-workshop and create your repo
  • git remote add origin "XYZ"
  • git push --set-upstream origin master
  • # browse, fork from somebody else?

everybody else

  • fork this repo to your namespace
  • clone it locally (git clone "https://xxxx")
  • git checkout -b "my-feature-branch"
  • # work here
  • # if you will succed make PR
  • # if you will fail your develpment
    • git checkout -b mergeorigin
    • git merge "<branch name>"
  • promote to slack

hard way, dont't fork, merge with me

  • git remote add upstream git@github.rtp.raleigh.ibm.com:devops-workshop/pmichalec-base-linux.git
  • git fetch upstream
  • git checkout -b test-merge-with-peter
  • git merge -s theirs --no-commit upstream/master # ???
  • # resolve conflicts, commit
  • git commit -m "merged"
  • # merge with your master
  • git checkout master
  • git merge test-merge-with-peter
  • git push

what else

whate about to check commits history at my repo 

 

Fix Lab I/II.

Review


Questions


Feedback

Break

(last today, ask for it when needed)

You are no longer cave man !

Write your

workstation cookbook

Components

  • Home Sick
  • Apt/Yum OR Brew OR Chocolately
  • Pantry cookbook
  • ... TBD ...

Chef Repository

chef-repo

  • Monolitic, chef repository with all cookbooks in "cookbooks" directory
  • Berkshelf way, cookbooks has own repos. Berkshelf solve dependency, download cookbooks and manage Berksfile.

 

  • to upload / download artifacts to chef server
  • place from where you manage nodes
  • env+roles OR completely Policy based

chef-repo-#{ENV}

  • Chef cookbook as many others  **
  • Contain definition for roles, environments, data bags
  • May contain chef provisioning recipes
    • for applications, environments, etc ...
  • May contain chef audit recipes (InSpec)
  • Integration tests ServerSpec/InSpec  
  • Rakefile, scripts, core configuration

knife-zero-repo

  • For one-time / simple deployments
  • Runs Chef-Zero in memory
  • Filesystem as persistent storage
  • Commands
    • bootstrap
    • converge
    • chef_client
    • diagnose 

rake -T

rake berks                     # Berks install/update/vendor
rake bootstrap:chefonly        # Boostrap CHEF ONLY with SSH
rake bootstrap:ci              # Boostrap CI against OpenStack env
rake bootstrap:gtshub          # Boostrap gtshub using SSH
rake bootstrap:setup           # Bootstrap local cfg for chef-provisioning setup
rake bootstrap:test            # Boostrap TEST against OpenStack env
rake bootstrap:vagrant         # Bootstrap Vagrant env
rake chef:hosts                # Create a hosts.yml file based on knife information
rake chef:hosts_local          # Create a hosts.yml file based on local ChefZero information
rake chef:nodes_local          # Print ip + fqdn from nodes records
rake convert                   # Convert ruby classes to json
rake convert:env_to_json       # Convert ruby environments from ruby to json, creating/overwriting json files
rake convert:metadata_to_json  # Convert all metadata from ruby to json
rake convert:role_to_json      # Convert ruby roles from ruby to json, creating/overwriting json files
rake destroy:ci                # Destroy CI environment
rake dev:bootstrap             # Run task bootstrap:ci in local DEV environment
rake dev:cls                   # Clean up local chef-zero instances
rake dev:setup                 # Setup local development ENV variables
rake foodcritic                # Foodcritic
rake knife:bootstrap           # Bootstrap new node using knife
rake lint                      # Lint & syntax checks
rake provision:chefonly        # Provision CHEF server with SSH
rake provision:ci              # Provision CI environment against Openstack
rake provision:gtshub          # Provision gtshub environment with SSH
rake provision:test            # Provision TEST environment with Openstack
rake rubocop                   # Run RuboCop style and lint checks
rake rubocop:auto_correct      # Auto-correct RuboCop offenses
rake spec                      # Run serverspec to all hosts
rake vagrant:create            # Create the Vagrant machine
rake vagrant:destroy           # Stop and delete the Vagrant machine
rake vagrant:login             # Log into the Vagrant machine via SSH
rake vagrant:provision         # Provision the Vagrant machine with Chef

knife zero

  • Knife-Zero is not replacement of Knife-Solo. Knife-Zero adds the function which can do a target remotely to the local_mode which is a subset of the Chef-Server/Client environment.
  • https://knife-zero.github.io

 

Run chef-client at remote node with chef-zero(local-mode) via HTTP over SSH port fowarding.

Chef Lab IV.

Tasks

  • Configure your repo & knife.rb
  • Connect to training chef server
  • Upload cookbooks
  • Provision a node

Chef Provisioning

Chef Provisioning

Amazon Web Services A Chef provisioning driver for Amazon Web Services (AWS).
Docker A Chef provisioning driver for Docker.
Fog A Chef provisioning driver for Fog.
Hanlon A Chef provisioning driver for Hanlon.
LXC A Chef provisioning driver for LXC.
Microsoft Azure A Chef provisioning driver for Microsoft Azure.
OpenNebula A Chef provisioning driver for OpenNebula.
SSH A Chef provisioning driver for SSH.
Vagrant A Chef provisioning driver for Vagrant.
vSphere A Chef provisioning driver for VMware vSphere.

Chef Provisioning

machine 'server_a' do
  recipe 'base_recipes'
end

machine 'server_b' do
  recipe 'base_recipes'
  recipe 'theserver'
end


machine_batch do
  machine 'db' do
    recipe 'mysql'
  end
  1.upto(50) do |i|
    machine "#{web}#{i}" do
      recipe 'apache'
    end
  end
end

Chef Provisioning

  • aws_auto_scaling_group
  • aws_cache_cluster
  • aws_cache_replication_group
  • aws_cache_subnet_group
  • aws_cloudsearch_domain
  • aws_dhcp_options
  • aws_ebs_volume
  • aws_eip_address
  • aws_image
  • aws_instance
  • aws_internet_gateway
  • aws_key_pair
  • aws_launch_configuration
  • aws_load_balancer
  • aws_network_acl
  • aws_network_interface
  • aws_rds_instance
  • aws_rds_subnet_group
  • aws_route_table
  • aws_s3_bucket
  • aws_security_group
  • aws_server_certificate
  • aws_sns_topic
  • aws_sqs_queue
  • aws_subnet
  • aws_vpc

Chef Lab VI.

Let's rock

Guess Buzzword ?

Continuous *

  • Integration

  • Deployment

  • Delivery

  • Improve

  • Validation

Continuous

That's topic for another workshop

Best practice in general

  • Orchestration, Configuration, Releases, Provisioning
  • The same process and deployment everywhere
  • Build and publish artifacts, one for all environments
  • Manage releases
    • Handle "cookbook" versions cross environments
  • Handle configuration cross environments
  • Fully automated deployment
  • Fully automated CI tests & compliance verification
  • Hybrid cloud options available

CI / CD Pipeline

  • Developers check out code into their private workspaces. When done, the commit changes to the repository.

  • The CI server monitors the repository, checks out on changes.

  • The CI server builds the system and runs unit and integration tests.

  • The CI server releases deployable artefacts for testing.

  • The CI server assigns a build label to the version of the code it just built.

  • The CI server informs the team of the successful build.

  • If the build or tests fail, the CI server alerts the team.

  • The team fix the issue at the earliest opportunity.
    Continue to continually integrate and test throughout the project.

Continuous Integration

Team Responsibilities

  • Check in frequently
  • Don’t check in broken code
  • Don’t check in untested code
  • Don’t check in when the build is broken
  • Don’t go home after checking in until the system builds

CI Best Practices

Chef.io

  • Chef-Delivery

Alternatives

  • Build a CI pipeline around other in house hosted tools (Jazz.net, Jenkins, TravisCI, ...)
  • Build a CD in Chef-provisioning, UrbanCode, IBM Cloud Orchestrator)
  • Run a custom solution

CI/CD pipeline

CI/CD pipeline

CI/CD pipeline

App A

App C

App B

ENTERPR

I

SE

Example: Drone.io

CI Overview

.drone.yml

cache:
  mount:
  - .berkshelf/cookbooks

env:
  - CI=drone
  - LABREPO=https://git.vums.blueit/chef-cookbooks

build:
  image: epcim/drone-chef-ci
  commands:

# RUN TESTS
  # Rspec
  - chef exec rspec spec
  # Kitchen soap
  - KITCHEN_LOCAL_YAML=.kitchen.cloud.yml chef exec kitchen test    || E=$?; test -n $E && /bin/sleep 300  #--concurrency 2
  - KITCHEN_LOCAL_YAML=.kitchen.cloud.yml chef exec kitchen destroy
  # Exit on fail
  - test ${E:-0} -ne 0 && exit $E

# RUN LINT CHECKERS
  - chef exec foodcritic "-f any -f ~FC014 -f ~FC015 -f ~FC024 -f ~FC043" .
  - chef exec rubocop -a || chef exec rubocop . #|| echo "ignoring failed rubocop check"

# TAG SUCCESSFUL VERSION
  - VERSION=`ruby -r 'chef/cookbook/metadata' -e "m=Chef::Cookbook::Metadata.new;m.from_file('metadata.rb');puts m.version"`
  - git config remote.origin.url `git config --get remote.origin.url | sed "s/https:\/\//https:\/\/$GIT_USERNAME:$GIT_PASSWORD@/"`
  - git tag -a v${VERSION} -m 'CI tagged successful build'
  - git push origin v${VERSION} 2> /dev/null || echo $?


notify:
  slack:
    webhook_url: 'https:#hooks.slack.com/services/T0SDDASD43/BSDFD84QY/buISDF436dsfjsdlEoo57qO'
    username: 'drone.io'
    channel: '#lab-cookboks'
    on_started: false
    on_success: true
    on_failure: false

Drone.io

Drone.io addons

.drone.yml - full

cache:
  mount:
  - .berkshelf/cookbooks

clone:
  skip_verify: true
  environment:
  - DRONE_GITLAB_SKIP_VERIFY=true
  - GIT_CURL_VERBOSE=1
  - GIT_SSL_NO_VERIFY=1

env:
  - CI=drone
  - LABREPO=https://git.vums.blueit/chef-cookbooks

build:
  image: epcim/drone-chef-ci
  privileged: true
  environment:
  - OS_USERNAME=$$OS_USERNAME
  - OS_PASSWORD=$$OS_PASSWORD
  - OS_TENANT_NAME=$$OS_TENANT_NAME
  - OS_AUTH_URL=$$OS_AUTH_URL
  - BOOTSTRAP_SSH_KEY=$$BOOTSTRAP_SSH_KEY
  - BOOTSTRAP_SSH_PUB=$$BOOTSTRAP_SSH_PUB
  - GIT_USERNAME=$$GIT_USERNAME
  - GIT_PASSWORD=$$GIT_PASSWORD
  - GIT_COMMITTER_NAME=$$GIT_COMMITTER_NAME
  - GIT_COMMITTER_EMAIL=$$GIT_COMMITTER_EMAIL
  commands:
### INITIALIZE CI ENV  ###
  - . /root/.bash_profile
  - eval "$(chef shell-init sh)"
  - export LANG="en_US.UTF-8"
  - export SSL_CERT_FILE=/opt/chefdk/embedded/ssl/certs/cacert.pem
  - chef exec berks install

### INITIALIZE REMOTE CI ENV ###
  - BOOTSTRAP_KEYFILE=/root/.ssh/bootstrap_insecure
  - echo "$BOOTSTRAP_SSH_KEY" >  $BOOTSTRAP_KEYFILE; chmod 0600 $BOOTSTRAP_KEYFILE
  - eval `ssh-agent`
  - ssh-add $BOOTSTRAP_KEYFILE
  - ssh-add ./id_rsa_bootstrap_insecure

  #- chef exec bundle install
  - chef gem install kitchen-openstack

# RUN TESTS
  # Rspec
  - chef exec rspec spec
  # Kitchen soap
  - KITCHEN_LOCAL_YAML=.kitchen.cloud.yml chef exec kitchen test    || E=$?; test -n $E && /bin/sleep 300  #--concurrency 2
  - KITCHEN_LOCAL_YAML=.kitchen.cloud.yml chef exec kitchen destroy
  # Exit on fail
  - test ${E:-0} -ne 0 && exit $E

# RUN LINT CHECKERS
  - chef exec foodcritic "-f any -f ~FC014 -f ~FC015 -f ~FC024 -f ~FC043" .
  - chef exec rubocop -a || chef exec rubocop . #|| echo "ignoring failed rubocop check"

# TAG SUCCESSFUL VERSION
  - VERSION=`ruby -r 'chef/cookbook/metadata' -e "m=Chef::Cookbook::Metadata.new;m.from_file('metadata.rb');puts m.version"`
  - git config remote.origin.url `git config --get remote.origin.url | sed "s/https:\/\//https:\/\/$GIT_USERNAME:$GIT_PASSWORD@/"`
  - git tag -a v${VERSION} -m 'CI tagged successful build'
  - git push origin v${VERSION} 2> /dev/null || echo $?


notify:
  slack:
    webhook_url: 'https:#hooks.slack.com/services/T0SDDASD43/BSDFD84QY/buISDF436dsfjsdlEoo57qO'
    username: 'drone.io'
    channel: '#lab-cookboks'
    on_started: false
    on_success: true
    on_failure: false

.drone.yml - backup

image: epcim/drone-chef-ci:dind

env:
    - CI=drone
    - LABREPO=https://gitlab.lab.local/chef-cookbooks

script:
    - source /root/.bash_profile
    - eval "$(chef shell-init bash)"
    - export LANG="en_US.UTF-8"
    - export SSL_CERT_FILE=/opt/chefdk/embedded/ssl/certs/cacert.pem

    - chef exec berks install
    - chef gem install chef-rewind

    - wrapdocker

    - chef exec rspec spec
    - chef exec kitchen test

    - chef exec foodcritic "-f any -f ~FC014 -f ~FC015 -f ~FC024 -f ~FC043" .
    - chef exec rubocop -a || chef exec rubocop .

notify:
  slack:
    webhook_url: 'https:#hooks.slack.com/services/T0SDDASD43/BSDFD84QY/buISDF436dsfjsdlEoo57qO'
    username: 'drone.io'
    channel: '#lab-cookboks'
    on_started: false
    on_success: true
    on_failure: false

Chef Lab VII.

Visibility

  • Monitoring
  • Audit & Compliance
  • Reporting
  • Analytics

Monitoring

  • Healt checks
  • Notification
  • Metrics

Alert on what is actionable

  • Get the attention of the right humans
  • As few alerts as possible
  • Routed to the people who can take action
  • Start with the is it up alert

Chef Audit


# it's just a "serverspec"...

control_group 'SSH Service compliance' do

  control 'ssh service' do
    it 'should be listening on port 22' do
      expect(port(22)).to be_listening
    end
  end

  control 'ssh configuration' do
    it 'ssh disables root logins' do
      expect(file('/etc/ssh/sshd_config')).to contain('PermitRootLogin no')
    end

    it 'is disabled' do
      expect(file('/etc/ssh/sshd_config')).to_not \
        match(/^PasswordAuthentication\s+yes/i)
    end
  end
end


Chef InSpec

# use basic tests
describe package('nginx') do
  it { should be_installed }
end

# extend tests with metadata
control '01' do
  impact 0.7
  title 'Verify nginx service'
  desc 'Ensures nginx service is up and running'
  describe service('nginx') do
    it { should be_enabled }
    it { should be_installed }
    it { should be_running }
  end
end

# implement os dependent tests
web_user = 'www-data'
web_user = 'nginx' if os[:family] == 'centos'

describe user(web_user) do
  it { should exist }

Sensu

Grafana dashboards

Grafana & Sensu

ELK

Chef Delivery

Chef Delivery

Chef Delivery

Chef Delivery

Chef Delivery

Behind the scene is actually a cookbook

https://github.com/chef-cookbooks/delivery-truck 

Chef Compliance

lab_ cookbooks

  • lab_gitlab
  • lab_audit
  • lab_backup
  • lab_base
  • lab_openvpn
  • lab_chef-server
  • lab_bind
  • lab_drone
  • lab_nagios
  • lab_firewall
  • lab_trac
  • lab_redmine
  • lab_openldap
  • lab_artifactory
  • lab_maven
  • lab_docker
  • lab_subversion
  • lab_postfix
  • lab_jenkins
  • lab_sensu
  • lab_oracle
  • lab_websphere
  • lab_owncloud
  • ...

wrappers

  • percona ( MySQL XtraDB Percona cluster )
  • sensu ( Sensu monitoring + Graphite, Grafana )
  • gitzone ( git managed BIND zones )

applications

Hack Session

Recap


Questions


Feedback

Backup slides

Resources used

TBD

  • http://chef.github.io/devops-kungfu
  • https://ianunruh.com/
  • https://www.brianchristner.io

 

Chef workshop - LEVEL DEVil

By Petr Michalec

Chef workshop - LEVEL DEVil

Two day DevOps w/Chef training.

  • 1,435
Loading comments...

More from Petr Michalec