DevOps Meet(up)

How to write community cookbooks and ...

Petr Michalec

Developer. IBMer. Vi(m) lover. DevOps kid. Performing cloud infrastructure and application architecture with passion for the edge thing.

 

Proud Chef user since 2013

 

Reach me on Twitter as @epcim or

 

Before starting a cookbook

  • explore supermarket, github
    • choose best available, latest update, active ...
  • consider to write a generic cookbook (library)
  • consider to share feature PR to community cookbook
  • explore other best practice cookbooks
  • follow community writing style

When and what

Supermarket

Supermarket

  • Cookbooks
  • Tools
    • apps, tools, utils
    • InSpec compliance profiles

Community Stats

2,889 Cookbooks | 67,950 Chefs

Supermarket

Make your lawyers happy

  • Contributor License Agreement
  • Curry bot
  • Git tracks authors, contribution

 

Chef style

You are what you eat ...


# 14 Consider extracting long ruby_block to library
# 15 Consider converting definition to a Custom Resource
# 24 Consider adding platform equivalents
# 43 Prefer new notification syntax

cd <cookbook>
foodcritic --epic-fail any -f ~FC014 -f ~FC015 -f ~FC024 -f ~FC043 .

FC001: Use strings in preference to symbols to access node attributes: ./recipes/root.rb:77
FC019: Access node attributes in a consistent manner: ./recipes/system.rb:39
FC019: Access node attributes in a consistent manner: ./recipes/system.rb:40
FC024: Consider adding platform equivalents: ./recipes/integration.rb:46




Workflow

  • Develop cookbook  (recipes, ...)
    • Update metadata.rb, README, ...
    • Lint, Syntax
    • Verify locally
    • Upload code to feature branch in repository
    • If passing CI, merge with master / tag
    • Consume by another cookbook ( or repo, ... )
  • Publish stable versions on supermarket

Backup Slides

Ecosystem

Chef Development Kit (ChefDK)

$ chef generate app cookbook_name
$ chef generate attribute
$ chef generate recipe
$ chef generate lwrp

$ chef-apply name_of_recipe.rb
  • chef, chef zero
  • chefspec
  • Berkshelf
  • knife
  • ohai
  • inspec 
  • rubocop
  • foodcritic

Use Gemfile to document ruby dependencies

source 'https://rubygems.org'

gem 'chef'
gem 'chef-rewind'
gem 'chef-sugar'
gem 'chef-vault'


group :test do
  # gem 'debugger'
  gem 'rubocop'
  gem 'minitest'
  gem 'minitest-chef-handler'
  gem 'serverspec'
  # gem 'infrataster'
end

group :integration do
  # gem 'test-kitchen'
  gem 'kitchen-docker'
  gem 'kitchen-dokken'
  gem 'kitchen-vagrant'
  # gem 'kitchen-localhost'
end

group :chef-repo do
  gem 'knife-zero'
  gem 'knife-spork'
  gem 'slack-notifier'
  gem 'berkshelf' # , github: "berkshelf/berkshelf" # , tag: "v3.0.0.beta8"     # bleeding edge
end

group :development do

  gem 'chefspec'
  gem 'fauxhai'
  # gem 'chef-zero'
  # gem 'berkshelf', github: 'berkshelf/berkshelf'
  
  # KNIFE
  gem 'knife-spec'
  gem 'knife-softlayer'
  gem 'knife-openstack'
  # gem 'knife-windows'
  # gem 'knife-vsphere'
  # gem 'knife-solo'
  # gem 'knife-solo_data_bag'


  # VAGRANT
  # gem 'vagrant',   github: 'mitchellh/vagrant'
  gem 'vagrant-omnibus', github: 'schisamo/vagrant-omnibus'
  gem 'vagrant-berkshelf' # , github: "berkshelf/vagrant-berkshelf"
  gem 'json', '1.7.7' # needed for conflict resolution
end


group :optional do
  # gem 'knife-cookbook-readme'
  gem 'knife-cookbook-doc'
  gem 'yard-chef'
  gem 'colorize'
end
  • chef-provisioning
  • chef-vault
  • knife-spork
  • < ask for others >
include_recipe 'apache2'

user 'userdoc' do
  supports manage_home: true
  comment 'userdoc (bind user)'
  gid 'users'
  home '/home/userdoc'
  password '$1$9p26dAeZ$qZ7P4XW.ASDG4DSD3234vK0'
end

directory '/var/www/userdoc' do
  mode 00775
  owner 'apache2'
  group 'users'
  recursive true
end

web_app 'userdoc' do
  server_name "udoc.#{node['domain']}"
  server_aliases ["userdoc.#{node['system']['domain_name']}"
  docroot '/var/www/userdoc'
  cookbook 'apache2'
  directory_options 'Indexes FollowSymLinks'
end

cookbooks, recipes

cookbook
|
├── .kitchen.yml
├── Berksfile
├── metadata.rb
├── attributes
│   └── default.rb
├── recipes
│   └── default.rb
└── templates
    └── default
        └── index.html.erb
# metadata.md

name 'gitzone'
maintainer 'Petr Michalec'
maintainer_email 'epcim@apealive.net'
license 'Apache 2.0'
description 'Installs/Configures gitzone managed zone files for Bind'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '1.0.15'

recipe 'gitzone::default',   'Install and fully configure gitzone using the install and configure recipes'
recipe 'gitzone::install',   'Install gitzone scripts'
recipe 'gitzone::configure', 'Configure gitzone (including BIND and default zones)'
recipe 'gitzone::configure_zonefile', 'Deploy zone files'

supports 'ubuntu'
supports 'centos'

depends 'bind'
depends 'git'
depends 'sudo'
depends 'system'
depends 'build-essential'
depends 'simple_iptables'
$ knife search node "platform:centos AND languages_ruby_version:1.8*"

$ knife search node "chef_environment:production AND platform:ubuntu"

$ knife ssh 'name:* AND NOT name:chef*' 'sudo postmap -p /etc/postfix/transport'

$ knife bootstrap --environment dev --node-name catalog.lab.dev 10.10.50.15 \
    --ssh-password passw0rd --run-list "role[base]" --server-url "https://10.10.50.2" \
    --json-attributes '{ "hostname": "catalog","name": "catalog.lab.dev", \
    "system":{ "short_hostname":"catalog", "domain_name":"lab.dev" } }' \
    --secret-file .chef/encrypted_data_bag_secret


$ knife node run_list add ora.kb.dev "role[git], recipe[backup::gitlab]"

$ knife role from file roles/*.rb

$ knife environment from file staging.rb

knife

** KNIFE COMMANDS **

bootstrap, client, configure, cookbook, cookbook site, data bag, delete, deps,
diff, download, edit, environment, exec, list, node, raw, recipe_list, role,
search, serve, show, ssh, ssl check, ssl fetch, status, tag, upload, user

** NODE COMMANDS **
knife node bulk delete REGEX (options)
knife node create NODE (options)
knife node delete NODE (options)
knife node edit NODE (options)
knife node environment set NODE ENVIRONMENT
knife node from file FILE (options)
knife node list (options)
knife node run_list add [NODE] [ENTRY[,ENTRY]] (options)
knife node run_list remove [NODE] [ENTRY[,ENTRY]] (options)
knife node run_list set NODE ENTRIES (options)
knife node show NODE (options)
** SOFTLAYER COMMANDS **

knife softlayer datacenter list (options)
knife softlayer datacenter show DATACENTER
knife softlayer flavor list (options)
knife softlayer global ip list (options)
knife softlayer image list
knife softlayer key pair create
knife softlayer key pair list
knife softlayer server create (options)
knife softlayer server destroy (options)
knife softlayer server list (options)
knife softlayer server relaunch <NODE NAME> [<NODE NAME>]
knife softlayer vlan create
knife softlayer vlan list (options)
knife softlayer vlan show ID (options)
  • knife-softlayer
  • knife-ec2
  • knife-azure
  • knife-windows

knife plugins

require 'chef/provisioning'
with_driver 'fog:OpenStack'

...

# PRE ALLOCATE
machine_batch do
  %w(chef web repo ci).each do |m|
    machine "#{m}.lab.ci" do
      action :allocate
      add_machine_options bootstrap_options: PER_MACHINE_BOOTSTRAP_OPTIONS[m] || {}
      retries 1
    end
  end
end

# BOOTSTRAP/CONVERGE NODE
machine 'chef.lab.ci' do
  ohai_hints 'openstack' => '{}'
  attribute %w(apt compile_time_update), true
  attribute %w(resolver nameservers), search(:node, 'role:dns').map \
    { |x| x.automatic.ipaddress } unless search(:node, 'role:dns').empty?
  recipe 'lab_base::default'
  role 'chef'
  add_machine_options bootstrap_options: PER_MACHINE_BOOTSTRAP_OPTIONS['chef'] || {}
  files('/etc/chef/encrypted_data_bag_secret' => "#{dtbgsec}")
  action [:ready, :converge]
  converge true
end

​Chef-Zero

Simple, easy-run, fast-start in-memory Chef server for testing and solo purposes

chef-provisioning, chef-zero

​Chef-Provisioning

A collection of resources that enable the creation of machines and machine infrastructures using the chef-client.

Cookbook

  • attributes
  • definitions
  • files
  • libraries
  • recipes
  • resources
  • templates
  • metadata.rb
  • spec
    • unit tests
  • test
    • test data, cookboooks
    • verification scripts
    • intended for integration testing

Chef repository

  • .chef
    • keys, certs, ...
    • knife.rb
  • data bags
  • environments
  • roles
  • Policyfiles
  • Deployment "scripts"
    • Rakefile
    • *.sh
  • May contain
    • another cookbooks
    • recipes, attr., files, ...
    • Berksfile
    • Gemfile

Cookbook types

  • base, environment, ...
  • application
  • library
  • wrapper
  • infrastructure, chef-provisioning, (delivery truck)

Chef repo types

  • Monolitic, chef repository with all cookbooks in "cookbooks" directory
  • Berkshelf, cookbooks has own repos. Berkshelf solve dependency, download cookbooks and manage Berksfile.

 

Attribute types

  • default
  • force_default
  • normal
  • override
  • force_override
  • automatic (ohai)

Attribute source

  • policyfile
  • environment
  • role
  • cookbook
  • automatic
  • node
    • attributes (current state)

Backup Slides

Env. + Roles

  • to define
    • attributes
    • run_list
  • takes advantage of attribute precedence
  • example roles: base, web, database, mail, ...
  • example environments
    • dev, test, staging, prod
    • per application

Policyfiles

  • new approach
  • out of beta, approaching completion of the "1.0" feature set
  • classic env + roles will remain
  • policy
    • group
    • name
  • named run_lists
  • multiple sources
  • Make cookbook generic, reusable
    • avoid configuring any service that is not strictly related to the particular cookbook (e.g. monitoring, firewall etc.) or at least make it optional (standalone recipes, configurable by attr)
  • Use wrappers, application cookbooks, base, role or environment cookbooks to put offtopic aside
  • ​Use Berkshelf rather than monolitic approach

Best practices

  • generate/write README and CHANGELOG
  • ​be clear what is in scope
    • document roadmap and your intention
    • document what way PR are accepted
    • notify soon prior major changes
  • use "chef generate ..."
  • provide Rakefile
  • provide CI configs (travis, drone.io, kitchen, etc)
  • provide usable defaults (write Policyfile.rb)

Best practices

avoid over-engenered logic in

  • attributes/default.rb
  • templates/*

Best practices

default['service']['branch'] = 'stable'

case node['platform_family']
when 'debian'
  default['service']['repo'] = "https://example.com/debian/#{node['service']['branch']}"
when 'rhel'
  default['service']['repo'] = "https://example.com/rhel/#{node['service']['branch']}"
end
<% unless node["percona"]["server"]["relay_log"].empty? %>
relay-log = <%= node["percona"]["server"]["relay_log"] %>
<% end %>

expire_logs_days = <%= node["percona"]["server"]["expire_logs_days"] %>
max_binlog_size  = <%= node["percona"]["server"]["max_binlog_size"] %>
binlog_format    = <%= node["percona"]["server"]["binlog_format"] %>
slave_net_timeout = <%= node["percona"]["server"]["slave_net_timeout"] %>

<%- if node["percona"]["version"] >= "5.6" %>
binlog_checksum  = <%= node["percona"]["server"]["binlog_checksum"] %>
<% if node["percona"]["server"]["master_verify_checksum"] %>
master_verify_checksum = <%= node["percona"]["server"]["master_verify_checksum"] %>
<% end %>
<% if node["percona"]["server"]["slave_sql_verify_checksum"] %>
slave_sql_verify_checksum = <%= node["percona"]["server"]["master_verify_checksum"] %>
<% end %>
<% end %>

Templates

  • treate attributes <% unless node['cb']['attr'].empty %>
  • consider
    • to loop over attributes: node['cb].map => key=value
    • to use a template from a product (w/comments)
    • to write custom resource providers
  • avoid hundrets of defaults attributes specified
  • avoid (too many) if/else in template

Best practices

Wrapper cookbooks

  • start by forking a community cookbook
  • consider what is your private content and what is worth to share back in community cookbook
  • Usefull to
    • override attributes (!!)
    • custom templates, files
    • integration with other in-house cookbooks
    • additional recipes

Best practices

Re: Templates, haven't seen but

  • Consider touch templates over custom resources
  • Consider to implement in-line style modification that understand config syntax than render templates.
  • Other ideas?

Best practices

When sending a PR

  • lint/syntax with rubocop + foodcritic
  • document cahanges
  • state your intention behind change
  • state your future interest in cookbook
  • do not modify version in metadata.rb

Best practices

Best practices

Other / more to talk

  • Pin versions, pin dependencies, Policyfiles usage
  • Roles, usage and the buzz around
  • Environment per application approach
  • < Discussion is open >

Best practices

http://www.slideshare.net/aestasit/patterns-for-infrastructure-as-code-for-baltic-devops-2016

  • Summary of patterns I
    • Pattern: Reproducable Images
    • Pattern: Secret Isolating
    • Pattern: Ecrypted Secrets
    • Pattern: Infrastructure Component DSL
    • Pattern: Incremental Configuration
    • Pattern: Configuration Composition
    • Pattern: Configuration Discovery
    • Pattern: Extra­Packaging Code
  • Summary of patterns III
    • Antipattern: Golden Image
    • Antipattern: Postponing Secret Isolation
    • Antipattern: "Fancy­File­Copying"
    • Antipattern: Data as Code
    • Antipattern: Ignoring Styling Guidelines
  • Summary of patterns II
    • Pattern: Configuration Data Source
    • Pattern: Metrics as Code
    • Pattern: Control Panel as Code
    • Pattern: Community Module Wrapper
    • Pattern: Infrastructure Query Language
    • Pattern: Automation over Documentation
    • Pattern: Environment Template

  • Summary of patterns IV
    • Antipattern: Not Treating IaC as Code
    • Antipattern: DevOoops
    • Antipattern: Private Fork of a Community Module
    • Antipattern: "Other Stuff"
    • Antipattern: Big Ball of Mud

 

Cookbook TDD

Cookbook TDD

  • ​Cookbooks
    • Unit test (generated by ChefDK using ChefSpec)
    • Functional, Integration tests (KitchenCI,ServerSpec,InSpec)

 

  • Chef-Repo ( infrastructure)
    • Functional, Integration tests (Rake,ServerSpec,Infrataster,InSpec, ...)
    • Audit/Compliance verification (Smoke tests)

Cookbook structure

  • attributes
  • definitions
  • files
  • libraries
  • recipes
  • resources
  • templates
  • metadata.rb
  • spec
    • unit tests
  • test
    • intended for integration testing
    • test data, cookboooks
    • verification scripts

You should not miss

  • .kitchen.yml
  • .kitchen.local.yml
  • .kitchen.cloud.yml

 

  • lib/*
  • .rubocop.rb
  • Rakefile
  • spec
    • spec_helper.rb
  • test
    • chef/*
    • integration/*

Cookbook TTD

Integration tests

Continuous Integration test Continuous Deployment cycle

Test Kitchen

Test Kitchen

.kitchen.yml

---
# https://docs.chef.io/config_yml_kitchen.html

driver:
    name: vagrant
    use_sudo: false

provisioner:
  name: chef_zero
  cookbook_path: ['.','cookbooks', 'test/cookbooks']

platforms:
    - name: centos-7.1
      run_list:
        - recipe[selinux::disabled]
    - name: ubuntu-14.04

suites:
  - name: default
    run_list:
        - recipe[lab_base]
        - recipe[lab_gitlab::db]
        - recipe[lab_gitlab::default]
    attributes:
        apt:
            compile_time_update: true
        mysql:
          server_root_password: test
        gitlab:
          database:
            host: 127.0.0.1
            password: test

# vim: sw=2 ts=2 sts=2

Test dir. structure

$ cd <cookbook>

$ tree

/test
├── chef
│   ├── data_bags
│   │   └── users
│   |       └── testuser.json
│   └── encrypted_data_bag_secret
|
└── integration
    ├── default
    │   ├── serverspec
    │   │   └── default_spec.rb
    │   └── ssh_spec.rb
    └── helpers
        └── serverspec
            └── spec_helper.rb

8 directories, 4 files

ServerSpec

# Serverspec
$ cat default/serverspec/default_spec.rb 

require 'spec_helper'

describe 'base-linux::default' do
  # Serverspec examples can be found at
  # http://serverspec.org/resource_types.html

  describe port(80) do
    it { should be_listening }
  end

  describe command('curl http://localhost') do
    its(:stdout) { should match /Hello, world!/ }
  end

  describe file('/etc/services') do
    its(:selinux_label) { should eq 'system_u:object_r:etc_t:s0' } 
  end 

  describe host('serverspec.org') do
    it { should be_resolvable.by('dns') }
  end

end

InSpec

# InSpec
$ cat test/integration/default/ssh_spec.rb 

only_if do
  command('sshd').exist?
end

describe service('ssh') do
  it { should be_enabled }
  it { should be_running }
end

control 'sshd-11' do
  impact 1.0
  title 'Server: Set protocol version to SSHv2'
  desc 'Set the SSH protocol version to 2'
  describe sshd_config do
    its('Protocol') { should eq('2') }
  end
end

control 'sshd-7' do
  impact 1.0
  title 'Server: Do not permit root-based login with password.'
  desc "To reduce the potential to gain full privileges, do not allow login as root with password"
  describe sshd_config do
    its('PermitRootLogin') { should match(/no|without-password/) }
  end
end

Resources

Unit tests (Chef Spec)

Spec dir. structure

$ cd <cookbook>

$ tree

/spec
├── spec_helper.rb
└── unit
    └── recipes
        ├── default_spec.rb
        ├── sysctl_spec.rb
        └── users_spec.rb

2 directories, 4 files

Chef-Spec

require 'chefspec'

describe 'cookbook_name::install' do
  let(:chef_run) { ChefSpec::Runner.new.converge(described_recipe) }

  it 'creates a template with attributes' do
    expect(chef_run).to create_template('/tmp/with_attributes').with(
      user: 'user',
      group: 'group',
      backup: false,
    )
  end

  it 'includes resource that have guards that evalute to true' do
    expect(chef_run).to start_service('true_guard')
  end

  it 'includes the `other` recipe' do
    expect(chef_run).to include_recipe('include_recipe::other')
  end

  it 'removes a package with an explicit action' do
    expect(chef_run).to remove_package('explicit_action')
  end

end

Chef-Spec

# Cookbook Name:: base-linux
# Spec:: default
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

require 'spec_helper'

describe 'base-linux::default' do                                           
  context 'When all attributes are default, on an unspecified platform' do
    before do
      stub_command('which sudo').and_return('/usr/bin/sudo')
    end

    let(:chef_run) do
      runner = ChefSpec::ServerRunner.new
      runner.node.set['virtualization']['system'] = 'vmware'
      runner.converge(described_recipe)
    end

    it 'converges successfully' do
      expect { chef_run }.to_not raise_error
    end
  end
end

Chef-Spec

# cd <repo>/cookbooks/base-linux

# cat spec/spec_helper.rb

require 'chefspec'
require 'chefspec/berkshelf'

RSpec.configure do |config|

  # Specify the Chef log_level (default: :warn)
  config.log_level = :debug

end

Resources

#cheffriends

 


Questions


 

Backup Slides

Chef DevOps link

 

Upload/Share CB to supermarket

Chef workflow

"simple" web app

PartA, develpop cookbooks/repo/recipes

https://asciinema.org/a/31362

 

PartB - execute serverspec/inspec audit

https://asciinema.org/a/31360

Debug RSPEC

# cd <repo>/cookbooks/base-linux

# cat spec/spec_helper.rb

require 'chefspec'
require 'chefspec/berkshelf'

RSpec.configure do |config|

  # Specify the Chef log_level (default: :warn)
  config.log_level = :debug

end

Debug RSPEC

# rspec spec |wc -l

348945

Chef DK versions

# chef exec chef -v

Chef Development Kit Version: 0.10.0
chef-client version: 12.6.0
berks version: 4.0.1
kitchen version: 1.5.0

# berks

Resolving cookbook dependencies...
Fetching 'base-linux' from source at .
Using apparmor (2.0.1)
Using apt (3.0.0)
Using base-linux (0.1.0) from source at .
Using chef-sugar (3.3.0)
Using chef_handler (1.3.0)
Using compat_resource (12.7.3)
Using cron (1.7.5)
Using firewall (2.4.0)
Using hostsfile (2.4.5)
Using motd (0.6.3)
Using ntp (1.10.1)
Using ohai (2.1.0)
Using selinux (0.9.0)
Using sudo (2.9.0)
Using sysctl (0.7.0) from https://github.com/svanzoest-cookbooks/sysctl (at master)
Using system (0.10.1)
Using users (2.0.3)
Using windows (1.39.2)
Using yum (3.10.0)
Using yum-epel (0.6.6)

Conf. mgmt. tools

       Orchestration

 

 

 

Infrastructure as code

OS/Application configuration

Angle of view matters

  • Point & shoot or long-term manage?
  • Traditional IT apps & infrastrures vs. cloud?
  • Local or distributed (hybrid) ?
  • Platform specific?
    • PaaS as (Heroku/CF/Bluemix)
    • Containers (Docker, rkt, k8s )
  • Dynamic configuration and service discovery
  • Level of orchestration
  • . . .
  • . . .

Recognized classic tools

  • Chef
  • Puppet
  • Ansible
  • SaltStack

 

w/Agent

Platforms supproted

Language

DSL

Encryption

Config store

ACL

Frequency

Conf. mgmt. tools

Comparison

Chef | Puppet | Ansible | SaltStack | uDeploy| GRTE | CloudFactory | Bob scripts | Alice scripts | ...

 

 To be considered

  • Whether it's agent less  (Ansible) or need an agent at endpoint (Chef, puppet)

  • The level of abstraction Fabric vs. Chef|Salt|Puppet

  • Whether the aim is to install or install & maintain

  • Level of customized / automated configuration you need to apply vs. how advanced and complex are available community resources (cookbooks etc)

  • Integration to in house tools (CI,...)

  • Platforms and environments involved

  • Configure vs. deploy

Backup slides

Platforms

Traditional IT

IaaS - OpenStack

IBM Cloud Orchestrator, RedHat CloudForms, HP Helion, ...

PaaS - Cloud Foundry

Heroku, IBM Bluemix

Patterns

Heat patterns, IBM PureApps

Orchestration

  • containers provide the unit of execution for applications in PaaS
  • Container orchestration (Kubernetes, Swarm) vs. Distributed configuration (etcd, consul, ...)

Backup slide

Containers

  • Focused on infrastructure -OpenStack
  • Focused on application - UrbanCode, Chef Delivery, ...

Deployment tools

.drone.yml

cache:
  mount:
  - .berkshelf/cookbooks

env:
  - CI=drone
  - LABREPO=https://git.xxxx.yy/chef-cookbooks

build:
  image: epcim/drone-chef-ci
  commands:

# RUN TESTS
  # Rspec
  - chef exec rspec spec
  # Kitchen soap
  - KITCHEN_LOCAL_YAML=.kitchen.cloud.yml chef exec kitchen test    || E=$?; test -n $E && /bin/sleep 300  #--concurrency 2
  - KITCHEN_LOCAL_YAML=.kitchen.cloud.yml chef exec kitchen destroy
  # Exit on fail
  - test ${E:-0} -ne 0 && exit $E

# RUN LINT CHECKERS
  - chef exec foodcritic "-f any -f ~FC014 -f ~FC015 -f ~FC024 -f ~FC043" .
  - chef exec rubocop -a || chef exec rubocop . #|| echo "ignoring failed rubocop check"

# TAG SUCCESSFUL VERSION
  - VERSION=`ruby -r 'chef/cookbook/metadata' -e "m=Chef::Cookbook::Metadata.new;m.from_file('metadata.rb');puts m.version"`
  - git config remote.origin.url `git config --get remote.origin.url | sed "s/https:\/\//https:\/\/$GIT_USERNAME:$GIT_PASSWORD@/"`
  - git tag -a v${VERSION} -m 'CI tagged successful build'
  - git push origin v${VERSION} 2> /dev/null || echo $?


notify:
  slack:
    webhook_url: 'https:#hooks.slack.com/services/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f57qO'
    username: 'drone.io'
    channel: '#lab-cookboks'
    on_started: false
    on_success: true
    on_failure: false

Drone.io

Drone.io addons

.drone.yml - full

cache:
  mount:
  - .berkshelf/cookbooks

clone:
  skip_verify: true
  environment:
  - DRONE_GITLAB_SKIP_VERIFY=true
  - GIT_CURL_VERBOSE=1
  - GIT_SSL_NO_VERIFY=1

env:
  - CI=drone
  - LABREPO=https://git.xxxxx.yy/chef-cookbooks

build:
  image: epcim/drone-chef-ci
  privileged: true
  environment:
  - OS_USERNAME=$$OS_USERNAME
  - OS_PASSWORD=$$OS_PASSWORD
  - OS_TENANT_NAME=$$OS_TENANT_NAME
  - OS_AUTH_URL=$$OS_AUTH_URL
  - BOOTSTRAP_SSH_KEY=$$BOOTSTRAP_SSH_KEY
  - BOOTSTRAP_SSH_PUB=$$BOOTSTRAP_SSH_PUB
  - GIT_USERNAME=$$GIT_USERNAME
  - GIT_PASSWORD=$$GIT_PASSWORD
  - GIT_COMMITTER_NAME=$$GIT_COMMITTER_NAME
  - GIT_COMMITTER_EMAIL=$$GIT_COMMITTER_EMAIL
  commands:
### INITIALIZE CI ENV  ###
  - . /root/.bash_profile
  - eval "$(chef shell-init sh)"
  - export LANG="en_US.UTF-8"
  - export SSL_CERT_FILE=/opt/chefdk/embedded/ssl/certs/cacert.pem
  - chef exec berks install

### INITIALIZE REMOTE CI ENV ###
  - BOOTSTRAP_KEYFILE=/root/.ssh/bootstrap_insecure
  - echo "$BOOTSTRAP_SSH_KEY" >  $BOOTSTRAP_KEYFILE; chmod 0600 $BOOTSTRAP_KEYFILE
  - eval `ssh-agent`
  - ssh-add $BOOTSTRAP_KEYFILE
  - ssh-add ./id_rsa_bootstrap_insecure

  #- chef exec bundle install
  - chef gem install kitchen-openstack

# RUN TESTS
  # Rspec
  - chef exec rspec spec
  # Kitchen soap
  - KITCHEN_LOCAL_YAML=.kitchen.cloud.yml chef exec kitchen test    || E=$?; test -n $E && /bin/sleep 300  #--concurrency 2
  - KITCHEN_LOCAL_YAML=.kitchen.cloud.yml chef exec kitchen destroy
  # Exit on fail
  - test ${E:-0} -ne 0 && exit $E

# RUN LINT CHECKERS
  - chef exec foodcritic "-f any -f ~FC014 -f ~FC015 -f ~FC024 -f ~FC043" .
  - chef exec rubocop -a || chef exec rubocop . #|| echo "ignoring failed rubocop check"

# TAG SUCCESSFUL VERSION
  - VERSION=`ruby -r 'chef/cookbook/metadata' -e "m=Chef::Cookbook::Metadata.new;m.from_file('metadata.rb');puts m.version"`
  - git config remote.origin.url `git config --get remote.origin.url | sed "s/https:\/\//https:\/\/$GIT_USERNAME:$GIT_PASSWORD@/"`
  - git tag -a v${VERSION} -m 'CI tagged successful build'
  - git push origin v${VERSION} 2> /dev/null || echo $?


notify:
  slack:
    webhook_url: 'https:#hooks.slack.com/services/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'
    username: 'drone.io'
    channel: '#lab-cookboks'
    on_started: false
    on_success: true
    on_failure: false

chef-repo

  • Monolitic, chef repository with all cookbooks in "cookbooks" directory
  • Berkshelf way, cookbooks has own repos. Berkshelf solve dependency, download cookbooks and manage Berksfile.

 

  • to upload / download artifacts to chef server
  • place from where you manage nodes
  • env+roles OR completely Policy based

chef-repo-#{ENV}

  • Chef cookbook as many others  **
  • Contain definition for roles, environments, data bags
  • May contain chef provisioning recipes
    • for applications, environments, etc ...
  • May contain chef audit recipes (InSpec)
  • Integration tests ServerSpec/InSpec  
  • Rakefile, scripts, core configuration

knife-zero-repo

  • For one-time / simple deployments
  • Runs Chef-Zero in memory
  • Filesystem as persistent storage
  • Commands
    • bootstrap
    • converge
    • chef_client
    • diagnose 

rake -T

rake berks                     # Berks install/update/vendor
rake bootstrap:chefonly        # Boostrap CHEF ONLY with SSH
rake bootstrap:ci              # Boostrap CI against OpenStack env
rake bootstrap:gtshub          # Boostrap gtshub using SSH
rake bootstrap:setup           # Bootstrap local cfg for chef-provisioning setup
rake bootstrap:test            # Boostrap TEST against OpenStack env
rake bootstrap:vagrant         # Bootstrap Vagrant env
rake chef:hosts                # Create a hosts.yml file based on knife information
rake chef:hosts_local          # Create a hosts.yml file based on local ChefZero information
rake chef:nodes_local          # Print ip + fqdn from nodes records
rake convert                   # Convert ruby classes to json
rake convert:env_to_json       # Convert ruby environments from ruby to json, creating/overwriting json files
rake convert:metadata_to_json  # Convert all metadata from ruby to json
rake convert:role_to_json      # Convert ruby roles from ruby to json, creating/overwriting json files
rake destroy:ci                # Destroy CI environment
rake dev:bootstrap             # Run task bootstrap:ci in local DEV environment
rake dev:cls                   # Clean up local chef-zero instances
rake dev:setup                 # Setup local development ENV variables
rake foodcritic                # Foodcritic
rake knife:bootstrap           # Bootstrap new node using knife
rake lint                      # Lint & syntax checks
rake provision:chefonly        # Provision CHEF server with SSH
rake provision:ci              # Provision CI environment against Openstack
rake provision:gtshub          # Provision gtshub environment with SSH
rake provision:test            # Provision TEST environment with Openstack
rake rubocop                   # Run RuboCop style and lint checks
rake rubocop:auto_correct      # Auto-correct RuboCop offenses
rake spec                      # Run serverspec to all hosts
rake vagrant:create            # Create the Vagrant machine
rake vagrant:destroy           # Stop and delete the Vagrant machine
rake vagrant:login             # Log into the Vagrant machine via SSH
rake vagrant:provision         # Provision the Vagrant machine with Chef

knife zero

  • Knife-Zero is not replacement of Knife-Solo. Knife-Zero adds the function which can do a target remotely to the local_mode which is a subset of the Chef-Server/Client environment.
  • https://knife-zero.github.io

 

Run chef-client at remote node with chef-zero(local-mode) via HTTP over SSH port fowarding.

Chef - DevOps Meetup

By Petr Michalec

Chef - DevOps Meetup

How to write community cookbooks, best practice, discussion...

  • 1,635
Loading comments...

More from Petr Michalec