Cookbooks
CHAPTER 4
TOPICS
-
Cookbooks
-
Chef generator
-
recipe naming
-
run list
-
notification and handlers
cookbooks
-
cookbooks are the packages with recipes and supporting files
-
have 1:1 mapping with the applications
-
let you create modular code
anatomy
recipes
files
templates
attributes
spec
test
README
metadata
generator
-
chef is a utility which comes with code generator -
chef generate can let you create cookbooks, templates, attributes etc.
root@int01:/workspace/chap5# chef generate
Usage: chef generate GENERATOR [options]
Available generators:
app Generate an application repo
cookbook Generate a single cookbook
recipe Generate a new recipe
attribute Generate an attributes file
template Generate a file template
file Generate a cookbook file
lwrp Generate a lightweight resource/provider
repo Generate a Chef code repository
policyfile Generate a Policyfile for use with the install/push commands
generator Copy ChefDK's generator cookbook so you can customize it
build-cookbook Generate a build cookbook for use with Delivery
app
cookbook
Group exercise
You have been asked to deploy a java application server with tomcat. You have been tasked to create automation code with chef to set it up.
Problem Statement
APP server
cookbook
Group exercise
- We would create a cookbook to setup tomcat server and apply it to automate the task
- As a prerequisite, we also need to install Java. We will create a cookbook to install openjdk.
Solution:
One repo per App
COOKBOOK
APP
One repo per cookbook
.
`-- sysfoo
|-- README.md
|-- cookbooks
| `-- sysfoo
| |-- Berksfile
| |-- chefignore
| |-- metadata.rb
| |-- recipes
| | `-- default.rb
| `-- spec
| |-- spec_helper.rb
| `-- unit
| `-- recipes
| `-- default_spec.rb
`-- test
`-- recipes
`-- default_test.rb.
`-- mycookbook
|-- Berksfile
|-- README.md
|-- chefignore
|-- metadata.rb
|-- recipes
| `-- default.rb
|-- spec
| |-- spec_helper.rb
| `-- unit
| `-- recipes
| `-- default_spec.rb
`-- test
`-- recipes
`-- default_test.rbgenerating APP
Group exercise
chef generate app sysfoo.
`-- sysfoo
|-- README.md
|-- cookbooks
| `-- sysfoo
| |-- Berksfile
| |-- chefignore
| |-- metadata.rb
| |-- recipes
| | `-- default.rb
| `-- spec
| |-- spec_helper.rb
| `-- unit
| `-- recipes
| `-- default_spec.rb
`-- test
`-- recipes
`-- default_test.rb generating cookbook
Group exercise
cd sysfoo
chef generate cookbook cookbooks/java
chef generate cookbook cookbooks/tomcat
root@ws:/workspace/sysfoo# tree
.
|-- README.md
|-- cookbooks
| |-- java
| | |-- Berksfile
| | |-- README.md
| | |-- chefignore
| | |-- metadata.rb
| | |-- recipes
| | | `-- default.rb
| | |-- spec
| | | |-- spec_helper.rb
| | | `-- unit
| | | `-- recipes
| | | `-- default_spec.rb
| | `-- test
| | `-- recipes
| | `-- default_test.rb
| |-- sysfoo
| | |-- Berksfile
| | |-- chefignore
| | |-- metadata.rb
| | |-- recipes
| | | `-- default.rb
| | `-- spec
| | |-- spec_helper.rb
| | `-- unit
| | `-- recipes
| | `-- default_spec.rb
| `-- tomcat
| |-- Berksfile
| |-- README.md
| |-- chefignore
| |-- metadata.rb
| |-- recipes
| | `-- default.rb
| |-- spec
| | |-- spec_helper.rb
| | `-- unit
| | `-- recipes
| | `-- default_spec.rb
| `-- test
| `-- recipes
| `-- default_test.rb
`-- test
`-- recipes
`-- default_test.rb
22 directories, 24 files 
cookbooks
tomcat
recipes
default.rb
- chef generator creates a directory structure required for cookbooks
- it also generates the default recipe
- additional recipes can be added to cookbooks/xxx/recipes directory
????
????
writing recipes
cookbooks
tomcat
recipes
default.rb
- every feature gets its own recipe
- create a separate recipe for each phase of application lifecycle. This gives granular control
install.rb
config.rb
strategies
service.rb
deploy.rb
ssl.rb
java recipe
Group exercise
package 'java-1.7.0-openjdk' do
action :install
endfile: cookbooks/java/recipes/install.rb
now that we have written the recipe, we need a local environment to test this
kitchen.ci
Test Kitchen
.kitchen.yml
Group exercise
---
driver:
name: docker
provisioner:
name: chef_zero
# You may wish to disable always updating cookbooks in CI or other testing environments.
# For example:
# always_update_cookbooks: <%= !ENV['CI'] %>
always_update_cookbooks: true
verifier:
name: inspec
platforms:
- name: centos-6.8
driver_config:
image: codespaces/chef-node-centos-6
forward:
- 8080:8080
suites:
- name: default
run_list:
- recipe[sysfoo::default]
verifier:
inspec_tests:
- test/recipes
attributes:
Edit the ".kitchen.yml" file in "/sysfoo/kitchen.yml"
kitchen
Group exercise
root@ws:/workspace/chapter4/sysfoo# kitchen
Commands:
kitchen console # Kitchen Console!
kitchen converge [INSTANCE|REGEXP|all] # Change instance state to converge. Use a provisioner to configure one or more instances
kitchen create [INSTANCE|REGEXP|all] # Change instance state to create. Start one or more instances
kitchen destroy [INSTANCE|REGEXP|all] # Change instance state to destroy. Delete all information for one or more instances
kitchen diagnose [INSTANCE|REGEXP|all] # Show computed diagnostic configuration
kitchen driver # Driver subcommands
kitchen driver create [NAME] # Create a new Kitchen Driver gem project
kitchen driver discover # Discover Test Kitchen drivers published on RubyGems
kitchen driver help [COMMAND] # Describe subcommands or one specific subcommand
kitchen exec INSTANCE|REGEXP -c REMOTE_COMMAND # Execute command on one or more instance
kitchen help [COMMAND] # Describe available commands or one specific command
kitchen init # Adds some configuration to your cookbook so Kitchen can rock
kitchen list [INSTANCE|REGEXP|all] # Lists one or more instances
kitchen login INSTANCE|REGEXP # Log in to one instance
kitchen package INSTANCE|REGEXP # package an instance
kitchen setup [INSTANCE|REGEXP|all] # Change instance state to setup. Prepare to run automated tests. Install busser and related gems on o...
kitchen test [INSTANCE|REGEXP|all] # Test (destroy, create, converge, setup, verify and destroy) one or more instances
kitchen verify [INSTANCE|REGEXP|all] # Change instance state to verify. Run automated tests on one or more instances
kitchen version # Print Kitchen's version information
create
converge
verify
destroy
test
Kitchen create
Group exercise
cd /workspace/sysfoo
kitchen list
kitchen createThis will create a kitchen environment, basically an instance of centos-6.8
Verify using kitchen list command
kitchen list
Instance Driver Provisioner Verifier Transport Last Action
default-centos-68 Docker ChefZero Inspec Ssh CreatedGroup exercise
Kitchen converge
kitchen converge
kitchen listThis will change the instance state to converge and install chef and apply recipe defined in ".kitchen.yml" run_list
kitchen login
[kitchen@1ab95d811596 ~]$
[kitchen@1ab95d811596 ~]$ which java
/usr/bin/java
[kitchen@1ab95d811596 ~]$ java -version
java version "1.7.0_131"
OpenJDK Runtime Environment (rhel-2.6.9.0.el6_8-x86_64 u131-b00)
OpenJDK 64-Bit Server VM (build 24.131-b00, mixed mode)
[kitchen@1ab95d811596 ~]$ logout
Connection to localhost closed.This will change the instance state to converge and install chef client and apply recipe defined in ".kitchen.yml" run_list
To verify, login and check the installation of java in an instance
Upload COokbook
Group exercise
knife cookbook upload openjdk
Uploading openjdk [0.1.0]
Uploaded 1 cookbook. delete this
applying recipes
runlist
node
list of recipes
run lists
-
defines list of items to apply
-
items can be either recipes or roles
-
for now, we are going to stick to recipes
cookbooks
tomcat
recipes
default.rb
- recipes are referred to with cookbook::recipe convention
- "::" is added as namespace separators
- namespaces must map to the directory structure
abc.rb
xyz.rb
recipe identifiers
tomcat::abc
tomcat::xyz
subdir
pqr.rb
tomcat::subdir::pqr
add java install recipe to kitchen.yml
suites:
- name: default
run_list:
- recipe[java::install]
verifier:
inspec_tests:
- test/recipes
attributes:
kitchen converge
-----> Starting Kitchen (v1.13.2)
-----> Destroying <default-centos-68>...
sudo: unable to resolve host ws.codespaces.io
UID PID PPID C STIME TTY TIME CMD
root 6267 6248 0 05:25 ? 00:00:00 /usr/sbi
n/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid
91 8185 6267 2 05:32 ? 00:00:08 /usr/lib
/jvm/jre/bin/java -classpath :/usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar -Dcatalina.base=
/usr/share/tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/usr/shar
e/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager org.apache.catalina.startup.Bootstrap start
sudo: unable to resolve host ws.codespaces.io
c94a66ffa1fe0149555f2db01c821d42a221fbe6e0cf412c023a91ec782fef76
sudo: unable to resolve host ws.codespaces.io
c94a66ffa1fe0149555f2db01c821d42a221fbe6e0cf412c023a91ec782fef76
Finished destroying <default-centos-68> (0m1.80s).
-----> Kitchen is finished. (0m3.05s)
root@ws:/workspace/myapp# kitchen converge
-----> Starting Kitchen (v1.13.2)
-----> Creating <default-centos-68>...
sudo: unable to resolve host ws.codespaces.io
Sending build context to Docker daemon 163.3 kB
Step 1/16 : FROM centos:centos6
---> 30365b2e827c
Step 2/16 : ENV container docker
---> Using cache
---> 567877f93777
Step 3/16 : RUN yum clean all
---> Using cache
---> 26c27f345c79
Step 4/16 : RUN yum install -y sudo openssh-server openssh-clients which curl
---> Using cache
---> b5b2a5a704d1
Step 5/16 : RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -N ''
---> Using cache
---> 7310df2e1294
Step 6/16 : RUN ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key -N ''
---> Using cache tomcat
recipes
Group exercise
- install packages to setup web server application and pre requisites
- manage configurations for web servers
- start/stop service
phases
tomcat::service
tomcat::config
tomcat::install
lab
LAB exercise
create a tomcat::service recipe to start and enable tomcat service
create a tomcat::install recipe to install tomcat along with example apps. Packages to install are
- epel-release
- tomcat
- tomcat-webapps
tomcat::install
tomcat::service
add to run list in kitchen.yml
Group exercise
suites:
- name: default
run_list:
- recipe[java::install]
- recipe[tomcat::install]
- recipe[tomcat::service]
verifier:
inspec_tests:
- test/recipes
attributes:converge again
root@ws:/workspace/myapp# kitchen converge
-----> Starting Kitchen (v1.13.2)
-----> Converging <default-centos-68>...
Preparing files for transfer
Preparing dna.json
Preparing cookbooks from project directory
Removing non-cookbook files before transfer
Preparing validation.pem
Preparing client.rb
-----> Chef Omnibus installation detected (install only if missing)
Transferring files to <default-centos-68>
Starting Chef Client, version 13.0.118
resolving cookbooks for run list: ["java::default", "tomcat::install", "tomcat::service"]
Synchronizing Cookbooks:
- tomcat (0.1.0)
- java (0.1.0)
Installing Cookbook Gems:
Compiling Cookbooks...
Converging 4 resources
Recipe: java::default
* yum_package[epel-release] action install
- install version 6-8 of package epel-release
* yum_package[java-1.7.0-openjdk] action install
- install version 1.7.0.131-2.6.9.0.el6_8 of package java-1.7.0-openjdk
Recipe: tomcat::install
* yum_package[tomcat, tomcat-webapps] action install
- install version 7.0.75-1.el6 of package tomcat-webapps
- install version 7.0.75-1.el6 of package tomcat
Recipe: tomcat::service
* service[tomcat] action start
- start service service[tomcat]
* service[tomcat] action enable
- enable service service[tomcat]
Running handlers:
Running handlers complete
Chef Client finished, 5/5 resources updated in 01 minutes 41 seconds
Finished converging <default-centos-68> (1m44.02s).
-----> Kitchen is finished. (1m45.26s)Group exercise
validate
kitchen login
service tomcat status
exitGroup exercise
lab
Expected Output
- You should be able to browse to http://ipaddress:8080 port in the browser
- Tomcat home page is the expected output
http://IPADDR:8080
LAB exercise
simplify runlist
Group exercise
#
# Cookbook Name:: tomcat
# Recipe:: default
#
# Copyright (c) 2017 The Authors, All Rights Reserved.
include_recipe 'java::install'
include_recipe 'tomcat::install'
include_recipe 'tomcat::service'
tomcat::default
suites:
- name: default
run_list:
- recipe[tomcat]Add Entry in ./sysfoo/.kitchen.yml file
Group exercise
kitchen converge
......
.......
Chef::Exceptions::CookbookNotFound
----------------------------------
Cookbook java not found. If you're loading java from another cookbook, make sure you configure the dependency in your metadata
Cookbook Trace:
---------------
/tmp/kitchen/cache/cookbooks/tomcat/recipes/default.rb:7:in `from_file'
Relevant File Content:
----------------------
/tmp/kitchen/cache/cookbooks/tomcat/recipes/default.rb:
1: #
2: # Cookbook Name:: tomcat
3: # Recipe:: default
4: #
5: # Copyright (c) 2017 The Authors, All Rights Reserved.
6:
7>> include_recipe 'java'
8: # we can also call the above recipe as
9: # include_recipe 'java::default'
10:
11: include_recipe 'tomcat::install'
12: include_recipe 'tomcat::service'
13:
System Info:
------------
chef_version=13.0.118
platform=centos
platform_version=6.9
ruby=ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]
program_name=chef-client worker: ppid=1020;start=12:14:19;
executable=/opt/chef/bin/chef-client
Running handlers:
[2017-04-25T12:14:21+00:00] ERROR: Running exception handlers
[2017-04-25T12:14:21+00:00] ERROR: Running exception handlers
Running handlers complete
[2017-04-25T12:14:21+00:00] ERROR: Exception handlers complete
[2017-04-25T12:14:21+00:00] ERROR: Exception handlers complete
Chef Client failed. 0 resources updated in 01 seconds
[2017-04-25T12:14:21+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
[2017-04-25T12:14:21+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
[2017-04-25T12:14:21+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2017-04-25T12:14:21+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2017-04-25T12:14:21+00:00] ERROR: Cookbook java not found. If you're loading java from another cookbook, make sure you configure the dependenc
y in your metadata
[2017-04-25T12:14:21+00:00] ERROR: Cookbook java not found. If you're loading java from another cookbook, make sure you configure the dependenc
y in your metadata
[2017-04-25T12:14:21+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
[2017-04-25T12:14:21+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>> Converge failed on instance <default-centos-68>. Please see .kitchen/logs/default-centos-68.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration
- we will get "Converge failed on instance" with error
- Need to check with kitchen logs for error
- As you can see it was dependency error
CONVERGE AGAIN
Group exercise
depends
- we included java cookbook in tomcat default.rb
- We also need to add dependency in metadata.rb of tomcat cookbook that depends on java
name 'tomcat'
maintainer 'The Authors'
maintainer_email 'you@example.com'
license 'all_rights'
description 'Installs/Configures tomcat'
long_description 'Installs/Configures tomcat'
version '0.1.0'
depends 'java'DEPENDENCIES
manging files
-
We will need to manage configurations eg. tomcat.conf -
since chef is a centralized configuration management system, we will keep the files centrally in cookbooks, which will then be copied to all managed nodes
manging files
-
Generate tomcat.conf using chef generate file in tomcat cookbook directory. -
add tomcat::config recipe to copy these files to the relevant locations on destination hosts
cookbooks
tomcat
recipes
default.rb
install.rb
service.rb
config.rb
files
tomcat.conf
LAB exercise
destination path in nodes: /etc/tomcat/tomcat.conf
tomcat.conf
file: cookbooks/tomcat/files/default/tomcat.conf
Use chef generate file to create tomcat.conf file
chef generate file cookbooks/tomcat tomcat.confTOMCAT_CFG_LOADED="1"
JAVA_HOME="/usr/lib/jvm/jre"
JAVA_OPTS="-Xms64m -Xmx128m -XX:MaxPermSize=128M -Djava.security.egd=file:/dev/./urandom"
CATALINA_BASE="/usr/share/tomcat"
CATALINA_HOME="/usr/share/tomcat"
JASPER_HOME="/usr/share/tomcat"
CATALINA_TMPDIR="/var/cache/tomcat/temp"
TOMCAT_USER="tomcat"
SECURITY_MANAGER="false"
SHUTDOWN_WAIT="30"
SHUTDOWN_VERBOSE="false"
CATALINA_PID="/var/run/tomcat.pid"Group exercise
tomcat::config
cookbook_file '/etc/tomcat/tomcat.conf' do
source 'tomcat.conf'
owner 'tomcat'
group 'tomcat'
mode 0644
action :create
endfile: cookbooks/tomcat/recipes/config.rb
Use chef generate recipe to create config.rb
chef generate recipe cookbooks/tomcat configGroup exercise
tomcat::config
cookbook_file '/etc/tomcat/tomcat.conf' do
source 'tomcat.conf'
owner 'tomcat'
group 'tomcat'
mode 0644
action :create
end
cookbook_file '/etc/tomcat/tomcat-users.xml' do
source 'tomcat-users.xml'
owner 'tomcat'
group 'tomcat'
mode 0644
action :create
end
file: cookbooks/tomcat/recipes/config.rb
refreshing service
conf
service
refresh
Notifications
notifies :action, 'resource[name]', :timer
notifies :restart, 'service[tomcat]', :delayed
tomcat::config
cookbook_file '/etc/tomcat/tomcat.conf' do
source 'tomcat.conf'
owner 'tomcat'
group 'tomcat'
mode 0644
action :create
notifies :restart, 'service[tomcat]', :delayed
endUpdate file: cookbooks/tomcat/recipes/config.rb
Note: Add config.rb recipe to default.rb
tomcat::config
cookbook_file '/etc/tomcat/tomcat.conf' do
source 'tomcat.conf'
owner 'tomcat'
group 'tomcat'
mode 0644
action :create
notifies :restart, 'service[tomcat]', :delayed
end
cookbook_file '/etc/tomcat/tomcat-users.xml' do
source 'tomcat-users.xml'
owner 'tomcat'
group 'tomcat'
mode 0644
action :create
notifies :restart, 'service[tomcat]', :delayed
end
file: cookbooks/tomcat/recipes/config.rb
Group exercise
#
# Cookbook Name:: tomcat
# Recipe:: default
#
# Copyright (c) 2017 The Authors, All Rights Reserved.
include_recipe 'java::install'
include_recipe 'tomcat::install'
include_recipe 'tomcat::config'
include_recipe 'tomcat::service'
tomcat::default
Update config and Converge again
- It converges again and in logs you can see the tomcat service is restarted
verify notification service
kitchen convergekitchen login- check for tomcat status
- Verify the updated java memory
LAB exercise
summary