Puppet
FAST-TRACK
Workshop
Simon Josi / SWISS TXT
About me
(in respect to configuration management)
clusterssh user since 2005 ;)
cfengine user since 2006 (until 2007)
puppet user since 2007
never used chef, bcfg2, cfengine3 or your mom's favourite cfg mgmt tool... (I've heard and read about them though...)
clusterssh
i.e. execute the same command on multiple hosts and hope that all host behave the same way
easy to begin with
very manual
scalability issues ;)
MY CLUSTERSSH EXPERIENCE
managed up to ~10 at one time (still manually) @ Puzzle ITC
CFENGINE (2.x)
i.e. lots of boilerplate manifest code of which knowbody knows what it does => "WTF is THIS for?!?"
idempotent
declarative ?!?
lots non-declarative functions
e.g. AppendIfNoSuchLine, really?
no progress in development
12 years old in 2007
code looks bulky and old
MY CFENGINE EXPIERENCE
managed ~60 hosts @ Puzzle ITC
puppet
i.e. Model Driven Datacenter
declarative
abstract
idempotent
a model of "resources" and "relationships"
compiled
MY PUPPET EXPERIENCE
managed ~120 nodes with ~170 Modules @ Puzzle ITC
bootstrapped puppet setup @ Die Mobiliar
15 (!) environments, now ~3000 nodes
15 (!) environments, now ~3000 nodes
managed webstack & deployment @ Atizo
~10 nodes, ~110 Modules
PUPPET @ SWISSTXT
platform-services for private cloud offering
~144 nodes, ~50 modules
currently boostrapping puppet environment @ SWISS TXT
currently ~20 nodes, ~30 modules
the big picture
Puppet COMPONENTS
internal (puppet executable)
Language parser and compiler (for internal DSL)
Resource Abstraction Layer (RAL, Types and Providers)
Agent/Master (Client/Server)
PKI (PuppetCA)
external (other executables)
PuppetDB (formerly storedconfigs)
Facter for normalized node information
Hiera for external parameterization
Marionette Collective for centralized orchestration
OVERVIEW
Compiler output is a catalog
catalog is a DAG
types and providers
TYPES AND PROVIDERS
TYPES AND PROVIDERS
MASTER/AGENT
Master is a RESTful webservice
Agents consume this webservice
Secured with SSL client authentication
PuppetCA
Every host got a SSL certificate
Certificates are signed with the PuppetCA (Puppet's own Certificate Authority)
FACTER
or how do your nodes look like?
[root@ellen ~]# facter ... architecture => x86_64 domain => yokto.net fqdn => ellen.yokto.net hardwaremodel => x86_64 hostname => ellen interfaces => eth0,lo ipaddress => 195.141.111.126 ipaddress_eth0 => 195.141.111.126 ipaddress_lo => 127.0.0.1 is_virtual => true kernel => Linux kernelmajversion => 2.6 kernelversion => 2.6.32 lsbdistid => CentOS lsbdistrelease => 6.4 lsbmajdistrelease => 6 ...
available as variables anywhere in your puppet code
puppetdb
used for exported resources
declare a resource on one host, apply on another
central inventory of all facts from all nodes
Hiera
hierarchical parameter storage
MCollective
framework to build server orchestration or parallel job execution systems
architecture
anatomy of a puppet run
detailed agent/master communication
PUPPET'S DSL
restricted by design
specific to the domain
managing resources, declaring relationships
declarative (not imperative/no iterators)
uses functional programming paradigms to work with collections of items (e.g. map)
Give me code please!
class openssh::server {
package{'openssh-server':
ensure => present,
}
file{'/etc/ssh/sshd_config':
ensure => present,
source => 'puppet:///modules/openssh/sshd_config',
group => 'root',
mode => '0444',
require => Package['openssh-server'],
notify => Service['sshd'],
}
service{'sshd':
ensure => running,
enable => true,
require => [
Package['openssh-server'],
File ['/etc/ssh/sshd_config'],
],
}
}
The same without redundanT RELATIONSSHIPS OR DEFAULT VALUES
and the use of relationship operators
class openssh::server {
package{'openssh-server':
ensure => present,
} ->
file{'/etc/ssh/sshd_config':
source => 'puppet:///modules/openssh/sshd_config',
} ~>
service{'sshd':
ensure => running,
enable => true,
}
}language constructs
MANIFESTS
IMPORT
NODES
MODULES
CLASSES
NAMESPACING AND AUTOLOADING
DEFINED TYPES
NATIVE TYPES
FUNCTIONS
manifests
every file ending in .pp and containing puppet DSL code is called a manifests
IMPORT
included a file in place and evaluate it
just like php's include
never use it it in modules
only use it to bootstrap code
nodes
defined in site.pp (common practice)
puppet's entry point
only include/declare classes (best practice)
modules
a convention to structure/group
manifests
FILES
TEMPLATES
FUNCTIONS
FACTS
TYPES and providers
classes
singletons
parameterizable
parameterizable
can be declared multiple times (without parameters)
but will get applied only once
can de declared only once (with parameters)
namespacing and autoloading
modulepaths
Default: $confdir/modules:/usr/share/puppet/modules
MPC: $confdir/$environment/forge $confdir/$environment/site $confdir/$environment/role $confdir/$environment/platform-services
defined types
not singletons ;)
parameterizable
parameterizable
can de declared multiple times (with different identifiers)
look and behave like native resources
native types
file package service
user group host
mount exec augeas
ssh_authorized_key yumrepo
functions
are executed on the server during catalog compilation
are written in ruby
e.g. template compilation
NAMESPACING AND AUTOLOADING
$modulepath/modulename/init.pp =>
class modulename {
$modulepath/modulename/server.pp =>
class modulename::server {
$modulepath/modulename/server/debian.pp =>
class modulename::server::debian {
SCOPES
more LANGUAGE CONSTRUCTS
INHERITANCE
RESOURCE PARAMETERS AND METAPARAMETERS
VARIABLES AND SCOPES
RESOURCE ORDERING
NOTIFY/SUBSCRIBE
CONTROL FLOW STRUCTURES
EXPORTED RESOURCES
RESOURCE COLLECTORS
INHERITANCE
DO NOT USE IT
only in edge cases really useful or the way to go
parameters and metaparameters
each resource has its own set of supported parameters
metaparameters are the ones valid for all resources
variables and scopes
resource ordering
before/after
relationship/CHaining operators
notify/subscribe
control flow structures
exported resources
resource collectors
definitions and declarations
define vs declare
DEFINTIONS
the origin of things
class modulename::classname {
package{'package_name':} ->
file{'/etc/filename':} ~>
service{'service_name':}
} define modulename::definename(
$parameter_one = 'default_value',
) {
file{"/etc/$parameter_one"}
}
declarations
the use of things
include modulename::classname class{'modulename::classname':} class{'modulename::classname':
param_one => 'value',
}
modulename::definename{'instance_one':
param_one => 'value',
}
modulename::definename{'instance_two':
param_one => 'value',
}
TYPES OF PUPPET MODULES
PUBLIC/FORGE
SITE
ROLE
...
many more depending of how you want to model your infrastructure
can be seperated and integrated via multiple puppet modulepaths and the use of git submodules
COMPOSITION
PUBLIC/FORGE MODULES
found in the intertubes
site modules
site-specific use of a public module
reusable site-wide defaults/helpers for a public module
ROLE MODULES
composition of multiple forge and site modules into a "role"
COMPOSITION ORDER OF PUPPET MODULE TYPES
FORGE -> SITE -> ROLE -> NODE
sometimes you don't need an intermediate module, e.g.
FORGE -> NODE
ROLE -> NODE
FORGE -> ROLE -> NODE
you get the idea...
REAL WORLD EXAMPLE
manage openstack with puppet
NODE CLASSIFICATION PATTERNs
or wtf is an ENC
node defintions
node 'www1.example.com' {
include common
include apache
include squid
}
node /w+.example.com/ {
include common
include apache
include squid
}
externaL node classifier
[master] node_terminus = execexternal_nodes = /usr/local/bin/puppet_node_classifier
ENCs must return either a YAML hash or nothing. This hash may contain classes, parameters, and environment keys, and must contain at least either classes or parameters.
nodeless configuration
fact driven
node default {
include $role
}event propagation
class sshd:service {
service{'sshd':}
}
File['somefile']{
notify => Class['sshd::service'],
}
EVENT PROPAGATION
file{'/somefile':
source => 'puppet:///modules/module/file',
} ~>
mount{'/target':
device => '/dev/sdx1',
} ~>
service{'someservice':
}PuppetFAST-TRACKWorkshop
By Simon Josi
