Chapter 1

The sorry state of package management

Features of a Pkg Manager

• codifies software build process
• creates, distributes, fetches packages
• ensures dependencies are met
• manages package lifecycle & versioning 
• ensure package integrity & authenticity
• helps users install software
• helps distributions manage metadata

OS-level pkg managers

• BSDs: pkg / ports / pkgsrc
• Debian: dpkg / apt
• Red Hat: rpm / yum / DNF
• Arch: pacman
• Gentoo: emerge / portage

Common Challenges

• different, difficult automations
• lots of boilerplate code
• managing state is hard
• rollbacks are hard
• dependency hells
• reproducible builds
• setting up repos is painful
• distribution packaging policies

Solution Attempts

• jordansissel/fpm
  • easy package creation
  • many package managers
• omnibus packages
  • with all dependencies included
• third-party package managers
  • checkinstall

PkgMgmt all the way down

• programming language have lots of modules
• most are not packaged by distributions
• so languages started rolling their own…
  • Ruby: gem, bundler
  • Python: pip, wheel, virtualenv
  • Haskell: cabal
  • PHP: pear, pecl, composer
  • Node: npm, yarn
  • C/C++: conan
• … and made it even worse!

Down the rabbit hole

• language pkg managers generally not very mature
  • most don't even implement full SAT solvers :(
  • ​missing integrations in other package managers
• lots of developer-provided packages
  • broken dependencies
  • bad versioning
  • no CI, no automatic testing
• external library deps are not manageable

Trick Question

Say you have a really cool $hiplang app that uses an awesome web framework.

It depends on hundreds of $hiplang packages and some of those on C libraries.

How would you package that for your favorite OS to deploy it on your machines?

You
don't!
Docker plz!

Nope, not yet! ;)

Chapter 2

Config Management - aka "we'll get there eventually"

Once upon a time

• machines were physical
• multiple unrelated services per machine
• sysadmins edited config files by hand
• machines were caressed like babies
• hostnames were nicknames
• upgrades were done manually
• sysadmins had feelings for their systems

Along comes the "Cloud"

• lots of machines per physical system
• machines easy to create & tear down
• one logical service per machine
• distribution of base or golden images
• automated provisioning & configuration
• thousands of machines

Increased operational workload meant new tools for configuration management and orchestration were needed!

Benefits of CfgMgmt

• infrastructure as code
• abstractions (roles, profiles)
• testable & reviewable infrastructure
• consistency
• reproducibility
• service dependencies

Traditional CfgMgmt Tools

• CFEngine
• Puppet
• Chef
• Ansible
• Salt
• Rudder
• …

The issue with CfgMgmt

Traditional CfgMgmt tools mutate OS state.

Undeclared aspects of the target systems are undefined!

Implications

• convergence to target state
• eventual consistency

What would we need?

• declarative configuration
• OS-independent configuration
• fully reproducible
• idempotency
• atomic upgrades

"You should use Docker!"

• image concept provides necessary features
• simple: one service per container
• configuration in Dockerfiles not reproducible
• needs external container scheduler
  • like kubernetes, docker swarm, etc.
• can help a lot, but no silver bullet

Chapter 3

What we might use a few years from now

Application Containers

• upstreams will provide container images
• format needs standardization first
• Pioneers:
  • appimage
  • flatpak
  • snap

More Tooling around Docker

• standards for linking containers
• "plug and play containers"
• Pioneers:
  • habitat

s/orchestration/choreography/g

• centralized deployments becomes distributed
• basic idea: use raft/gossip from etcd & consul
• infrasture configures itself by what's available
• Pioneers:
  • kubernetes
  • mgmt

NixOS

• declarative Linux Distribution
• uses purely functional package manger Nix
• premise: pkgmgmt == cfgmgmt
• consistency through immutability
• fully reproducible, pinned to one git commit
• atomic upgrades & rollbacks
• testing framework builtin
• cloud deployments with nixops