PkgMgmt & CfgMgmt have failed
Long live Docker?
Franz Pletz // @fpletz
The speaker's bias
- Free Software fundamentalist
- Functional Programming aficionado
- Unix user for ~20 years
- "Full Stack Engineer"
- Webdev for 4 years
- Sysadmin for 10 years
- Puppet for 6 years
- Debian Maintainer
- NixOS Security Team
The sorry state of package management
Features of a Pkg Manager
- codifies software build process
- creates, distributes, fetches packages
- ensures dependencies are met
- manages package lifecycle & versioning
- ensure package integrity & authenticity
- helps users install software
- helps distributions manage metadata
OS-level pkg managers
- BSDs: pkg / ports / pkgsrc
- Debian: dpkg / apt
- Red Hat: rpm / yum / DNF
- Arch: pacman
- Gentoo: emerge / portage
- different, difficult automations
- lots of boilerplate code
- managing state is hard
- rollbacks are hard
- dependency hells
- reproducible builds
- setting up repos is painful
- distribution packaging policies
- easy package creation
- many package managers
- omnibus packages
- with all dependencies included
- third-party package managers
PkgMgmt all the way down
- programming language have lots of modules
- most are not packaged by distributions
- so languages started rolling their own…
- Ruby: gem, bundler
- Python: pip, wheel, virtualenv
- Haskell: cabal
- PHP: pear, pecl, composer
- Node: npm, yarn
- C/C++: conan
- … and made it even worse!
Down the rabbit hole
- language pkg managers generally not very mature
- most don't even implement full SAT solvers :(
- missing integrations in other package managers
- lots of developer-provided packages
- broken dependencies
- bad versioning
- no CI, no automatic testing
- external library deps are not manageable
Say you have a really cool $hiplang app that uses an awesome web framework.
It depends on hundreds of $hiplang packages and some of those on C libraries.
How would you package that for your favorite OS to deploy it on your machines?
Nope, not yet! ;)
Config Management - aka "we'll get there eventually"
Once upon a time
- machines were physical
- multiple unrelated services per machine
- sysadmins edited config files by hand
- machines were caressed like babies
- hostnames were nicknames
- upgrades were done manually
- sysadmins had feelings for their systems
Along comes the "Cloud"
- lots of machines per physical system
- machines easy to create & tear down
- one logical service per machine
- distribution of base or golden images
- automated provisioning & configuration
- thousands of machines
Increased operational workload meant new tools for configuration management and orchestration were needed!
Benefits of CfgMgmt
- infrastructure as code
- abstractions (roles, profiles)
- testable & reviewable infrastructure
- service dependencies
Traditional CfgMgmt Tools
The issue with CfgMgmt
Traditional CfgMgmt tools mutate OS state.
Undeclared aspects of the target systems are undefined!
- convergence to target state
- eventual consistency
What would we need?
- declarative configuration
- OS-independent configuration
- fully reproducible
- atomic upgrades
"You should use Docker!"
- image concept provides necessary features
- simple: one service per container
- configuration in Dockerfiles not reproducible
- needs external container scheduler
- like kubernetes, docker swarm, etc.
- can help a lot, but no silver bullet
What we might use a few years from now
- upstreams will provide container images
- format needs standardization first
More Tooling around Docker
- standards for linking containers
- "plug and play containers"
- centralized deployments becomes distributed
- basic idea: use raft/gossip from etcd & consul
- infrasture configures itself by what's available
- declarative Linux Distribution
- uses purely functional package manger Nix
- premise: pkgmgmt == cfgmgmt
- consistency through immutability
- fully reproducible, pinned to one git commit
- atomic upgrades & rollbacks
- testing framework builtin
- cloud deployments with nixops
Slides available at:
Package & Configuration Managed has failed