Introduction to Nix (and NixOS)
Suvash Thapaliya ~ @suvash
25th February 2015
Disclaimer
I'm pretty sure I don't know a lot things.
But, please feel free to ask questions !
The Problem
You UPGRADE ONE PACKAGE & find OTHER PACKAGES on your system NO LONGER WORKS. Turns out some shared dependencies got upgraded as well, and they’re not compatible.
You upgraded _______ and discover it no longer works.
There is NO easy way to UNDO the upgrade.
Your server dies. But because of MANUAL CHANGES over time you CANNOT exactly REPRODUCE what state of CONFIGURATION the server was in. (Hopefully not).
Your team develops an application that has many DEPENDENCIES. Your developers waste a lot of time setting up their build environments. Every time a dependency CHANGES, EVERYBODY on the team needs to UPGRADE their environments MANUALLY.
You've installed a bunch of things(and their dependencies). During UNINSTALL you forget/don't want to bother CLEANING UP all the non used DEPENDENCIES that were pulled in.
Some solutions....?
Servers
Chef
Puppet
Ansible....
Containers
Docker
LXC
Rocket ...
Applications
Bundler + rvm/rbenv
Pip + virtualenv
Cabal + sandboxes
....
Configuration
Management
Declarative
Purely Functional
Nix(OS) Family
Nix expressions
(language)
Nix package manager
Nixpkgs
Hydra
Nix Build System
NixOS
Linux + Nixpkgs
NixOps
NixOS + Operations tooling
Things we won't quite cover...
Nix expressions
(language)
Hydra
Nix Build System
NixOps
NixOS + Operations tooling
Why Nix ?
No update/grade-in-place (or immutable updates)
(as compared to apt/brew/yum/you-name-it....)
Atomic upgrades (or upgrades are transactional)
(if system crashes half-way through an upgrade it will still work....)
Cheap and instantaneous rollbacks
(guess why....)
Reproducibility
(with the exact same cryptographic hash....)
Origin
"The Purely Functional Software Deployment Model"
- PhD Thesis, Eelco Dolstra, 2006
"NixOS: A Purely Functional Linux Distribution"
- Eelco Dolstra and Andres Löh, 2008
and more .....
http://nixos.org/docs/papers.html
Philosophy
"The basic philosophy behind Nix - to solve deployment problems by treating them analogously to memory management issues in programming languages"
- The Purely Functional Software Deployment Model
- Nix is a system for software deployment
- Components(a set of files/tree in a filesystem) are the basic units of deployment
- Nix stores components in a component store, called the Nix store (usually /nix/store)
- The names of the components contain a cryptographic hash of all the inputs involved in building the component. (eg. /nix/store/b6q64986n0smwwfwrhg655lcg3pxhk56-git-2.1.4 )
- The collision resistant name is what provides Nix the ability to prevent undeclared dependencies and a component interface
Components & Closures
Hash is computed over all inputs
- Source of components
- the build script
- arguments or env. variables passed to build script
- all build time dependencies (compiler, linker, libraries, shell, etc.)
$ gcc -o main.c -lssl
# ssl lib already installed
# would not work in nix
# unless explicitly provided
# as an input to the build
Demo
# nix-env manages user-environments
# which are 'views' into the available components
$ nix-env -i cowsay
$ which cowsay
$ ls -lah $(which cowsay)
$ realpath $(which cowsay)
$ readlink -f $(which cowsay)
$ nix-env --list-generation
$ nix-env --rollback
$ which cowsay
$ nix-env --switch-generation X
$ which cowsay
$ nix-env --uninstall cowsay
$ nix-env --delete-generations 10d
$ nix-collect-garbage
More Demo
# pick the symlink to ~/.nix-profile
$ ls -l ~/
$ ls -l ~/.nix-profile
# up the profile directory
$ ls -l <symlink-to-nix-profile-without-the-profile-directory-at-the-end>
# where is it located actually (can be seen above too) ?
$ realpath ~/.nix-profile
# Custom profile
$ which cowsay
$ cd <some-dir>
$ nix-env -i cowsay -p custom-profile
$ ls -l
$ ./custom-profile/bin/cowsay -f dragon Nix is awesome
$ realpath ./custom-profile
$ which sl
$ nix-env -i sl -p custom-profile
$ ./custom-profile/bin/sl
$ nix-env --list-generation -p custom-profile
# compare the hash of custom profile and 'normal' profile
$ nix-env -i sl
# uninstall sl, remove custom profile, garbage collect and path no more
Purely functional : What does it mean ?
(in this context)
The Nix expression language is lazy and functional.
Package builds are intended to be pure i.e. only depend on the declared inputs
Nix 'tries hard to enforce this : no network downloads, no using other programs(/usr/..), clear the environment/PATH during build, optionally build in a chroot jail, etc.
*but it cannot guarantee purity in the absolute sense
(Nix cannot help if the build script/os depends on the phase of moon.)
The results of building a package - the paths to the Nix store - can be seen as a purely functional persistent data structure.
immutable objects that refer to each other, garbage-collected when unreachable.
Persistent Data Structures
Example Package
with import <nixpkgs> { };
stdenv.mkDerivation rec {
pname = "mosquitto";
version = "1.4";
name = "${pname}-${version}";
src = fetchurl {
url = http://mosquitto.org/files/source/mosquitto-1.4.tar.gz;
sha256 = "1imw5ps0cqda41b574k8hgz9gdr8yy58f76fg8gw14pdnvf3l7sr";
};
buildInputs = [ openssl libuuid ];
buildFlags = "mosquitto";
installPhase = ''
mkdir -p $out/bin
cp src/mosquitto $out/bin/
'';
meta = {
homepage = http://mosquitto.org/;
description = "An open source MQTT v3.1/3.1.1 broker";
platforms = stdenv.lib.platforms.linux;
license = stdenv.lib.licenses.epl10;
};
}
Custom Package Demo ?
# nix-build
$ nix-build ./mosquitto.nix
$ ls -l
$ ./result/bin/mosquitto
# nix-shell
$ nix-shell ./mosquitto.nix
nix-shell $ echo $src
nix-shell $ echo $out
nix-shell $ make / make install / etc..
# same idea can be used for development environments
# add cowsay, sl to the buildInputs :)
This all is pretty neat
Can i haz an OS around it ?
NixOS
Configuration with NixOS
Declarative Nix file for configuration.
No manual fiddling around (in fact not possible)
Basic NixOS configuration Demo
More NixOS demo
$ ls -l /run
# booted / current system
$ ls -l current-system-path
$ ls -l current-system-path/sw/bin
$ ls -l current-system-path/etc
# Profiles
# same as ~/.nix-profile
$ ls -l /nix/var/nix/profiles/per-user/user
$ ls -l /nix/var/nix/profiles/
$ nix-channel (--list | --add | --update... )
$ nixos-rebuild (switch | test | boot | --rollback ) # --upgrade
# compare /nix/var/nix/profiles/ after a rebuild
$ nix-env --list-generations
$ nix-env --switch-generation N -p /nix/var/nix/profiles/system
$ nix-env --remove-generations 10d -p /nix/var/nix/profiles/system
$ nix-store --optimise
$ nixos-option
Development with Nix
Multiple Rubies + isolated environment
Multiple Pythons + isolated environment
Cabal dependency issues
and the list goes on...
Instead, let Nix manage your project dependencies for you.
nix-shell + cabal example
# Scenario 1 : Your project already has a working .cabal file
$ cabal2nix ./. > default.nix
# Since the generated file above is supposed to be used for Nix packages, few changes are required
# Replace the list of function parameters with below
{ haskellPackages ? (import <nixpkgs> {}).haskellPackages }:
# Prepend the following to the expression body
with haskellPackages;
# Add cabal as the build tool for the project
buildTools = [cabalInstall];
# Based on dependencies not present on Nixpkgs, more other configuration can be done
$ nix-shell --pure
# OMG, no need for sandboxes
[nix-shell] $ cabal build
# Scenario 2 : Start out with a nix expression directly
Development/... environments with Nix
More language ecosystems supported...
Haskell / Ruby / Python / Golang / and more ..
It's just a nix expression.
Same idea can be extended for other environments.
that's it for now...
Hopefully I've got you interested in Nix
and NixOS.
There's a lot more interesting stuff to explore.
Lots of help at #nixos and ##nix-darwin
@suvash
Sources
- http://nixos.org/
- http://nixos.org/nixos/manual/
- https://nixos.org/wiki/Main_Page
- http://www.infoq.com/articles/configuration-management-with-nix
- http://nixos.org/~eelco/pubs/phd-thesis.pdf
- http://nixos.org/~eelco/pubs/nixos-jfp-final.pdf
- http://nixer.ghost.io/why/
- https://www.domenkozar.com/2014/01/02/getting-started-with-nix-package-manager/
- http://www.pavelkogan.com/2014/07/09/haskell-development-with-nix/
- https://ocharles.org.uk/blog/posts/2014-02-04-how-i-develop-with-nixos.html
- https://nixos.org/wiki/Develop_software_with_unique_profiles_using_Nix
Introduction to Nix
By suvash
Introduction to Nix
- 1,916