Introduction to Nix (and NixOS)

Suvash Thapaliya ~ @suvash

25th February 2015

Disclaimer

I'm pretty sure I don't know a lot things.

But, please feel free to ask questions !

The Problem

You UPGRADE ONE PACKAGE & find OTHER PACKAGES on your system NO LONGER WORKS. Turns out some shared dependencies got upgraded as well, and they’re not compatible. 

You upgraded _______ and discover it no longer works.

There is NO easy way to UNDO the upgrade.

Your server dies. But because of MANUAL CHANGES over time you CANNOT exactly REPRODUCE what state of CONFIGURATION the server was in. (Hopefully not).

Your team develops an application that has many DEPENDENCIES. Your developers waste a lot of time setting up their build environments. Every time a dependency CHANGES, EVERYBODY on the team needs to UPGRADE their environments MANUALLY.

You've installed a bunch of things(and their dependencies). During UNINSTALL you forget/don't want to bother CLEANING UP all the non used DEPENDENCIES that were pulled in.

Some solutions....?

Servers

Chef

Puppet

Ansible....

Containers

Docker

LXC

Rocket ...

Applications

Bundler + rvm/rbenv

Pip + virtualenv

Cabal + sandboxes

....

Configuration

Management

Declarative

Purely Functional

Nix(OS) Family

Nix expressions

(language)

Nix package manager

Nixpkgs

Hydra

Nix Build System

NixOS

Linux + Nixpkgs

NixOps

NixOS + Operations tooling

Things we won't quite cover...

Nix expressions

(language)

Hydra

Nix Build System

NixOps

NixOS + Operations tooling

Why Nix ?

No update/grade-in-place (or immutable updates)

(as compared to apt/brew/yum/you-name-it....)

Atomic upgrades (or upgrades are transactional)

(if system crashes half-way through an upgrade it will still work....)

Cheap and instantaneous rollbacks

(guess why....)

Reproducibility

(with the exact same cryptographic hash....)

Origin

"The Purely Functional Software Deployment Model"

- PhD Thesis, Eelco Dolstra, 2006

"NixOS: A Purely Functional Linux Distribution"

- Eelco Dolstra and Andres Löh, 2008

and more .....

http://nixos.org/docs/papers.html

Philosophy

"The basic philosophy behind Nix - to solve deployment problems by treating them analogously to memory management issues in programming languages"

- The Purely Functional Software Deployment Model

  • Nix is a system for software deployment
  • Components(a set of files/tree in a filesystem) are the basic units of deployment
  • Nix stores components in a component store, called the Nix store (usually /nix/store)
  • The names of the components contain a cryptographic hash of all the inputs involved in building the component. (eg. /nix/store/b6q64986n0smwwfwrhg655lcg3pxhk56-git-2.1.4 )
  • The collision resistant name is what provides Nix the ability to prevent undeclared dependencies and a component interface

Components & Closures

Hash is computed over all inputs

  • Source of components
  • the build script
  • arguments or env. variables passed to build script
  • all build time dependencies (compiler, linker, libraries, shell, etc.)
$ gcc -o main.c -lssl

# ssl lib already installed

# would not work in nix
# unless explicitly provided
# as an input to the build

Demo

# nix-env manages user-environments 
# which are 'views' into the available components

$ nix-env -i cowsay

$ which cowsay

$ ls -lah $(which cowsay)

$ realpath $(which cowsay)

$ readlink -f $(which cowsay)

$ nix-env --list-generation

$ nix-env --rollback

$ which cowsay

$ nix-env --switch-generation X

$ which cowsay

$ nix-env --uninstall cowsay

$ nix-env --delete-generations 10d

$ nix-collect-garbage

More Demo

# pick the symlink to ~/.nix-profile

$ ls -l ~/
$ ls -l ~/.nix-profile

# up the profile directory
$ ls -l <symlink-to-nix-profile-without-the-profile-directory-at-the-end>

# where is it located actually (can be seen above too) ?
$ realpath ~/.nix-profile

# Custom profile
$ which cowsay

$ cd <some-dir>

$ nix-env -i cowsay -p custom-profile

$ ls -l

$ ./custom-profile/bin/cowsay -f dragon Nix is awesome

$ realpath ./custom-profile

$ which sl

$ nix-env -i sl -p custom-profile

$ ./custom-profile/bin/sl

$ nix-env --list-generation -p custom-profile

# compare the hash of custom profile and 'normal' profile
$ nix-env -i sl 

# uninstall sl, remove custom profile, garbage collect and path no more

Purely functional : What does it mean ?

(in this context)

The Nix expression language is lazy and functional.

Package builds are intended to be pure i.e. only depend on the declared inputs

Nix 'tries hard to enforce this : no network downloads, no using other programs(/usr/..), clear the environment/PATH during build, optionally build in a chroot jail,  etc.

​*but it cannot guarantee purity in the absolute sense

(Nix cannot help if the build script/os depends on the phase of moon.)

The results of building a package - the paths to the Nix store - can be seen as a purely functional persistent data structure.

immutable objects that refer to each other, garbage-collected when unreachable.

Persistent Data Structures

Example Package

with import <nixpkgs> { };

stdenv.mkDerivation rec {
  pname = "mosquitto";
  version = "1.4";

  name = "${pname}-${version}";

  src = fetchurl {
    url = http://mosquitto.org/files/source/mosquitto-1.4.tar.gz;
    sha256 = "1imw5ps0cqda41b574k8hgz9gdr8yy58f76fg8gw14pdnvf3l7sr";
  };

  buildInputs = [ openssl libuuid ];

  buildFlags = "mosquitto";

  installPhase = ''
    mkdir -p $out/bin
    cp src/mosquitto $out/bin/
  '';

  meta = {
    homepage = http://mosquitto.org/;
    description = "An open source MQTT v3.1/3.1.1 broker";
    platforms = stdenv.lib.platforms.linux;
    license = stdenv.lib.licenses.epl10;
  };
}

Custom Package Demo ?

# nix-build

$ nix-build ./mosquitto.nix

$ ls -l

$ ./result/bin/mosquitto



# nix-shell

$ nix-shell ./mosquitto.nix

nix-shell $ echo $src

nix-shell $ echo $out

nix-shell $ make / make install / etc..



# same idea can be used for development environments

# add cowsay, sl to the buildInputs :)

This all is pretty neat

Can i haz an OS around it ?

NixOS

Configuration with NixOS

Declarative Nix file for configuration.

 

No manual fiddling around (in fact not possible)

Basic NixOS configuration Demo

More NixOS demo

$ ls -l /run

# booted / current system

$ ls -l current-system-path

$ ls -l current-system-path/sw/bin

$ ls -l current-system-path/etc

# Profiles

# same as ~/.nix-profile
$ ls -l /nix/var/nix/profiles/per-user/user

$ ls -l /nix/var/nix/profiles/

$ nix-channel (--list | --add | --update... )

$ nixos-rebuild (switch | test | boot | --rollback )  # --upgrade

# compare /nix/var/nix/profiles/ after a rebuild

$ nix-env --list-generations

$ nix-env --switch-generation N -p /nix/var/nix/profiles/system

$ nix-env --remove-generations 10d -p /nix/var/nix/profiles/system

$ nix-store --optimise

$ nixos-option

Development with Nix

Multiple Rubies + isolated environment

 

Multiple Pythons + isolated environment

 

Cabal dependency issues

 

and the list goes on...

Instead, let Nix manage your project dependencies for you.

nix-shell + cabal example

# Scenario 1 : Your project already has a working .cabal file

$ cabal2nix ./. > default.nix

# Since the generated file above is supposed to be used for Nix packages, few changes are required

# Replace the list of function parameters with below

{ haskellPackages ? (import <nixpkgs> {}).haskellPackages }:

# Prepend the following to the expression body

with haskellPackages;

# Add cabal as the build tool for the project

buildTools = [cabalInstall];

# Based on dependencies not present on Nixpkgs, more other configuration can be done

$ nix-shell --pure

# OMG, no need for sandboxes

[nix-shell] $ cabal build


# Scenario 2 : Start out with a nix expression directly

Development/... environments with Nix

More language ecosystems supported...

Haskell / Ruby / Python / Golang / and more ..

It's just a nix expression.

Same idea can be extended for other environments.

that's it for now...

Hopefully I've got you interested in Nix

and NixOS.

 

There's a lot more interesting stuff to explore.

Lots of help at #nixos and ##nix-darwin

@suvash

Sources

  • http://nixos.org/
  • http://nixos.org/nixos/manual/
  • https://nixos.org/wiki/Main_Page
  • http://www.infoq.com/articles/configuration-management-with-nix
  • http://nixos.org/~eelco/pubs/phd-thesis.pdf
  • http://nixos.org/~eelco/pubs/nixos-jfp-final.pdf
  • http://nixer.ghost.io/why/
  • https://www.domenkozar.com/2014/01/02/getting-started-with-nix-package-manager/
  • http://www.pavelkogan.com/2014/07/09/haskell-development-with-nix/
  • https://ocharles.org.uk/blog/posts/2014-02-04-how-i-develop-with-nixos.html
  • https://nixos.org/wiki/Develop_software_with_unique_profiles_using_Nix

Introduction to Nix

By suvash

Introduction to Nix

  • 1,866