Nix: the functional Package Manager

and

@ Lambda World Cádiz 2015

declarative Linux distribution

NixOS

Problem?

Functional approach

GHC runtime dependency graph

Nix

  • purely functional language to describe how to build packages and their dependencies
  • build results only depend on declared inputs.
  • packages never change after they have been built (they're immutable)
  • "The Purely Functional Software Deployment Model" - Eelco Dolstra PhD thesis (2006)
  • solves dependency hell set of problems

Nix

  • lazy evaluated, dynamically typed
  • C++
  • runs on POSIX (Linux / OS X / FreeBSD/ Cygwin)
  • (could work on Windows, if there was interest)
  • multi-user support
  • source/binary model
$ du -sh $(nix-store -qR `nix-build -A ghc`)
5.1M    /nix/store/5l472kj6h5zvccpzp0zcdpc1w2njfd58-linux-headers-3.12.32
34M     /nix/store/hd6km3hscbgl2yw8nx7lr5z9s8h89p04-glibc-2.21
364K    /nix/store/31w31mc8immhpnmxvcl4l0fvc3i5iwh0-zlib-1.2.8
5.1M    /nix/store/xag5ayq906w9zhlxs8wayv4kvpiyqphq-bash-4.3-p42
12M     /nix/store/4i01mpa2s194m44xrz41wmf4yr49jysq-ncurses-5.9
336K    /nix/store/8z667vv1agvpd3iknmk94j0ix6bv413i-attr-2.4.47
472K    /nix/store/sk0b1r840b686zc2m8mzyw8yyq1aymqh-acl-2.2.52
16M     /nix/store/8xfxnm4a7jvyh0i1sd967z6rws8lh21a-coreutils-8.24
98M     /nix/store/i9nn1fkcy95dzf0hb9wi8gbkid3iw1sa-gcc-4.9.3
21M     /nix/store/kxf1p7l7lgm6j5mjzkiwcwzc98s9f1az-binutils-2.23.1
92K     /nix/store/4n9yj90rdggqcx34ylbshryg3rf9cqpj-gcc-wrapper-4.9.3
46M     /nix/store/a3x9hggwq83f3fk88lqvgh6jvlklhk36-perl-5.20.2
2.7M    /nix/store/i4g323b60pskbayk4awl3ffz8m365264-gmp-5.1.3
999M    /nix/store/7hnlpxpmpka84zrhyq7z11bs0q3vgak1-ghc-7.10.2

GHC closure (runtime dependencies)

$ grep -R /nix/store/a3x9...-perl-5.20.2 /nix/store/7hnl...-ghc-7.10.2
/nix/store/7hnl...-ghc-7.10.2/lib/ghc-7.10.2/ghc-split:#!/nix/store/a3x9...-perl-5.20.2/bin/perl
/nix/store/7hnl...-ghc-7.10.2/lib/ghc-7.10.2/settings: ("perl command", "/nix/store/a3x9...-perl-5.20.2/bin/perl"),

Perl?!

$ tree `nix-build -A zlib`
/nix/store/31w31mc8immhpnmxvcl4l0fvc3i5iwh0-zlib-1.2.8
├── include
│   ├── zconf.h
│   └── zlib.h
├── lib
│   ├── libz.a
│   ├── libz.so -> libz.so.1.2.8
│   ├── libz.so.1 -> libz.so.1.2.8
│   ├── libz.so.1.2.8
│   └── pkgconfig
│       └── zlib.pc
└── share
    └── man
        └── man3
            └── zlib.3.gz

$PREFIX

$ ldd `which perl`
        linux-vdso.so.1 (0x00007ffc9fdd0000)
        libperl.so => /nix/store/lhgkssg79bxdba541h75mgdvjh35b8ic-perl-5.20.2/lib/perl5/5.20.2/x86_64-linux-thread-multi/CORE/libperl.so (0x00007fa537f3b000)
        libnsl.so.1 => /nix/store/npfsi1d9ka8zwnxzn3sr08hbwvpapyk7-glibc-2.21/lib/libnsl.so.1 (0x00007fa537d23000)
        libdl.so.2 => /nix/store/npfsi1d9ka8zwnxzn3sr08hbwvpapyk7-glibc-2.21/lib/libdl.so.2 (0x00007fa537b1f000)
        libm.so.6 => /nix/store/npfsi1d9ka8zwnxzn3sr08hbwvpapyk7-glibc-2.21/lib/libm.so.6 (0x00007fa53781c000)
        libcrypt.so.1 => /nix/store/npfsi1d9ka8zwnxzn3sr08hbwvpapyk7-glibc-2.21/lib/libcrypt.so.1 (0x00007fa5375e1000)
        libutil.so.1 => /nix/store/npfsi1d9ka8zwnxzn3sr08hbwvpapyk7-glibc-2.21/lib/libutil.so.1 (0x00007fa5373de000)
        libpthread.so.0 => /nix/store/npfsi1d9ka8zwnxzn3sr08hbwvpapyk7-glibc-2.21/lib/libpthread.so.0 (0x00007fa5371c1000)
        libc.so.6 => /nix/store/npfsi1d9ka8zwnxzn3sr08hbwvpapyk7-glibc-2.21/lib/libc.so.6 (0x00007fa536e21000)
        /nix/store/npfsi1d9ka8zwnxzn3sr08hbwvpapyk7-glibc-2.21/lib/ld-linux-x86-64.so.2 (0x00007fa5382f4000)

Dynamic libraries are linked deterministically

Profiles and User Environments

$ cat default.nix
derivation {
  name = "my-package";
  builder = ./builder.sh;
  system = "x86-64-linux";
  src = /home/user/bla.tar.gz;
}
$ cat builder.sh
tar xvfz $src
cd plan9port/
mkdir $out
cp planport.sh $out/bin/planport
$ nix-build
{ stdenv, fetchurl, fetchgit, openssl, zlib, pcre, libxml2 }:

stdenv.mkDerivation rec {
  name = "nginx-${version}";
  version = "1.4.4";

  src = fetchurl {
    url = "http://nginx.org/download/nginx-${version}.tar.gz";
    sha256 = "1f82845mpgmhvm151fhn2cnqjggw9w7cvsqbva9rb320wmc9m63w";
  };

  buildInputs = [ openssl zlib pcre libxml2 ];
  configureFlags = [ "--with-http_spdy_module" ];
  postInstall = "mv $out/sbin $out/bin";

  meta = with stdenv.lib; {
    description = "A reverse proxy and lightweight webserver";
    maintainers = [ maintainers.domenkozar ];
    platforms = platforms.all;
    license = licenses.bsd2;
  };
}

Purity

src = fetchurl {
  url = "http://nginx.org/download/nginx-${version}.tar.gz";
  sha256 = "1f82845mpgmhvm151fhn2cnqjggw9w7cvsqbva9rb320wmc9m63w";
};

fixed output derivation

/nix/store/r8vvq9kq18pz08v249h8my6r9vs7s0n3-nginx-1.5.13/

  • functional language
  • chroot
  • disabled networking
  • patch shebangs
  • /nix/store/ mounted as read-only
  • patches to tooling

A package is an output of a pure function (it depends only on the function inputs, without any side effects) 

Lazy language (hackable)

Override lambda function parameters for the file

libreNginx = nginx.override { openssl = libressl; }

Override derivation attributes

myNginx = nginx.overrideDerivation  (super:
  name = "nginx-git";
  src = fetchFromGitHub {
    sha256 = "04s7xcgmi5g58lirr48vf203n1jwdxf981x1p6ysbax24qwhs2kd";
    rev = "c0eb2f0759726f47bd06f5c8f46739f43ce55cac";
    repo = "nginx";
    owner = "nginx";
  };
})
$ git clone https://github.com/NixOS/nixpkgs.git && cd nixpkgs
$ git grep "nginx = "
pkgs/top-level/all-packages.nix:  nginx = callPackage ../servers/http/nginx { };

nix-shell - start an interactive shell based on a Nix expression

Haskell

  • ~50% of the packages available from Hackage
  • well over 8,000 builds in total

  • Stack support

  • 60+ Haskell package sets

  • maintained by Peter Simons

$ nix-build
/nix/store/4jzj9qwknqz24cx6zvn7y4cmnq19hvv4-my-ghc-app-0.0.1
$ ls -la result
result -> /nix/store/4jzj9qwknqz24cx6zvn7y4cmnq19hvv4-my-ghc-app-0.0.1

$ ./result/bin/mybin
>>>
$ nix-shell --pure
$ nix-copy-closure --sign result/ domenkozar@example.com

Build the app

Run it

Develop the app

Deploy it

Toolset

$ cabal2nix --shell . > shell.nix

Generate Nix expresssion based on Cabal

NixOS: packages + systemd + Linux kernel

{
  boot.loader.grub.device = "/dev/sda";
  fileSystems."/".device = "/dev/sda1";

  networking.firewall = {
    enable = true;
    allowedTCPPorts = [ 80 ];
  };

  environment.systemPackages = with pkgs; [
    wget
    git
    gnupg
    tmux
  ];
  
  services = {
    sshd.enable = true;
    munin-node.enable = true;
    munin-cron = {
      enable = true;
      hosts = ''
        [${config.networking.hostName}]
        address localhost
      '';
    };
  };
}

NixOS modules

{ config, lib, pkgs, ... }:

let
  configFile = pkgs.writeText "nginx.conf" ''
    user nginx nginx
    daemon off;
  '';
in {
  options = {
    services.nginx = {
      enable = lib.mkOption {
        default = false;
        description = "Enable the nginx Web Server.";
        types = types.bool;
      };
    };
  };

  config = mkIf config.services.nginx.enable {
    environment.systemPackages = [ pkgs.nginx ];

    systemd.services.nginx = {
      after = [ "network.target" ];
      wantedBy = [ "multi-user.target" ];
      path = [ pkgs.nginx ];
      serviceConfig = {
        ExecStart = "${nginx}/bin/nginx -c ${configFile}";
      };
    };
  };
}

Rollback!

Getting started with NixOps

$ nix-env -i nixops

trivial.nix

{
  webserver = { config, pkgs, ... }:
    { services.httpd.enable = true;
      services.httpd.adminAddr = "alice@example.org";
      services.httpd.documentRoot = "${pkgs.valgrind}/share/doc/valgrind/html";
    };
}

trivial-virtualbox.nix

{
  webserver = { config, pkgs, ... }:
    { deployment.targetEnv = "virtualbox";
      deployment.virtualbox.memorySize = 1024; # megabytes
    };
}

Provision and deploy

$ nixops create ./trivial.nix ./trivial-virtualbox.nix -d trivial
33bced96-5f26-11e1-b9d7-9630d48abec1

$ nixops deploy -d trivial
creating VirtualBox VM ‘webserver’...
...

Hydra build farm

Challanges

  • private nix store files
  • P2P / IPFS distribution of binary packages
  • statically typed Nix (2.0?)
  • get rid of Perl as dependency in Nix source
  • native Windows support
  • multiple outputs
  • binary determinism
  • low-level speed/memory improvements
  • GuixSD

 

conf.nixos.org

14th and 15 November @ Berlin

2 days of talks by core contributors

Thanks!

Nix the functional Package Manager and NixOS the declarative Linux distribution

By Domen Kožar

Nix the functional Package Manager and NixOS the declarative Linux distribution

  • 4,324