Rethinking Packaging, Development and Deployment 

Domen Kožar / @iElectric

Nix / Python / JavaScript consultant

PyCon Ukraine 2014

Issues with packaging in Python

Best tool for the job?

Stateful email verification

Stateless email verification

1. Take email of the user and sign it cryptographically (using HMAC)

   from itsdangerous import TimestampSigner
   
   s = TimestampSigner('secret-key')
   hash = s.sign('me@example.com')

2. Send email to the user with the link to our website containing the signature

   send_mail('http://example.com/email/verify/' + hash)

3. Once user has clicked the link in the email, verify digital signature with our private key and if it's valid, we confirm his/hers email

   s.unsign(hash, max_age=24 * 3600)

Stateful packaging

Stateless packaging

Nix project

Purely functional language

/nix/store/r8vvq9kq18pz08v249h8my6r9vs7s0n3-nginx-1.5.13/

• /nix/store is immutable (mounted read-only)
• r8vvq9kq18pz08v249h8my6r9vs7s0n3 is a SHA1 hash of the package function inputs (packaging metadata)

nginx package

{ stdenv, fetchurl, openssl, zlib, pcre, libxml2, libxslt, expat }:

stdenv.mkDerivation rec {
  name = "nginx-${version}";
  version = "1.5.13";

  src = fetchurl {
    url = "http://nginx.org/download/nginx-${version}.tar.gz";
    sha256 = "1f82845mpgmhvm151fhn2cnqjggw9w7cvsqbva9rb320wmc9m63w";
  };

  buildInputs = [ openssl zlib pcre libxml2 libxslt ];
  configureFlags = [ "--with-http_spdy_module" ];

  meta = with stdenv.lib; {
    description = "A reverse proxy and lightweight webserver";
    maintainers = [ maintainers.iElectric ];
    platforms = platforms.all;
    license = licenses.bsd2;
  };
}

Lazy language (hackability)

myNginx = pkgs.nginx.override { openssl = openssl101h; }

myNginx = pkgs.lib.overrideDerivation pkgs.nginx (args:
  src = fetchgit { ... }; 
})

Installing Nix

$ bash <(curl https://nixos.org/nix/install)
$ source ~/.nix-profile/etc/profile.d/nix.sh

Uninstalling Nix

$ rm -rf /nix
$ rm -rf ~/nix-profile/

Profiles / User Environment

Demo

Packaging Python software with Nix

{ fetchurl, buildPythonPackage, freetype, libjpeg, zlib, libtiff, libwebp }:

buildPythonPackage rec {
  name = "Pillow-2.3.0";

  src = fetchurl {
    url = "http://pypi.python.org/packages/source/P/Pillow/${name}.zip";
    sha256 = "0pzm0qk5ilqhwz74pydg1jwrds27vm47185dakdrxidb5bv3b5ia";
  };

  buildInputs = [ freetype libjpeg zlib libtiff libwebp ];
 
  meta = with stdenv.lib; {
    homepage = http://python-imaging.github.com/Pillow;
    description = "Fork of The Python Imaging Library (PIL)";
    license = "http://www.pythonware.com/products/pil/license.htm";
    maintainers = [ maintainers.iElectric ];
    platforms = platforms.linux;
  };
};

Ways to package Python packages for Nix

nix-shell

(virtualenv, but for all software packages)

Using nix-shell to provide tools

$ cat default.nix

with import <nixpkgs> {};
with pkgs.python27Packages; 

buildPythonPackage {
  name = "my-env";

  buildInputs = [ virtualenv pkgs.zlib pkgs.libjpeg pkgs.libxml2 pkgs.git ];

  src = null;
}

Activate nix-shell:

$ nix-shell

Don't preserve current bash environment:

$ nix-shell --pure

Demo: nix-shell and development of
a Python package

NixOS: declarative linux distro

{
  boot.loader.grub.device = "/dev/sda";
  fileSystems."/".device = "/dev/sda1";

  networking.firewall = {
    enable = true;
    allowedTCPPorts = [ 80 ];
  };

  environment.systemPackages = with pkgs; [
    wget
    git
    gnupg
    tmux
  ];
  
  services = {
    sshd.enable = true;

    munin-node.enable = true;
    munin-cron = {
      enable = true;
      hosts = ''
        [${config.networking.hostName}]
        address localhost
      '';
    };
  };
}

Deploying Pyramid with NixOS

let 
  myproject = import ../default.nix {};
  productionini = pkgs.writeText "myproject-production.ini" ''
    ...
  '';
in {
  systemd.services.myproject = {
    after = [ "network.target" ];
    description = "My Project";
    wantedBy = [ "multi-user.target" ];
    path = [ myproject ];
    environment.PYTHONPATH = "${myproject}/lib/python2.7/site-packages";
    serviceConfig = {
      ExecStart = "${pkgs.pythonPackages.pyramid}/bin/pserve ${productionini}";
      User = "myuser";
      Group = "myuser";
    };
  };
}

Getting started with NixOps

$ nix-env -i nixops

trivial.nix:

{
  webserver = { config, pkgs, ... }:
    { services.httpd.enable = true;
      services.httpd.adminAddr = "alice@example.org";
      services.httpd.documentRoot = "${pkgs.valgrind}/share/doc/valgrind/html";
    };
}

trivial-virtualbox.nix:

{
  webserver = { config, pkgs, ... }:
    { deployment.targetEnv = "virtualbox";
      deployment.virtualbox.memorySize = 1024; # megabytes
    };
}

Provision and deploy

$ nixops create -d trivial ./trivial.nix ./trivial-virtualbox.nix 33bced96-5f26-11e1-b9d7-9630d48abec1

$ nixops deploy -d trivial
creating VirtualBox VM ‘webserver’...

 Nixops Demo

2nd NixOS Sprint @ Ljubljana, Slovenia

Enjoy the purity! Questions?

• #nixos on Freenode IRC
• http://nixos.org

(Shameless plugs)

• Why Puppet/Chef/Ansible aren't good enough (and we can do better) - domenkozar.com
• Getting started with Nix Package Manager - domenkozar.com

• NixOS: declarative configuration Linux Distribution - FOSDEM 2014