Rethinking Packaging, Development and Deployment 

Domen Kožar / @iElectric

Nix / Python / JavaScript freelancer

EuroPython 2014

Core issues with packaging in Python

Best tool for the job?

Nix project

Purely functional language

• Software package is an output of a function that is deterministic (it depends only on function inputs, without any side effects)

/nix/store/r8vvq9kq18pz08v249h8my6r9vs7s0n3-nginx-1.5.13/

• /nix/store is immutable (mounted read-only)
• r8vvq9kq18pz08v249h8my6r9vs7s0n3 is crypto hash of function inputs
  

nginx package

{ stdenv, fetchurl, openssl, zlib, pcre, libxml2, libxslt, expat }:

stdenv.mkDerivation rec {
  name = "nginx-${version}";
  version = "1.5.13";

  src = fetchurl {
    url = "http://nginx.org/download/nginx-${version}.tar.gz";
    sha256 = "1f82845mpgmhvm151fhn2cnqjggw9w7cvsqbva9rb320wmc9m63w";
  };

  buildInputs = [ openssl zlib pcre libxml2 libxslt ];
  configureFlags = [ "--with-http_spdy_module" ];

  meta = with stdenv.lib; {
    description = "A reverse proxy and lightweight webserver";
    maintainers = [ maintainers.iElectric ];
    platforms = platforms.all;
    license = licenses.bsd2;
  };
}

Lazy (extensible)

myNginx = pkgs.nginx.override { openssl = openssl101h; }

myNginx = pkgs.lib.overrideDerivation pkgs.nginx (args:
  src = fetchgit { ... }; 
})

Installing Nix

$ bash <(curl https://nixos.org/nix/install)
$ source ~/.nix-profile/etc/profile.d/nix.sh

Uninstalling Nix

$ rm -rf /nix
$ rm -rf ~/nix-profile/

Profiles / User Environment

Demo

Packaging Python software with Nix

{ fetchurl, buildPythonPackage }:

buildPythonPackage rec {
  name = "Pillow-2.3.0";

  src = fetchurl {
    url = "http://pypi.python.org/packages/source/P/Pillow/${name}.zip";
    md5 = "56b6614499aacb7d6b5983c4914daea7";
  };

  buildInputs = [ pkgs.freetype pkgs.libjpeg pkgs.zlib pkgs.libtiff pkgs.libwebp ];
 
  meta = with stdenv.lib; {
    homepage = http://python-imaging.github.com/Pillow;
    description = "Fork of The Python Imaging Library (PIL)";
    license = "http://www.pythonware.com/products/pil/license.htm";
    maintainers = [ maintainers.iElectric ];
  };
};

Ways to package Python packages for Nix

nix-shell

(virtualenv, but for all software packages)

Using nix-shell to provide tools

$ cat default.nix

with import <nixpkgs> {};
with pkgs.python27Packages; 

buildPythonPackage {
  name = "some-env";

  buildInputs = [ virtualenv mysql psycopg2 pillow git ];

  src = null;
}

Activate nix-shell:

$ nix-shell

Don't preserve current bash environment:

$ nix-shell --pure

Demo

development of
a Python package demo

NixOS

{
  boot.loader.grub.device = "/dev/sda";
  fileSystems."/".device = "/dev/sda1";

  networking.firewall = {
    enable = true;
    allowedTCPPorts = [ 80 ];
  };

  environment.systemPackages = with pkgs; [
    wget
    git
    gnupg
    tmux
  ];
  
  services = {
    sshd.enable = true;

    munin-node.enable = true;
    munin-cron = {
      enable = true;
      hosts = ''
        [${config.networking.hostName}]
        address localhost
      '';
    };
  };
}

Deploying Pyramid with NixOS

let 
  myproject = import ../default.nix {};
  productionini = pkgs.writeText "myproject-production.ini" ''
    ...
  '';
in {
  systemd.services.myproject = {
    after = [ "network.target" ];
    description = "My Project";
    wantedBy = [ "multi-user.target" ];
    path = [ myproject ];
    environment.PYTHONPATH = "${myproject}/lib/python2.7/site-packages";
    serviceConfig = {
      ExecStart = "${pkgs.pythonPackages.pyramid}/bin/pserve ${productionini}";
      User = "myuser";
      Group = "myuser";
    };
  };
}

Getting started with NixOps

$ nix-env -i nixops

trivial.nix:

{
  webserver = { config, pkgs, ... }:
    { services.httpd.enable = true;
      services.httpd.adminAddr = "alice@example.org";
      services.httpd.documentRoot = "${pkgs.valgrind}/share/doc/valgrind/html";
    };
}

trivial-virtualbox.nix:

{
  webserver = { config, pkgs, ... }:
    { deployment.targetEnv = "virtualbox";
      deployment.virtualbox.memorySize = 1024; # megabytes
    };
}

Provision and deploy

$ nixops create -d trivial ./trivial.nix ./trivial-virtualbox.nix 33bced96-5f26-11e1-b9d7-9630d48abec1

$ nixops deploy -d trivial
creating VirtualBox VM ‘webserver’...

Demo

NixOS Sprint @ Ljubljana, Slovenia

23rd to 27th of August 2014

Enjoy the purity! Questions?

• #nixos on Freenode
• http://nixos.org

(Shameless plugs)

• Why Puppet/Chef/Ansible aren't good enough (and we can do better) - domenkozar.com

• NixOS: declarative configuration Linux Distribution - FOSDEM 2014