NIX usage on HPC clusters

GriCAD feedback

Pierre-Antoine Bouttier

Workshop Software deployement for HPC - 05/31/2018

Too Long ; Didn't Listen

Nix overview

 

Nix concepts

 

Nix In Real Life (on our clusters)

 

Nix overview

Nix, the package manager

  • Functional package manager (no side effect)
  • Reliable and repoducible (research, experimentation)
  • Available on Linux and OS X (portability)
  • Packages creation/installation without root privileges (Cool for HPC clusters!)

Nix overview

Nixpgs, the main package repository

  • Around 6,500 packages
  • No external dependency
  • And you can set up your own repository (also called channel)

 

NixOS exists

  • "The Purely Functional Linux Distribution"

Nix concepts

 

 

 

 

Package definition

A package is described in an functional language (Haskell-like)

  •  
{ stdenv, fetchurl }:

stdenv.mkDerivation rec {
  pname = "cutee";
  version = "0.4.2";
  name = "${pname}-${version}";

  src = fetchurl {
    url    = "http://www.codesink.org/download/${pname}-${version}.tar.gz";
    sha256 = "18bzvhzx8k24mpcim5669n3wg9hd0sfsxj8zjpbr24hywrlppgc2";
  };

  buildFlags = "cutee";

  installPhase = ''
    mkdir -p $out/bin
    cp cutee $out/bin
  '';

  meta = with stdenv.lib; {
    description = "C++ Unit Testing Easy Environment";
    homepage    = http://www.codesink.org/cutee_unit_testing.html;
    license     = licenses.gpl2Plus;
    maintainers = with maintainers; [ leenaars];
    platforms = platforms.linux;
  };
}

Nix concepts

Packages storage in one directory

 

=> Independant from the system

/nix/store

Nix concepts

/nix/store/an9dli66ng2jzvqf13b2i230mm9fq7qk-cdo-1.7.2

# Hash = mix(sources + deps + flags, ...) 
# Compilation + nouvelle option → nouveau hash

/nix/store/srf6grrfy9vkc9fsplk8xk292lm8jvz5-cdo-1.7.2

Packages identification through Hash;

Package construction = One sub-directory

=> Dependancies tree conserved

=> packages unicity

Nix  concepts

Profiles (1/3)

  • Package installation: creation of symlink in the user /home to /nix/store
     
  • A nix profile: environment defined by a set of symlinks, stored in the user /home
     
  • Current profile
export PATH=~/.nix-profile/bin:$PATH

Nix  concepts

Profiles (2/3)

Nix  concepts

Profiles (3/3)

 

  • Users can create as many profiles as they want
  • What is my current profile? ls -al ~/.nix-profile
  • Create/switch between profiles? nix-env --switch-profile $NIX_USER_PROFILE_DIR/some_name
  • What are my available profiles? ls -al $NIX_USER_PROFILE_DIR
  • Profile "versioning" (rollback)
  • Play with profiles!

Nix concepts

Channels

  • Nixpkgs snapshot + other packages
  • Ex: nixpkgs-unstable, ciment-channel...

     

Binary-cache

  • web server that provides prebuilt library)
  • Avoid compilation of an already installed software
  • A shared binary-cache: very useful for us
     
nix-env -i -A nixpkgs-unstable.openmpi
nix-env -i -A ciment-channel.openmpi

Nix in real life

(on our clusters)

HPC cluster

  A set of interconnected Linux computing nodes and one or several head nodes sharing some common network filesystems

 

 

Nix in real life

(on our clusters)

The /nix store

  • /nix directory shared on all the computing and head nodes.
  • set-up a mount on an NFS filesystem on all of your nodes

 

 

  # fstab entry for NIX
  head1:/home/nix          /nix    nfs     defaults        0       0

Nix in real life

(on our clusters)

 

Nix installation on the cluster - Main steps

 

  • Install nix tools as a module
  • Set up the nix daemon (security of the shared nix store)
  • Multi-user profile script

 

 

Nix in real life

(on our clusters)

Multi-user profile script

 

  • sets the PATH to nix tools binaries
  • sets the NIX_PATH variable
  • initializes per-user directories and configuration files
  • sets the NIX_REMOTE variable that is necessary to use the NIX daemon

 

  source /applis/site/nix.sh

Nix in real life

(on our clusters)

The CIMENT channel

 

  • A copy of nixpkgs-stable channel...
  • ...plus some packages...
    • ...e.g. intel compilers...
    • or research codes (possibly go upstream)

Shared binary cache

Nix in real life

(on our clusters)

Our workflow

Nix in real life

(on our clusters)

 

Pros

  • Reproducible and reliable
     
  • Users can install and create their own packages
     
  • Portable
     
  • For the most common packages, with a shared binary-cache, quick installation
     
  • Contribution to a living community

Nix in real life

(on our clusters)

 

Cons

  • The language learning curve is steep...
     
  • ...especially for IT beginners
     
  • Difficulty to have generic and easy packaging solution for some coding language: python...

 

Thank you

Questions/remarks?

nix_feedback

By pabouttier

Made with Slides.com

nix_feedback

  • 1,567

More from pabouttier