Introduction to Nix

Ruby meetup

Ljubljana, June 11th 2024

by Simon Žlender

About me

Simon Žlender, GitHub: @szlend
Engineering Manager at "a fintech company"
I lead the Development Experience team
My toolset: Rust / Elixir / ex-Ruby
Jackhammer of all tools: Nix

👋

It works fine on my machine™

"Probably something weird on your machine"

"Just read my 3 page long outdated setup guide"
"Try reinstalling Ruby"
"Did you symlink OpenSSL v1.0.2h to /usr/lib?"

"Bro just build with --with-xml2-dir=/please/help"

Your dev environment is not

We 🤍 Gemfile.lock

So why don't we have lock files for system dependencies?

"Just use Docker"

"Why is the Rails server on my Mac now 50% slower?"

"Wait why are these files now owned by root?"

"I need Arm images for MacOS now? Multi-arch what?"

"Debuggers are for people who write shit code."

"OK maybe debuggers are useful sometimes. I'll just run my code editor in Docker as well."

"Why are these walls soft-padded?"

Sane people use Asdf

Toolchain version management
Really good for ruby, node, etc.
Lacks the really tricky bits:

It can't manage system libraries

"We have system lock files at home"

It's called Nix

So Nix is a package manager?

Nix manages packages yes
Cross-platform (Linux/MacOS)
Nixpkgs - Official Nix package repository
Channels - nixpkgs-unstable / nixos-24.11
Over 90.000 packages
Community maintained
Largest package repository out there*

* as tracked by repology.org

Different package versions can co-exist

/nix/store/23fsf8idl4s1awpxra2r51wfqwsc0d6d-rustc-1.67.1
/nix/store/abhjlgf3rmnckl54nfj24gh82ygzk9b4-rustc-1.70.0
/nix/store/japwpy9z6fndzakc17r1sw1pkbsk0961-rustc-1.71.1

Packages are symlinked into profiles

~/.nix-profile (included in PATH)
├── bin
│   ├── aws   -> /nix/store/fqii9...-aws-cli2/bin/aws
│   ├── cargo -> /nix/store/w7hr1...-rustup-1.75.0/bin/cargo
...

Package manager, but also...

Nix - the sandboxed build tool
Nix - the programmable configuration language
NixOS - the Nix-based Linux operating system

NO

OK, slow down. So I want to install a package. I should just

`nix-whatever-install ruby` right?

DO NOT USE THE NIX CLI TO MANAGE PACKAGES

Please for your sanity's sake...

But why...

Nobody experienced with Nix actually uses the CLI to manage packages
As a result, nobody cares about making the UX better
Just don't. It's not a thing. Forget it exists.

~~nix profile install~~

~~nix-env -iA~~

OK so what should I use then?

What is a dev shell?

It's just a shell with a bunch of injected environmental variables
Most notably:
- PATH - where to look for applications
- CC - the path to the C compiler
- CFLAGS - C compiler flags
- LDFLAGS - system linker flags
- etc...
Nix can build a dev shell that suits your exact needs

Let's write a Rails dev shell

pkgs.mkShell {
  packages = [
    pkgs.ruby_3_3
    pkgs.rubyPackages_3_3.rails
  ];
}

$ nix develop

$ echo $PATH
/nix/store/pkp3jxxrv5k2q30d7dlrhi3iavf7yx30-ruby-3.3.1/bin # etc...

$ which ruby
/nix/store/pkp3jxxrv5k2q30d7dlrhi3iavf7yx30-ruby-3.3.1/bin/ruby

$ ruby --version
ruby 3.3.1 (2024-04-23 revision c56cd86388) [arm64-darwin23]

$ rails --version
Rails 7.1.3.2

$ rails new -n my-app .

Bundle complete! 14 Gemfile dependencies, 82 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.

Let's enter the shell

We can now generate a Rails project

$ bundle exec rails s

=> Booting Puma
=> Rails 7.1.3.2 application starting in development
...
* Listening on http://127.0.0.1:3000

$ bundle add rmagick

Installing rmagick 6.0.1 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

...

pkg-config.rb:504:in `parse_pc': .pc doesn't exist: <MagickCore> (PackageConfig::NotFoundError)

Ok now let's add everyone's favorite image manipulation tool!

Oh boy, it's one of those days again.

Let me open that bottle of rakija and hope ~~Google~~ ChatGPT is up to date on this one.

On second thought, forget AI. Let's use some actual brain cells for this one.

pkg-config.rb:504:in `parse_pc': .pc doesn't exist: <MagickCore> (PackageConfig::NotFoundError)

Here's our first clue. Something called pkg-config is complaining about a missing Package Config for MagickCore.

Install pkg-config

Rule #1 when dealing with build errors

I CANT STRESS THIS ENOUGH. PKG-CONFIG WILL SOLVE 90% OF ALL YOUR LIFE PROBLEMS.

Rule #2: RTFM (Read the friendly manual)

pkgs.mkShell {
  packages = [
    pkgs.ruby_3_3
    pkgs.rubyPackages_3_3.rails
    pkgs.pkg-config
    pkgs.imagemagick
  ];
};

Shocking, I know. The rmagick README says to install imagemagick libs. Let's put them in the shell.

$ $ pkg-config --list-all
...
ImageMagick    convert, edit, and compose images (ABI Q16HDRI)


$ bundle add rmagick
Installing rmagick 6.0.1 with native extensions

Let's try this again:

Congratulations!

You have not only configured rmagick for yourself. But also for your colleagues and future self.

And you didn't have to write a 3 page setup manual that nobody will read.

Full devshell for fellow boilerplate enjoyers

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { nixpkgs, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs { system = system; };
      in
      {
        devShells.default = pkgs.mkShell {
          packages = [
            pkgs.ruby_3_3
            pkgs.rubyPackages_3_3.rails
            pkgs.pkg-config
            pkgs.imagemagick
          ];
        };
      });
}

Inputs are locked in flake.lock

{
  "nodes": {
    "nixpkgs": {
      "locked": {
        "lastModified": 1717786204,
        "narHash": "sha256-4q0s6m0GUcN7q+Y2DqD27iLvbcd1G50T2lv08kKxkSI=",
        "owner": "nixos",
        "repo": "nixpkgs",
        "rev": "051f920625ab5aabe37c920346e3e69d7d34400e",
        "type": "github"
      },
  ...
}

$ nix flake update
• Updated input 'nixpkgs':
    'github:nixos/nixpkgs/b1d9ab70662946ef0850d488da1c9019f3a9752a' (2024-06-02)
  → 'github:nixos/nixpkgs/051f920625ab5aabe37c920346e3e69d7d34400e' (2024-06-07)

What about CI?

OK that's cool and all

Load the dev shell in (GitLab) CI

my-job:
  image: docker.nix-community.org/nixpkgs/nix-flakes:nixos-24.05
  before_script:
    - source <(nix print-dev-env)
  script:
    - ruby --version

Congratulations

Now the CI environment is identical to your local machine

And I didn't even need to build a container?

What about production?

YES

Can I deploy Nix packages to productinon?

Maybe?

Can I SHOULD I deploy Nix packages to productinon?

How can I deploy to production?

Build Nix-based containers
Install to Nix-enabled Linux server
Provision to NixOS

Not enough to get into this today...

Prerequisite: My service needs to be built in the Nix sandbox.

Caveats

Not every language is great to package in Nix
Ruby is one of the not-so-great ones
Well supported languages:
- Through sheer will power: C/C++, Node/npm
- Because of actual good build systems: Rust, Go
Python and Haskell have fairly big Nix communities but I don't have any personal experience with them

What about global system packages?

I don't spend all my day in dev shells.

home-manager

Declaratively manage your HOME environment
This includes the packages installed on your system
But also your dotfiles

Provision packages

{ pkgs, ... }: # Your inputs

# Your home configuration
{
  home.packages = [
    pkgs.jq
    pkgs.ruby
    pkgs.vim
  ];
}

$ home-manager switch

$ which ruby
~/.nix-profile/bin/ruby

$ vim ~/.config/home-manager/home.nix

{
  programs.git = {
    enable = true;
    userName = "Simon Žlender";
    userEmail = "pub.git@zlender.si";
  
    extraConfig = {
      pull.ff = "only";
      push.autoSetupRemote = "true";
    };
  };
}

$ ls -la .config/git/config
.config/git/config -> /nix/store/79yz3g3s37qw3vayhl55kkjm2xhf9hjk-home-manager-files/.config/git/config

$ cat .config/git/config
[pull]
	ff = "only"

[push]
	autoSetupRemote = "true"

[user]
	email = "pub.git@zlender.si"
	name = "Simon Žlender"

Provision dotfiles

{
  programs.fish = {
    enable = true;
    shellAbbrs = {
      sc = "systemctl";
      jc = "journalctl";
    };
  };
  
  programs.starship = {
    enable = true;
  };
  
  programs.direnv = {
    enable = true;
    nix-direnv.enable = true;
  };
}

Provision your shell

{ pkgs, ... }:

{
  home.packages = [ pkgs.gradle ];

  home.file = {
    ".gradle/gradle.properties".text = ''
      org.gradle.console=verbose
      org.gradle.daemon.idletimeout=3600000
    '';
  };
}

What if my app isn't supported?

https://nix-community.github.io/home-manager/options.xhtml

Endless options...

But your entire OS

Like home-manager...

{
  services.plex = {
    enable = true;
  };

  services.nginx.virtualHosts."plex.myhost.com" = {
    forceSSL = true;
    enableACME = true;

    locations."/" = {
      proxyPass = "http://localhost:32400";
      proxyWebsockets = true;
    };
  };
  
  security.acme = {
    acceptTerms = true;
    defaults = {
      email = "admin@myhost.com";
    };
  };

  networking.firewall.allowedTCPPorts = [ 443 ];
}

Provision services

Plex, nginx, letsencrypt and firewall in few lines of nix

{ pkgs, ... }:

{
  systemd.services.my-service = {
    wantedBy = [ "multi-user.target" ];

    serviceConfig = {
      ExecStart = pkgs.my-service;
    };
  };
}

Define custom services

Oh no, I turned my system into a brick 🧱🗿

No worries, NixOS got you covered

But for MacOS

Like NixOS...

🍎

👨

🎩

{ pkgs, ...}:

{
  system.defaults.NSGlobalDomain.InitialKeyRepeat = 10;
  system.defaults.NSGlobalDomain.KeyRepeat = 1;
  
  system.defaults.dock.autohide = true;
  system.defaults.dock.orientation = "left";
  
  system.defaults.finder.AppleShowAllExtensions = true;
  
  environment.systemPackages = [
    pkgs.vim
  ];
}

Provision global system configuration

$ darwin-rebuild switch

Recommendations

✅ Nix dev shells
✅ Nix in CI
❔ Nix in production
✅ Home-manager on your laptop
❌ Home-manager for GUI applications
❔ NixOS on your laptop
✅ NixOS on your home server
✅ Nix-darwin

Where to start?

Use the DeterminateSystems Nix Installer
- https://zero-to-nix.com/start/install
Prefer Nix Flakes over nix-channel
Setup a dev shell for a personal project
Use nix-direnv instead of manually entering the shell
Give home-manager a shot
Go door-to-door and spread the holy word

BONUS ROUND

If we have time...

🏎️💨

Why does my shell script work on Linux, but not on the Mac?

$ bash --version

GNU bash, version 3.2.57(1)-release (arm64-apple-darwin23)

THANKS APPLE

✨

$ nix develop
$ bash --version

GNU bash, version 5.2.26(1)-release (aarch64-apple-darwin23.4.0)

$ sed -i 's/foo/bar/g' my-file

sed: 1: "my-file": invalid command code m

$ sed --version

/usr/bin/sed: illegal option -- -
usage: sed script [-Ealnru] [-i extension] [file ...]
	sed [-Ealnu] [-i extension] [-e script] ... [-f script_file] ... [file ...]

Differences in core system utils: sed

$ nix develop
$ sed -i 's/foo/bar/g' my-file

$ sed --version

sed (GNU sed) 4.9

$ echo 1.2.3 | grep -E "\d\.\d\.\d"
1.2.3

$ grep --version

grep (BSD grep, GNU compatible) 2.6.0-FreeBSD

$ nix develop
$ echo 1.2.3 | grep -E "\d\.\d\.\d"
grep: warning: stray \ before d
grep: warning: stray \ before d
grep: warning: stray \ before d

$ grep --version

grep (GNU grep) 3.11

Differences in core system utils: grep

Lesson learned:

Cross-platforms shell scripting is INSANITY

#!/usr/bin/env nix-shell
#!nix-shell -i bash
#!nix-shell -p coreutils gnused gnugrep

bash --version
coreutils --version
sed --version
grep --version

$ ./bash.sh
GNU bash, version 5.2.26(1)-release (aarch64-apple-darwin23.4.0)
...
coreutils (GNU coreutils) 9.5
...
sed (GNU sed) 4.9
...
grep (GNU grep) 3.11

Nix-shell shebang

💥

The lazy man's dependency management

#!/usr/bin/env nix-shell
#!nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/refs/heads/nixos-24.05.tar.gz
#!nix-shell -i ruby
#!nix-shell -p "ruby.withPackages (p: [ p.nokogiri ])"

require 'nokogiri'
require 'open-uri'

doc = Nokogiri::HTML(URI.open('https://www.rug.si/'))
doc.css('.post-wrap h1 a').each do |link|
  puts link.content
end

Introduction to Nix

About me

It works fine on my machine™

"Probably something weird on your machine"

"Just read my 3 page long outdated setup guide""Try reinstalling Ruby""Did you symlink OpenSSL v1.0.2h to /usr/lib?"

"Bro just build with --with-xml2-dir=/please/help"

Your dev environment is not

We 🤍 Gemfile.lock

So why don't we have lock files for system dependencies?

"Just use Docker"

Sane people use Asdf

"We have system lock files at home"

It's called Nix

So Nix is a package manager?

Package manager, but also...

NO

OK, slow down. So I want to install a package. I should just

`nix-whatever-install ruby` right?

DO NOT USE THE NIX CLI TO MANAGE PACKAGES

Please for your sanity's sake...

But why...

OK so what should I use then?

What is a dev shell?

Let's write a Rails dev shell

Install pkg-config

Rule #2: RTFM (Read the friendly manual)

Congratulations!

You have not only configured rmagick for yourself. But also for your colleagues and future self.

Full devshell for fellow boilerplate enjoyers

Inputs are locked in flake.lock

What about CI?

OK that's cool and all

Load the dev shell in (GitLab) CI

Congratulations

Now the CI environment is identical to your local machine

What about production?

YES

Can I deploy Nix packages to productinon?

Maybe?

Can I SHOULD I deploy Nix packages to productinon?

How can I deploy to production?

Caveats

What about global system packages?

home-manager

Provision packages

Provision dotfiles

Provision your shell

What if my app isn't supported?

Endless options...

But your entire OS

Like home-manager...

Provision services

Define custom services

Oh no, I turned my system into a brick 🧱🗿

But for MacOS

Like NixOS...

Provision global system configuration

Recommendations

Where to start?

BONUS ROUND

If we have time...

Why does my shell script work on Linux, but not on the Mac?

THANKS APPLE

Differences in core system utils: sed

Differences in core system utils: grep

Lesson learned:

Cross-platforms shell scripting is INSANITY

Nix-shell shebang

Nix-shell shebang

Ruby edition

THANK YOU

Introduction to Nix

"Just read my 3 page long outdated setup guide"
"Try reinstalling Ruby"
"Did you symlink OpenSSL v1.0.2h to /usr/lib?"