NixOS/nix-pills
NixOS/nix-pills
1
0
Fork 0
mirror of https://github.com/NixOS/nix-pills synced 2024-09-19 04:00:13 -04:00
Code Activity

Convert code section to fenced, with the appropriate syntax

Browse source
This commit is contained in:
Noam Yorav-Raphael 2024-04-08 23:44:28 +03:00 committed by Jan Tojnar
parent b4eeb0cdfc
commit 1699c9f1db
20 changed files with 1417 additions and 1037 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
pills/01-why-you-should-give-it-try.md
View file

@ -48,8 +48,10 @@ What we have is basically a store of all packages (with different versions occup
In fact, there's no ldconfig cache either. So where does bash find libc?
$ ldd `which bash`
libc.so.6 => /nix/store/94n64qy99ja0vgbkf675nyk39g9b978n-glibc-2.19/lib/libc.so.6 (0x00007f0248cce000)
```console
$ ldd `which bash`
libc.so.6 => /nix/store/94n64qy99ja0vgbkf675nyk39g9b978n-glibc-2.19/lib/libc.so.6 (0x00007f0248cce000)
```
It turns out that when bash was built, it was built against that specific version of glibc in the Nix store, and at runtime it will require exactly that glibc version.

24
pills/02-install-on-your-running.md
View file

@ -22,7 +22,9 @@ Note: In a multi-user installation, such as the one used in NixOS, the store is
Start looking at the output of the install command:
copying Nix to /nix/store..........................
```
copying Nix to /nix/store..........................
```
That's the `/nix/store` we were talking about in the first article. We're copying in the necessary software to bootstrap a Nix system. You can see bash, coreutils, the C compiler toolchain, perl libraries, sqlite and Nix itself with its own tools and libnix.
@ -32,7 +34,9 @@ You may have noticed that `/nix/store` can contain not only directories, but als
Right after copying the store, the installation process initializes a database:
initialising Nix database...
```
initialising Nix database...
```
Yes, Nix also has a database. It's stored under `/nix/var/nix/db`. It is a sqlite database that keeps track of the dependencies between derivations.
@ -94,10 +98,12 @@ We'll talk about `manifest.nix` more in the next article.
More output from the installer:
downloading Nix expressions from `http://releases.nixos.org/nixpkgs/nixpkgs-14.10pre46060.a1a2851/nixexprs.tar.xz'...
unpacking channels...
created 2 symlinks in user environment
modifying /home/nix/.profile...
```
downloading Nix expressions from `http://releases.nixos.org/nixpkgs/nixpkgs-14.10pre46060.a1a2851/nixexprs.tar.xz'...
unpacking channels...
created 2 symlinks in user environment
modifying /home/nix/.profile...
```
Nix expressions are written in the [Nix language](https://nix.dev/tutorials/nix-language) and used to describe packages and how to build them. [Nixpkgs](https://nixos.org/nixpkgs/) is the repository containing all of the expressions: <https://github.com/NixOS/nixpkgs>.
@ -119,8 +125,10 @@ You can, but there's a good reason to keep using `/nix` instead of a different d
You can see for yourself, don't worry if you see multiple bash derivations:
$ ldd /nix/store/*bash*/bin/bash
[...]
```console
$ ldd /nix/store/*bash*/bin/bash
[...]
```
Keeping the store in `/nix` means we can grab the binary cache from nixos.org (just like you grab packages from debian mirrors) otherwise:

130
pills/03-enter-environment.md
View file

@ -10,7 +10,9 @@ In the previous article we created a Nix user, so let's start by switching to it
If that's not the case:
$ source ~/.nix-profile/etc/profile.d/nix.sh
```console
$ source ~/.nix-profile/etc/profile.d/nix.sh
```
To remind you, `~/.nix-profile/etc` points to the `nix-2.1.3` derivation. At this point, we are in our Nix user profile.
@ -20,11 +22,13 @@ Finally something practical! Installation into the Nix environment is an interes
Back to the installation:
$ nix-env -i hello
installing 'hello-2.10'
[...]
building '/nix/store/0vqw0ssmh6y5zj48yg34gc6macr883xk-user-environment.drv'...
created 36 symlinks in user environment
```console
$ nix-env -i hello
installing 'hello-2.10'
[...]
building '/nix/store/0vqw0ssmh6y5zj48yg34gc6macr883xk-user-environment.drv'...
created 36 symlinks in user environment
```
Now you can run `hello`. Things to notice:
@ -38,15 +42,19 @@ Now you can run `hello`. Things to notice:
We can list generations without walking through the `/nix` hierarchy:
$ nix-env --list-generations
1 2014-07-24 09:23:30
2 2014-07-25 08:45:01 (current)
```console
$ nix-env --list-generations
1 2014-07-24 09:23:30
2 2014-07-25 08:45:01 (current)
```
Listing installed derivations:
$ nix-env -q
nix-2.1.3
hello-2.10
```console
$ nix-env -q
nix-2.1.3
hello-2.10
```
So, where did `hello` really get installed? `which hello` is `~/.nix-profile/bin/hello` which points to the store. We can also list the derivation paths with `nix-env -q --out-path`. So that's what those derivation paths are called: the **output** of a build.
@ -56,21 +64,25 @@ At this point you probably want to run `man` to get some documentation. Even if
Let's inspect the [profile](https://nixos.org/manual/nix/stable/package-management/profiles.html) a bit:
$ ls -l ~/.nix-profile/
dr-xr-xr-x 2 nix nix 4096 Jan 1 1970 bin
lrwxrwxrwx 1 nix nix 55 Jan 1 1970 etc -> /nix/store/ig31y9gfpp8pf3szdd7d4sf29zr7igbr-nix-2.1.3/etc
[...]
```console
$ ls -l ~/.nix-profile/
dr-xr-xr-x 2 nix nix 4096 Jan 1 1970 bin
lrwxrwxrwx 1 nix nix 55 Jan 1 1970 etc -> /nix/store/ig31y9gfpp8pf3szdd7d4sf29zr7igbr-nix-2.1.3/etc
[...]
```
Now that's interesting. When only `nix-2.1.3` was installed, `bin` was a symlink to `nix-2.1.3`. Now that we've actually installed some things (`man`, `hello`), it's a real directory, not a symlink.
$ ls -l ~/.nix-profile/bin/
[...]
man -> /nix/store/83cn9ing5sc6644h50dqzzfxcs07r2jn-man-1.6g/bin/man
[...]
nix-env -> /nix/store/ig31y9gfpp8pf3szdd7d4sf29zr7igbr-nix-2.1.3/bin/nix-env
[...]
hello -> /nix/store/58r35bqb4f3lxbnbabq718svq9i2pda3-hello-2.10/bin/hello
[...]
```console
$ ls -l ~/.nix-profile/bin/
[...]
man -> /nix/store/83cn9ing5sc6644h50dqzzfxcs07r2jn-man-1.6g/bin/man
[...]
nix-env -> /nix/store/ig31y9gfpp8pf3szdd7d4sf29zr7igbr-nix-2.1.3/bin/nix-env
[...]
hello -> /nix/store/58r35bqb4f3lxbnbabq718svq9i2pda3-hello-2.10/bin/hello
[...]
```
Okay, that's clearer now. `nix-env` merged the paths from the installed derivations. `which man` points to the Nix profile, rather than the system `man`, because `~/.nix-profile/bin` is at the head of `$PATH`.
@ -78,15 +90,19 @@ Okay, that's clearer now. `nix-env` merged the paths from the installed derivati
The last command installed `man`. We should be at generation 3, unless you changed something in the middle. Let's say we want to rollback to the old generation:
$ nix-env --rollback
switching from generation 3 to 2
```console
$ nix-env --rollback
switching from generation 3 to 2
```
Now `nix-env -q` does not list `man` anymore. `` ls -l `which man` `` should now be your system copy.
Enough with the rollback, let's go back to the most recent generation:
$ nix-env -G 3
switching from generation 2 to 3
```console
$ nix-env -G 3
switching from generation 2 to 3
```
I invite you to read the manpage of `nix-env`. `nix-env` requires an operation to perform, then there are common options for all operations, as well as options specific to each operation.
@ -100,20 +116,24 @@ To query and manipulate the store, there's the `nix-store` command. We can do so
To show the direct runtime dependencies of `hello`:
$ nix-store -q --references `which hello`
/nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27
/nix/store/58r35bqb4f3lxbnbabq718svq9i2pda3-hello-2.10
```console
$ nix-store -q --references `which hello`
/nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27
/nix/store/58r35bqb4f3lxbnbabq718svq9i2pda3-hello-2.10
```
The argument to `nix-store` can be anything as long as it points to the Nix store. It will follow symlinks.
It may not make sense to you right now, but let's print reverse dependencies of `hello`:
$ nix-store -q --referrers `which hello`
/nix/store/58r35bqb4f3lxbnbabq718svq9i2pda3-hello-2.10
/nix/store/fhvy2550cpmjgcjcx5rzz328i0kfv3z3-env-manifest.nix
/nix/store/yzdk0xvr0b8dcwhi2nns6d75k2ha5208-env-manifest.nix
/nix/store/mp987abm20c70pl8p31ljw1r5by4xwfw-user-environment
/nix/store/ppr3qbq7fk2m2pa49i2z3i32cvfhsv7p-user-environment
```console
$ nix-store -q --referrers `which hello`
/nix/store/58r35bqb4f3lxbnbabq718svq9i2pda3-hello-2.10
/nix/store/fhvy2550cpmjgcjcx5rzz328i0kfv3z3-env-manifest.nix
/nix/store/yzdk0xvr0b8dcwhi2nns6d75k2ha5208-env-manifest.nix
/nix/store/mp987abm20c70pl8p31ljw1r5by4xwfw-user-environment
/nix/store/ppr3qbq7fk2m2pa49i2z3i32cvfhsv7p-user-environment
```
Was it what you expected? It turns out that our environments depend upon `hello`. Yes, that means that the environments are in the store, and since they contain symlinks to `hello`, therefore the environment depends upon `hello`.
@ -125,15 +145,19 @@ The `manifest.nix` file contains metadata about the environment, such as which d
The closures of a derivation is a list of all its dependencies, recursively, including absolutely everything necessary to use that derivation.
$ nix-store -qR `which man`
[...]
```console
$ nix-store -qR `which man`
[...]
```
Copying all those derivations to the Nix store of another machine makes you able to run `man` out of the box on that other machine. That's the base of deployment using Nix, and you can already foresee the potential when deploying software in the cloud (hint: `nix-copy-closures` and `nix-store --export`).
A nicer view of the closure:
$ nix-store -q --tree `which man`
[...]
```console
$ nix-store -q --tree `which man`
[...]
```
With the above command, you can find out exactly why a _runtime_ dependency, be it direct or indirect, exists for a given derivation.
@ -145,10 +169,12 @@ There isn't anything like `apt` which solves a SAT problem in order to satisfy d
## Recovering the hard way
$ nix-env -e '*'
uninstalling 'hello-2.10'
uninstalling 'nix-2.1.3'
[...]
```console
$ nix-env -e '*'
uninstalling 'hello-2.10'
uninstalling 'nix-2.1.3'
[...]
```
Oops, that uninstalled all derivations from the environment, including Nix. That means we can't even run `nix-env`, what now?
@ -158,18 +184,24 @@ First, pick one `nix-2.1.3` derivation: `ls /nix/store/*nix-2.1.3`, say `/nix/st
The first option is to rollback:
$ /nix/store/ig31y9gfpp8pf3szdd7d4sf29zr7igbr-nix-2.1.3/bin/nix-env --rollback
```console
$ /nix/store/ig31y9gfpp8pf3szdd7d4sf29zr7igbr-nix-2.1.3/bin/nix-env --rollback
```
The second option is to install Nix, thus creating a new generation:
$ /nix/store/ig31y9gfpp8pf3szdd7d4sf29zr7igbr-nix-2.1.3/bin/nix-env -i /nix/store/ig31y9gfpp8pf3szdd7d4sf29zr7igbr-nix-2.1.3/bin/nix-env
```console
$ /nix/store/ig31y9gfpp8pf3szdd7d4sf29zr7igbr-nix-2.1.3/bin/nix-env -i /nix/store/ig31y9gfpp8pf3szdd7d4sf29zr7igbr-nix-2.1.3/bin/nix-env
```
## Channels
So where are we getting packages from? We said something about this already in the [second article](02-install-on-your-running.md). There's a list of channels from which we get packages, although usually we use a single channel. The tool to manage channels is [nix-channel](https://nixos.org/manual/nix/stable/command-ref/nix-channel.html).
$ nix-channel --list
nixpkgs http://nixos.org/channels/nixpkgs-unstable
```console
$ nix-channel --list
nixpkgs http://nixos.org/channels/nixpkgs-unstable
```
If you're using NixOS, you may not see any output from the above command (if you're using the default), or you may see a channel whose name begins with "nixos-" instead of "nixpkgs".

200
pills/04-basics-of-language.md
View file

@ -26,27 +26,33 @@ Nix 2.0 contains a command named `nix repl` which is a simple command line tool
Launch `nix repl`. First of all, Nix supports basic arithmetic operations: `+`, `-`, `*` and `/`. (To exit `nix repl`, use the command `:q`. Help is available through the `:?` command.)
nix-repl> 1+3
4
```console
nix-repl> 1+3
4
nix-repl> 7-4
3
nix-repl> 7-4
3
nix-repl> 3*2
6
nix-repl> 3*2
6
```
Attempting to perform division in Nix can lead to some surprises.
nix-repl> 6/3
/home/nix/6/3
```console
nix-repl> 6/3
/home/nix/6/3
```
What happened? Recall that Nix is not a general purpose language, it's a domain-specific language for writing packages. Integer division isn't actually that useful when writing package expressions. Nix parsed `6/3` as a relative path to the current directory. To get Nix to perform division instead, leave a space after the `/`. Alternatively, you can use `builtins.div`.
nix-repl> 6/ 3
2
```console
nix-repl> 6/ 3
2
nix-repl> builtins.div 6 3
2
nix-repl> builtins.div 6 3
2
```
Other operators are `||`, `&&` and `!` for booleans, and relational operators such as `!=`, `==`, `<`, `>`, `<=`, `>=`. In Nix, `<`, `>`, `<=` and `>=` are not much used. There are also other operators we will see in the course of this series.
@ -62,10 +68,12 @@ Not all urls or paths can be parsed this way. If a syntax error occurs, it's sti
There's not much to say here, except that dash (`-`) is allowed in identifiers. That's convenient since many packages use dash in their names. In fact:
nix-repl> a-b
error: undefined variable `a-b' at (string):1:1
nix-repl> a - b
error: undefined variable `a' at (string):1:1
```console
nix-repl> a-b
error: undefined variable `a-b' at (string):1:1
nix-repl> a - b
error: undefined variable `a' at (string):1:1
```
As you can see, `a-b` is parsed as identifier, not as a subtraction.
@ -73,22 +81,26 @@ As you can see, `a-b` is parsed as identifier, not as a subtraction.
It's important to understand the syntax for strings. When learning to read Nix expressions, you may find dollars (`$`) ambiguous, but they are very important . Strings are enclosed by double quotes (`"`), or two single quotes (`''`).
nix-repl> "foo"
"foo"
nix-repl> ''foo''
"foo"
```console
nix-repl> "foo"
"foo"
nix-repl> ''foo''
"foo"
```
In other languages like Python you can also use single quotes for strings (e.g. `'foo'`), but not in Nix.
It's possible to [interpolate](https://nixos.org/manual/nix/stable/expressions/language-values.html) whole Nix expressions inside strings with the `${...}` syntax and only that syntax, not `$foo` or `{$foo}` or anything else.
nix-repl> foo = "strval"
nix-repl> "$foo"
"$foo"
nix-repl> "${foo}"
"strval"
nix-repl> "${2+3}"
error: cannot coerce an integer to a string, at (string):1:2
```console
nix-repl> foo = "strval"
nix-repl> "$foo"
"$foo"
nix-repl> "${foo}"
"strval"
nix-repl> "${2+3}"
error: cannot coerce an integer to a string, at (string):1:2
```
Note: ignore the `foo = "strval"` assignment, special syntax in `nix repl`.
@ -96,24 +108,30 @@ As said previously, you cannot mix integers and strings. You need to explicitly
Using the syntax with two single quotes is useful for writing double quotes inside strings without needing to escape them:
nix-repl> ''test " test''
"test " test"
nix-repl> ''${foo}''
"strval"
```console
nix-repl> ''test " test''
"test \" test"
nix-repl> ''${foo}''
"strval"
```
Escaping `${...}` within double quoted strings is done with the backslash. Within two single quotes, it's done with `''`:
nix-repl> "\${foo}"
"${foo}"
nix-repl> ''test ''${foo} test''
"test ${foo} test"
```console
nix-repl> "\${foo}"
"${foo}"
nix-repl> ''test ''${foo} test''
"test ${foo} test"
```
## Lists
Lists are a sequence of expressions delimited by space (_not_ comma):
nix-repl> [ 2 "foo" true (2+3) ]
[ 2 "foo" true 5 ]
```console
nix-repl> [ 2 "foo" true (2+3) ]
[ 2 "foo" true 5 ]
```
Lists, like everything else in Nix, are immutable. Adding or removing elements from a list is possible, but will return a new list.
@ -121,30 +139,38 @@ Lists, like everything else in Nix, are immutable. Adding or removing elements f
An attribute set is an association between string keys and Nix values. Keys can only be strings. When writing attribute sets you can also use unquoted identifiers as keys.
nix-repl> s = { foo = "bar"; a-b = "baz"; "123" = "num"; }
nix-repl> s
{ "123" = "num"; a-b = "baz"; foo = "bar"; }
```console
nix-repl> s = { foo = "bar"; a-b = "baz"; "123" = "num"; }
nix-repl> s
{ "123" = "num"; a-b = "baz"; foo = "bar"; }
```
For those reading Nix expressions from nixpkgs: do not confuse attribute sets with argument sets used in functions.
To access elements in the attribute set:
nix-repl> s.a-b
"baz"
nix-repl> s."123"
"num"
```console
nix-repl> s.a-b
"baz"
nix-repl> s."123"
"num"
```
Yes, you can use strings to address keys which aren't valid identifiers.
Inside an attribute set you cannot normally refer to elements of the same attribute set:
nix-repl> { a = 3; b = a+4; }
error: undefined variable `a' at (string):1:10
```console
nix-repl> { a = 3; b = a+4; }
error: undefined variable `a' at (string):1:10
```
To do so, use [recursive attribute sets](https://nixos.org/manual/nix/stable/expressions/language-constructs.html#recursive-sets):
nix-repl> rec { a = 3; b = a+4; }
{ a = 3; b = 7; }
```console
nix-repl> rec { a = 3; b = a+4; }
{ a = 3; b = 7; }
```
This is very convenient when defining packages, which tend to be recursive attribute sets.
@ -152,10 +178,12 @@ This is very convenient when defining packages, which tend to be recursive attri
These are expressions, not statements.
nix-repl> a = 3
nix-repl> b = 4
nix-repl> if a > b then "yes" else "no"
"no"
```console
nix-repl> a = 3
nix-repl> b = 4
nix-repl> if a > b then "yes" else "no"
"no"
```
You can't have only the `then` branch, you must specify also the `else` branch, because an expression must have a value in all cases.
@ -163,35 +191,47 @@ You can't have only the `then` branch, you must specify also the `else` branch,
This kind of expression is used to define local variables for inner expressions.
nix-repl> let a = "foo"; in a
"foo"
```console
nix-repl> let a = "foo"; in a
"foo"
```
The syntax is: first assign variables, then `in`, then an expression which can use the defined variables. The value of the whole `let` expression will be the value of the expression after the `in`.
nix-repl> let a = 3; b = 4; in a + b
7
```console
nix-repl> let a = 3; b = 4; in a + b
7
```
Let's write two `let` expressions, one inside the other:
nix-repl> let a = 3; in let b = 4; in a + b
7
```console
nix-repl> let a = 3; in let b = 4; in a + b
7
```
With `let` you cannot assign twice to the same variable. However, you can shadow outer variables:
nix-repl> let a = 3; a = 8; in a
error: attribute `a' at (string):1:12 already defined at (string):1:5
nix-repl> let a = 3; in let a = 8; in a
8
```console
nix-repl> let a = 3; a = 8; in a
error: attribute `a' at (string):1:12 already defined at (string):1:5
nix-repl> let a = 3; in let a = 8; in a
8
```
You cannot refer to variables in a `let` expression outside of it:
nix-repl> let a = (let c = 3; in c); in c
error: undefined variable `c' at (string):1:31
```console
nix-repl> let a = (let c = 3; in c); in c
error: undefined variable `c' at (string):1:31
```
You can refer to variables in the `let` expression when assigning variables, like with recursive attribute sets:
nix-repl> let a = 4; b = a + 5; in b
9
```console
nix-repl> let a = 4; b = a + 5; in b
9
```
So beware when you want to refer to a variable from the outer scope, but it's also defined in the current let expression. The same applies to recursive attribute sets.
@ -199,25 +239,31 @@ So beware when you want to refer to a variable from the outer scope, but it's al
This kind of expression is something you rarely see in other languages. You can think of it like a more granular version of `using` from C++, or `from module import *` from Python. You decide per-expression when to include symbols into the scope.
nix-repl> longName = { a = 3; b = 4; }
nix-repl> longName.a + longName.b
7
nix-repl> with longName; a + b
7
```console
nix-repl> longName = { a = 3; b = 4; }
nix-repl> longName.a + longName.b
7
nix-repl> with longName; a + b
7
```
That's it, it takes an attribute set and includes symbols from it in the scope of the inner expression. Of course, only valid identifiers from the keys of the set will be included. If a symbol exists in the outer scope and would also be introduced by the `with`, it will _not_ be shadowed. You can however still refer to the attribute set:
nix-repl> let a = 10; in with longName; a + b
14
nix-repl> let a = 10; in with longName; longName.a + b
7
```console
nix-repl> let a = 10; in with longName; a + b
14
nix-repl> let a = 10; in with longName; longName.a + b
7
```
## Laziness
Nix evaluates expressions only when needed. This is a great feature when working with packages.
nix-repl> let a = builtins.div 4 0; b = 6; in b
6
```console
nix-repl> let a = builtins.div 4 0; b = 6; in b
6
```
Since `a` is not needed, there's no error about division by zero, because the expression is not in need to be evaluated. That's why we can have all the packages defined on demand, yet have access to specific packages very quickly.

170
pills/05-functions-and-imports.md
View file

@ -10,18 +10,22 @@ I remind you how to enter the Nix environment: `source ~/.nix-profile/etc/profil
Functions are anonymous (lambdas), and only have a single parameter. The syntax is extremely simple. Type the parameter name, then "`:`", then the body of the function.
nix-repl> x: x*2
«lambda»
```console
nix-repl> x: x*2
«lambda»
```
So here we defined a function that takes a parameter `x`, and returns `x*2`. The problem is that we cannot use it in any way, because it's unnamed... joke!
We can store functions in variables.
nix-repl> double = x: x*2
nix-repl> double
«lambda»
nix-repl> double 3
6
```console
nix-repl> double = x: x*2
nix-repl> double
«lambda»
nix-repl> double 3
6
```
As usual, please ignore the special syntax for assignments inside `nix repl`. So, we defined a function `x: x*2` that takes one parameter `x`, and returns `x*2`. This function is then assigned to the variable `double`. Finally we did our first function call: `double 3`.
@ -33,37 +37,43 @@ In summary: to call a function, name the variable, then space, then the argument
How do we create a function that accepts more than one parameter? For people not used to functional programming, this may take a while to grasp. Let's do it step by step.
nix-repl> mul = a: (b: a*b)
nix-repl> mul
«lambda»
nix-repl> mul 3
«lambda»
nix-repl> (mul 3) 4
12
```console
nix-repl> mul = a: (b: a*b)
nix-repl> mul
«lambda»
nix-repl> mul 3
«lambda»
nix-repl> (mul 3) 4
12
```
We defined a function that takes the parameter `a`, the body returns another function. This other function takes a parameter `b` and returns `a*b`. Therefore, calling `mul 3` returns this kind of function: `b: 3*b`. In turn, we call the returned function with `4`, and get the expected result.
You don't have to use parentheses at all, Nix has sane priorities when parsing the code:
nix-repl> mul = a: b: a*b
nix-repl> mul
«lambda»
nix-repl> mul 3
«lambda»
nix-repl> mul 3 4
12
nix-repl> mul (6+7) (8+9)
221
```console
nix-repl> mul = a: b: a*b
nix-repl> mul
«lambda»
nix-repl> mul 3
«lambda»
nix-repl> mul 3 4
12
nix-repl> mul (6+7) (8+9)
221
```
Much more readable, you don't even notice that functions only receive one argument. Since the argument is separated by a space, to pass more complex expressions you need parentheses. In other common languages you would write `mul(6+7, 8+9)`.
Given that functions have only one parameter, it is straightforward to use **partial application**:
nix-repl> foo = mul 3
nix-repl> foo 4
12
nix-repl> foo 5
15
```console
nix-repl> foo = mul 3
nix-repl> foo 4
12
nix-repl> foo 5
15
```
We stored the function returned by `mul 3` into a variable foo, then reused it.
@ -71,22 +81,26 @@ We stored the function returned by `mul 3` into a variable foo, then reused it.
Now this is a very cool feature of Nix. It is possible to pattern match over a set in the parameter. We write an alternative version of `mul = a: b: a*b` first by using a set as argument, then using pattern matching.
nix-repl> mul = s: s.a*s.b
nix-repl> mul { a = 3; b = 4; }
12
nix-repl> mul = { a, b }: a*b
nix-repl> mul { a = 3; b = 4; }
12
```console
nix-repl> mul = s: s.a*s.b
nix-repl> mul { a = 3; b = 4; }
12
nix-repl> mul = { a, b }: a*b
nix-repl> mul { a = 3; b = 4; }
12
```
In the first case we defined a function that accepts a single parameter. We then access attributes `a` and `b` from the given set. Note how the parentheses-less syntax for function calls is very elegant in this case, instead of doing `mul({ a=3; b=4; })` in other languages.
In the second case we defined an argument set. It's like defining a set, except without values. We require that the passed set contains the keys `a` and `b`. Then we can use those `a` and `b` in the function body directly.
nix-repl> mul = { a, b }: a*b
nix-repl> mul { a = 3; b = 4; c = 6; }
error: anonymous function at (string):1:2 called with unexpected argument `c', at (string):1:1
nix-repl> mul { a = 3; }
error: anonymous function at (string):1:2 called without required argument `b', at (string):1:1
```console
nix-repl> mul = { a, b }: a*b
nix-repl> mul { a = 3; b = 4; c = 6; }
error: anonymous function at (string):1:2 called with unexpected argument `c', at (string):1:1
nix-repl> mul { a = 3; }
error: anonymous function at (string):1:2 called without required argument `b', at (string):1:1
```
Only a set with exactly the attributes required by the function is accepted, nothing more, nothing less.
@ -94,22 +108,28 @@ Only a set with exactly the attributes required by the function is accepted, not
It is possible to specify **default values** of attributes in the argument set:
nix-repl> mul = { a, b ? 2 }: a*b
nix-repl> mul { a = 3; }
6
nix-repl> mul { a = 3; b = 4; }
12
```console
nix-repl> mul = { a, b ? 2 }: a*b
nix-repl> mul { a = 3; }
6
nix-repl> mul { a = 3; b = 4; }
12
```
Also you can allow passing more attributes (**variadic**) than the expected ones:
nix-repl> mul = { a, b, ... }: a*b
nix-repl> mul { a = 3; b = 4; c = 2; }
```console
nix-repl> mul = { a, b, ... }: a*b
nix-repl> mul { a = 3; b = 4; c = 2; }
```
However, in the function body you cannot access the "c" attribute. The solution is to give a name to the given set with the **@-pattern**:
nix-repl> mul = s@{ a, b, ... }: a*b*s.c
nix-repl> mul { a = 3; b = 4; c = 2; }
24
```console
nix-repl> mul = s@{ a, b, ... }: a*b*s.c
nix-repl> mul { a = 3; b = 4; c = 2; }
24
```
That's it, you give a name to the whole parameter with name@ before the set pattern.
@ -133,43 +153,59 @@ Let's start with the bare metal.
`a.nix`:
3
```nix
3
```
`b.nix`:
4
```nix
4
```
`mul.nix`:
a: b: a*b
```nix
a: b: a*b
```
nix-repl> a = import ./a.nix
nix-repl> b = import ./b.nix
nix-repl> mul = import ./mul.nix
nix-repl> mul a b
12
```console
nix-repl> a = import ./a.nix
nix-repl> b = import ./b.nix
nix-repl> mul = import ./mul.nix
nix-repl> mul a b
12
```
Yes it's really that simple. You import a file, and it gets parsed as an expression. Note that the scope of the imported file does not inherit the scope of the importer.
`test.nix`:
x
```nix
x
```
nix-repl> let x = 5; in import ./test.nix
error: undefined variable `x' at /home/lethal/test.nix:1:1
```console
nix-repl> let x = 5; in import ./test.nix
error: undefined variable `x' at /home/lethal/test.nix:1:1
```
So how do we pass information to the module? Use functions, like we did with `mul.nix`. A more complex example:
`test.nix`:
{ a, b ? 3, trueMsg ? "yes", falseMsg ? "no" }:
if a > b
then builtins.trace trueMsg true
else builtins.trace falseMsg false
```nix
{ a, b ? 3, trueMsg ? "yes", falseMsg ? "no" }:
if a > b
then builtins.trace trueMsg true
else builtins.trace falseMsg false
```
nix-repl> import ./test.nix { a = 5; trueMsg = "ok"; }
trace: ok
true
```console
nix-repl> import ./test.nix { a = 5; trueMsg = "ok"; }
trace: ok
true
```
Explaining:

146
pills/06-our-first-derivation.md
View file

@ -22,14 +22,18 @@ The `derivation` function receives a set as its first argument. This set require
First of all, what's the name of our system as seen by nix?
nix-repl> builtins.currentSystem
"x86_64-linux"
```console
nix-repl> builtins.currentSystem
"x86_64-linux"
```
Let's try to fake the name of the system:
nix-repl> d = derivation { name = "myname"; builder = "mybuilder"; system = "mysystem"; }
nix-repl> d
«derivation /nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv»
```console
nix-repl> d = derivation { name = "myname"; builder = "mybuilder"; system = "mysystem"; }
nix-repl> d
«derivation /nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv»
```
Oh oh, what's that? Did it build the derivation? No it didn't, but it **did create the .drv file**. `nix repl` does not build derivations unless you tell it to do so.
@ -55,7 +59,7 @@ Note: If your version of nix doesn't have `nix derivation show`, use `nix show-d
</div>
```
```console
$ nix derivation show /nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv
{
"/nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv": {
@ -103,28 +107,34 @@ Back to our fake derivation.
Let's build our really fake derivation:
nix-repl> d = derivation { name = "myname"; builder = "mybuilder"; system = "mysystem"; }
nix-repl> :b d
[...]
these derivations will be built:
/nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv
building path(s) `/nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-myname'
error: a `mysystem' is required to build `/nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv', but I am a `x86_64-linux'
```console
nix-repl> d = derivation { name = "myname"; builder = "mybuilder"; system = "mysystem"; }
nix-repl> :b d
[...]
these derivations will be built:
/nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv
building path(s) `/nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-myname'
error: a `mysystem' is required to build `/nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv', but I am a `x86_64-linux'
```
The `:b` is a `nix repl` specific command to build a derivation. You can see more commands with `:?` . So in the output you can see that it takes the `.drv` as information on how to build the derivation. Then it says it's trying to produce our out path. Finally the error we were waiting for: that derivation can't be built on our system.
We're doing the build inside `nix repl`, but what if we don't want to use `nix repl`? You can **realise** a `.drv` with:
$ nix-store -r /nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv
```console
$ nix-store -r /nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv
```
You will get the same output as before.
Let's fix the system attribute:
nix-repl> d = derivation { name = "myname"; builder = "mybuilder"; system = builtins.currentSystem; }
nix-repl> :b d
[...]
build error: invalid file name `mybuilder'
```console
nix-repl> d = derivation { name = "myname"; builder = "mybuilder"; system = builtins.currentSystem; }
nix-repl> :b d
[...]
build error: invalid file name `mybuilder'
```
A step forward: of course, that `mybuilder` executable does not really exist. Stop for a moment.
@ -132,23 +142,29 @@ A step forward: of course, that `mybuilder` executable does not really exist. St
It is useful to start by inspecting the return value from the derivation function. In this case, the returned value is a plain set:
nix-repl> d = derivation { name = "myname"; builder = "mybuilder"; system = "mysystem"; }
nix-repl> builtins.isAttrs d
true
nix-repl> builtins.attrNames d
[ "all" "builder" "drvAttrs" "drvPath" "name" "out" "outPath" "outputName" "system" "type" ]
```console
nix-repl> d = derivation { name = "myname"; builder = "mybuilder"; system = "mysystem"; }
nix-repl> builtins.isAttrs d
true
nix-repl> builtins.attrNames d
[ "all" "builder" "drvAttrs" "drvPath" "name" "out" "outPath" "outputName" "system" "type" ]
```
You can guess what `builtins.isAttrs` does; it returns true if the argument is a set. While `builtins.attrNames` returns a list of keys of the given set. Some kind of reflection, you might say.
Start from drvAttrs:
nix-repl> d.drvAttrs
{ builder = "mybuilder"; name = "myname"; system = "mysystem"; }
```console
nix-repl> d.drvAttrs
{ builder = "mybuilder"; name = "myname"; system = "mysystem"; }
```
That's basically the input we gave to the derivation function. Also the `d.name`, `d.system` and `d.builder` attributes are exactly the ones we gave as input.
nix-repl> (d == d.out)
true
```console
nix-repl> (d == d.out)
true
```
So out is just the derivation itself, it seems weird but the reason is that we only have one output from the derivation. That's also the reason why `d.all` is a singleton. We'll see multiple outputs later.
@ -156,8 +172,10 @@ The `d.drvPath` is the path of the `.drv` file: `/nix/store/z3hhlxbckx4g3n9sw91n
Something interesting is the `type` attribute. It's `"derivation"`. Nix does add a little of magic to sets with type derivation, but not that much. To help you understand, you can create yourself a set with that type, it's a simple set:
nix-repl> { type = "derivation"; }
«derivation ???»
```console
nix-repl> { type = "derivation"; }
«derivation ???»
```
Of course it has no other information, so Nix doesn't know what to say :-) But you get it, the `type = "derivation"` is just a convention for Nix and for us to understand the set is a derivation.
@ -169,53 +187,67 @@ The `outPath` attribute is the build path in the nix store: `/nix/store/40s0qmrf
Just like dependencies in other package managers, how do we refer to other packages? How do we refer to other derivations in terms of files on the disk? We use the `outPath`. The `outPath` describes the location of the files of that derivation. To make it more convenient, Nix is able to do a conversion from a derivation set to a string.
nix-repl> d.outPath
"/nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-myname"
nix-repl> builtins.toString d
"/nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-myname"
```console
nix-repl> d.outPath
"/nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-myname"
nix-repl> builtins.toString d
"/nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-myname"
```
Nix does the "set to string conversion" as long as there is the `outPath` attribute (much like a toString method in other languages):
nix-repl> builtins.toString { outPath = "foo"; }
"foo"
nix-repl> builtins.toString { a = "b"; }
error: cannot coerce a set to a string, at (string):1:1
```console
nix-repl> builtins.toString { outPath = "foo"; }
"foo"
nix-repl> builtins.toString { a = "b"; }
error: cannot coerce a set to a string, at (string):1:1
```
Say we want to use binaries from coreutils (ignore the nixpkgs etc.):
nix-repl> :l <nixpkgs>
Added 3950 variables.
nix-repl> coreutils
«derivation /nix/store/1zcs1y4n27lqs0gw4v038i303pb89rw6-coreutils-8.21.drv»
nix-repl> builtins.toString coreutils
"/nix/store/8w4cbiy7wqvaqsnsnb3zvabq1cp2zhyz-coreutils-8.21"
```console
nix-repl> :l <nixpkgs>
Added 3950 variables.
nix-repl> coreutils
«derivation /nix/store/1zcs1y4n27lqs0gw4v038i303pb89rw6-coreutils-8.21.drv»
nix-repl> builtins.toString coreutils
"/nix/store/8w4cbiy7wqvaqsnsnb3zvabq1cp2zhyz-coreutils-8.21"
```
Apart from the nixpkgs stuff, just think we added to the scope a series of variables. One of them is coreutils. It is the derivation of the coreutils package you all know of from other Linux distributions. It contains basic binaries for GNU/Linux systems (you may have multiple derivations of coreutils in the nix store, no worries):
$ ls /nix/store/*coreutils*/bin
[...]
```console
$ ls /nix/store/*coreutils*/bin
[...]
```
I remind you, inside strings it's possible to interpolate Nix expressions with `${...}`:
nix-repl> "${d}"
"/nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-myname"
nix-repl> "${coreutils}"
"/nix/store/8w4cbiy7wqvaqsnsnb3zvabq1cp2zhyz-coreutils-8.21"
```console
nix-repl> "${d}"
"/nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-myname"
nix-repl> "${coreutils}"
"/nix/store/8w4cbiy7wqvaqsnsnb3zvabq1cp2zhyz-coreutils-8.21"
```
That's very convenient, because then we could refer to e.g. the bin/true binary like this:
nix-repl> "${coreutils}/bin/true"
"/nix/store/8w4cbiy7wqvaqsnsnb3zvabq1cp2zhyz-coreutils-8.21/bin/true"
```console
nix-repl> "${coreutils}/bin/true"
"/nix/store/8w4cbiy7wqvaqsnsnb3zvabq1cp2zhyz-coreutils-8.21/bin/true"
```
## An almost working derivation
In the previous attempt we used a fake builder, `mybuilder` which obviously does not exist. But we can use for example bin/true, which always exits with 0 (success).
nix-repl> :l <nixpkgs>
nix-repl> d = derivation { name = "myname"; builder = "${coreutils}/bin/true"; system = builtins.currentSystem; }
nix-repl> :b d
[...]
builder for `/nix/store/qyfrcd53wmc0v22ymhhd5r6sz5xmdc8a-myname.drv' failed to produce output path `/nix/store/ly2k1vswbfmswr33hw0kf0ccilrpisnk-myname'
```console
nix-repl> :l <nixpkgs>
nix-repl> d = derivation { name = "myname"; builder = "${coreutils}/bin/true"; system = builtins.currentSystem; }
nix-repl> :b d
[...]
builder for `/nix/store/qyfrcd53wmc0v22ymhhd5r6sz5xmdc8a-myname.drv' failed to produce output path `/nix/store/ly2k1vswbfmswr33hw0kf0ccilrpisnk-myname'
```
Another step forward, it executed the builder (bin/true), but the builder did not create the out path of course, it just exited with 0.
@ -223,7 +255,7 @@ Obvious note: every time we change the derivation, a new hash is created.
Let's examine the new `.drv` now that we referred to another derivation:
```
```console
$ nix derivation show /nix/store/qyfrcd53wmc0v22ymhhd5r6sz5xmdc8a-myname.drv
{
"/nix/store/qyfrcd53wmc0v22ymhhd5r6sz5xmdc8a-myname.drv": {

82
pills/07-working-derivation.md
View file

@ -20,8 +20,10 @@ In summary, we want the builder to be bash, and pass it an argument, `builder.sh
First of all, let's write our `builder.sh` in the current directory:
declare -xp
echo foo > $out
```sh
declare -xp
echo foo > $out
```
The command `declare -xp` lists exported variables (`declare` is a builtin bash function). As we covered in the previous pill, Nix computes the output path of the derivation. The resulting `.drv` file contains a list of environment variables passed to the builder. One of these is `$out`.
@ -31,7 +33,7 @@ In addition, we print out the environment variables during the build process. We
Like for coreutils in the previous pill, we get a blessed bash for free from our magic nixpkgs stuff:
```
```console
nix-repl> :l <nixpkgs>
Added 3950 variables.
nix-repl> "${bash}"
@ -40,7 +42,7 @@ nix-repl> "${bash}"
So with the usual trick, we can refer to bin/bash and create our derivation:
```
```console
nix-repl> d = derivation { name = "foo"; builder = "${bash}/bin/bash"; args = [ ./builder.sh ]; system = builtins.currentSystem; }
nix-repl> :b d
[1 built, 0.0 MiB DL]
@ -57,7 +59,7 @@ Note that we used `./builder.sh` and not `"./builder.sh"`. This way, it is parse
We can use `nix-store --read-log` to see the logs our builder produced:
```
```console
$ nix-store --read-log /nix/store/gczb4qrag22harvv693wwnflqy7lx5pb-foo
declare -x HOME="/homeless-shelter"
declare -x NIX_BUILD_CORES="4"
@ -98,7 +100,7 @@ In terms of autotools, `$out` will be the `--prefix` path. Yes, not the make `DE
We added something else to the derivation this time: the args attribute. Let's see how this changed the .drv compared to the previous pill:
```
```console
$ nix derivation show /nix/store/i76pr1cz0za3i9r6xq518bqqvd2raspw-foo.drv
{
"/nix/store/i76pr1cz0za3i9r6xq518bqqvd2raspw-foo.drv": {
@ -138,19 +140,23 @@ Given that `builder.sh` is a plain file, it has no .drv associated with it. The
Start off by writing a simple C program called `simple.c`:
void main() {
puts("Simple!");
}
```c
void main() {
puts("Simple!");
}
```
And its `simple_builder.sh`:
export PATH="$coreutils/bin:$gcc/bin"
mkdir $out
gcc -o $out/simple $src
```sh
export PATH="$coreutils/bin:$gcc/bin"
mkdir $out
gcc -o $out/simple $src
```
Don't worry too much about where those variables come from yet; let's write the derivation and build it:
```
```console
nix-repl> :l <nixpkgs>
nix-repl> simple = derivation { name = "simple"; builder = "${bash}/bin/bash"; args = [ ./simple_builder.sh ]; gcc = gcc; coreutils = coreutils; src = ./simple.c; system = builtins.currentSystem; }
nix-repl> :b simple
@ -179,18 +185,20 @@ We then create `$out` as a directory and place the binary inside it. Note that g
Drop out of nix repl and write a file `simple.nix`:
let
pkgs = import <nixpkgs> { };
in
pkgs.stdenv.mkDerivation {
name = "simple";
builder = "${pkgs.bash}/bin/bash";
args = [ ./simple_builder.sh ];
gcc = pkgs.gcc;
coreutils = pkgs.coreutils;
src = ./simple.c;
system = builtins.currentSystem;
}
```nix
let
pkgs = import <nixpkgs> { };
in
pkgs.stdenv.mkDerivation {
name = "simple";
builder = "${pkgs.bash}/bin/bash";
args = [ ./simple_builder.sh ];
gcc = pkgs.gcc;
coreutils = pkgs.coreutils;
src = ./simple.c;
system = builtins.currentSystem;
}
```
Now you can build it with `nix-build simple.nix`. This will create a symlink `result` in the current directory, pointing to the out path of the derivation.
@ -210,17 +218,19 @@ The value returned by the nixpkgs function is a set; more specifically, it's a s
Below is a revised version of the `simple.nix` file, using the `inherit` keyword:
let
pkgs = import <nixpkgs> { };
in
pkgs.stdenv.mkDerivation {
name = "simple";
builder = "${pkgs.bash}/bin/bash";
args = [ ./simple_builder.sh ];
inherit (pkgs) gcc coreutils;
src = ./simple.c;
system = builtins.currentSystem;
}
```nix
let
pkgs = import <nixpkgs> { };
in
pkgs.stdenv.mkDerivation {
name = "simple";
builder = "${pkgs.bash}/bin/bash";
args = [ ./simple_builder.sh ];
inherit (pkgs) gcc coreutils;
src = ./simple.c;
system = builtins.currentSystem;
}
```
Here we also take the opportunity to introduce the [`inherit` keyword](https://nixos.org/manual/nix/stable/expressions/language-constructs.html#inheriting-attributes). `inherit foo;` is equivalent to `foo = foo;`. Similarly, `inherit gcc coreutils;` is equivalent to `gcc = gcc; coreutils = coreutils;`. Lastly, `inherit (pkgs) gcc coreutils;` is equivalent to `gcc = pkgs.gcc; coreutils = pkgs.coreutils;`.

270
pills/08-generic-builders.md
View file

@ -12,63 +12,69 @@ In the previous pill we packaged a simple .c file, which was being compiled with
Let's create a builder script for GNU hello world, hello_builder.sh:
export PATH="$gnutar/bin:$gcc/bin:$gnumake/bin:$coreutils/bin:$gawk/bin:$gzip/bin:$gnugrep/bin:$gnused/bin:$bintools/bin"
tar -xzf $src
cd hello-2.12.1
./configure --prefix=$out
make
make install
```sh
export PATH="$gnutar/bin:$gcc/bin:$gnumake/bin:$coreutils/bin:$gawk/bin:$gzip/bin:$gnugrep/bin:$gnused/bin:$bintools/bin"
tar -xzf $src
cd hello-2.12.1
./configure --prefix=$out
make
make install
```
And the derivation hello.nix:
let
pkgs = import <nixpkgs> { };
in
derivation {
name = "hello";
builder = "${pkgs.bash}/bin/bash";
args = [ ./hello_builder.sh ];
inherit (pkgs)
gnutar
gzip
gnumake
gcc
coreutils
gawk
gnused
gnugrep
;
bintools = pkgs.binutils.bintools;
src = ./hello-2.12.1.tar.gz;
system = builtins.currentSystem;
}
```nix
let
pkgs = import <nixpkgs> { };
in
derivation {
name = "hello";
builder = "${pkgs.bash}/bin/bash";
args = [ ./hello_builder.sh ];
inherit (pkgs)
gnutar
gzip
gnumake
gcc
coreutils
gawk
gnused
gnugrep
;
bintools = pkgs.binutils.bintools;
src = ./hello-2.12.1.tar.gz;
system = builtins.currentSystem;
}
```
<div class="info">
<h4>Nix on darwin</h4>
Darwin (i.e. macOS) builds typically use `clang` rather than `gcc` for a C compiler. We can adapt this early example for darwin by using this modified version of `hello.nix`:
let
pkgs = import <nixpkgs> { };
in
derivation {
name = "hello";
builder = "${pkgs.bash}/bin/bash";
args = [ ./hello_builder.sh ];
inherit (pkgs)
gnutar
gzip
gnumake
coreutils
gawk
gnused
gnugrep
;
gcc = pkgs.clang;
bintools = pkgs.clang.bintools.bintools_bin;
src = ./hello-2.12.1.tar.gz;
system = builtins.currentSystem;
}
```nix
let
pkgs = import <nixpkgs> { };
in
derivation {
name = "hello";
builder = "${pkgs.bash}/bin/bash";
args = [ ./hello_builder.sh ];
inherit (pkgs)
gnutar
gzip
gnumake
coreutils
gawk
gnused
gnugrep
;
gcc = pkgs.clang;
bintools = pkgs.clang.bintools.bintools_bin;
src = ./hello-2.12.1.tar.gz;
system = builtins.currentSystem;
}
```
Later, we will show how Nix can automatically handle these differences. For now, please be just aware that changes similar to the above may be needed in what follows.
@ -82,24 +88,26 @@ Please note the `--prefix=$out` we were talking about in the [previous pill](07-
Let's create a generic `builder.sh` for autotools projects:
set -e
unset PATH
for p in $buildInputs; do
export PATH=$p/bin${PATH:+:}$PATH
done
```sh
set -e
unset PATH
for p in $buildInputs; do
export PATH=$p/bin${PATH:+:}$PATH
done
tar -xf $src
tar -xf $src
for d in *; do
if [ -d "$d" ]; then
for d in *; do
if [ -d "$d" ]; then
cd "$d"
break
fi
done
fi
done
./configure --prefix=$out
make
make install
./configure --prefix=$out
make
make install
```
What do we do here?
@ -119,45 +127,51 @@ As you can see, there's no reference to "hello" in the builder anymore. It still
Now let's rewrite `hello.nix`:
let
pkgs = import <nixpkgs> { };
in
derivation {
name = "hello";
builder = "${pkgs.bash}/bin/bash";
args = [ ./builder.sh ];
buildInputs = with pkgs; [
gnutar
gzip
gnumake
gcc
coreutils
gawk
gnused
gnugrep
binutils.bintools
];
src = ./hello-2.12.1.tar.gz;
system = builtins.currentSystem;
}
```nix
let
pkgs = import <nixpkgs> { };
in
derivation {
name = "hello";
builder = "${pkgs.bash}/bin/bash";
args = [ ./builder.sh ];
buildInputs = with pkgs; [
gnutar
gzip
gnumake
gcc
coreutils
gawk
gnused
gnugrep
binutils.bintools
];
src = ./hello-2.12.1.tar.gz;
system = builtins.currentSystem;
}
```
All clear, except that buildInputs. However it's easier than any black magic you are thinking of at this moment.
Nix is able to convert a list to a string. It first converts the elements to strings, and then concatenates them separated by a space:
nix-repl> builtins.toString 123
"123"
nix-repl> builtins.toString [ 123 456 ]
"123 456"
```console
nix-repl> builtins.toString 123
"123"
nix-repl> builtins.toString [ 123 456 ]
"123 456"
```
Recall that derivations can be converted to a string, hence:
nix-repl> :l <nixpkgs>
Added 3950 variables.
nix-repl> builtins.toString gnugrep
"/nix/store/g5gdylclfh6d224kqh9sja290pk186xd-gnugrep-2.14"
nix-repl> builtins.toString [ gnugrep gnused ]
"/nix/store/g5gdylclfh6d224kqh9sja290pk186xd-gnugrep-2.14 /nix/store/krgdc4sknzpw8iyk9p20lhqfd52kjmg0-gnused-4.2.2"
```console
nix-repl> :l <nixpkgs>
Added 3950 variables.
nix-repl> builtins.toString gnugrep
"/nix/store/g5gdylclfh6d224kqh9sja290pk186xd-gnugrep-2.14"
nix-repl> builtins.toString [ gnugrep gnused ]
"/nix/store/g5gdylclfh6d224kqh9sja290pk186xd-gnugrep-2.14 /nix/store/krgdc4sknzpw8iyk9p20lhqfd52kjmg0-gnused-4.2.2"
```
Simple! The buildInputs variable is a string with out paths separated by space, perfect for bash usage in a for loop.
@ -169,27 +183,29 @@ A natural approach would be to create a function that accepts an attribute set,
Create `autotools.nix`:
pkgs: attrs:
let
defaultAttrs = {
builder = "${pkgs.bash}/bin/bash";
args = [ ./builder.sh ];
baseInputs = with pkgs; [
gnutar
gzip
gnumake
gcc
coreutils
gawk
gnused
gnugrep
binutils.bintools
];
buildInputs = [ ];
system = builtins.currentSystem;
};
in
derivation (defaultAttrs // attrs)
```nix
pkgs: attrs:
let
defaultAttrs = {
builder = "${pkgs.bash}/bin/bash";
args = [ ./builder.sh ];
baseInputs = with pkgs; [
gnutar
gzip
gnumake
gcc
coreutils
gawk
gnused
gnugrep
binutils.bintools
];
buildInputs = [ ];
system = builtins.currentSystem;
};
in
derivation (defaultAttrs // attrs)
```
Ok now we have to remember a little about [Nix functions](05-functions-and-imports.md). The whole nix expression of this `autotools.nix` file will evaluate to a function. This function accepts a parameter `pkgs`, then returns a function which accepts a parameter `attrs`.
@ -207,23 +223,27 @@ So we use `defaultAttrs` as base set, and add (or override) the attributes from
A couple of examples ought to be enough to clear out the behavior of the operator:
nix-repl> { a = "b"; } // { c = "d"; }
{ a = "b"; c = "d"; }
nix-repl> { a = "b"; } // { a = "c"; }
{ a = "c"; }
```console
nix-repl> { a = "b"; } // { c = "d"; }
{ a = "b"; c = "d"; }
nix-repl> { a = "b"; } // { a = "c"; }
{ a = "c"; }
```
**Exercise:** Complete the new `builder.sh` by adding `$baseInputs` in the `for` loop together with `$buildInputs`. As you noticed, we passed that new variable in the derivation. Instead of merging buildInputs with the base ones, we prefer to preserve buildInputs as seen by the caller, so we keep them separated. Just a matter of choice.
Then we rewrite `hello.nix` as follows:
let
pkgs = import <nixpkgs> { };
mkDerivation = import ./autotools.nix pkgs;
in
mkDerivation {
name = "hello";
src = ./hello-2.12.1.tar.gz;
}
```nix
let
pkgs = import <nixpkgs> { };
mkDerivation = import ./autotools.nix pkgs;
in
mkDerivation {
name = "hello";
src = ./hello-2.12.1.tar.gz;
}
```
Finally! We got a very simple description of a package! Below are a couple of remarks that you may find useful as you're continuing to understand the nix language:

82
pills/09-automatic-runtime.md
View file

@ -8,21 +8,23 @@ Today we stop by the GNU `hello` program to analyze build and runtime dependenci
Let's start analyzing build dependencies for our GNU `hello` package:
$ nix-instantiate hello.nix
/nix/store/z77vn965a59irqnrrjvbspiyl2rph0jp-hello.drv
$ nix-store -q --references /nix/store/z77vn965a59irqnrrjvbspiyl2rph0jp-hello.drv
/nix/store/0q6pfasdma4as22kyaknk4kwx4h58480-hello-2.10.tar.gz
/nix/store/1zcs1y4n27lqs0gw4v038i303pb89rw6-coreutils-8.21.drv
/nix/store/2h4b30hlfw4fhqx10wwi71mpim4wr877-gnused-4.2.2.drv
/nix/store/39bgdjissw9gyi4y5j9wanf4dbjpbl07-gnutar-1.27.1.drv
/nix/store/7qa70nay0if4x291rsjr7h9lfl6pl7b1-builder.sh
/nix/store/g6a0shr58qvx2vi6815acgp9lnfh9yy8-gnugrep-2.14.drv
/nix/store/jdggv3q1sb15140qdx0apvyrps41m4lr-bash-4.2-p45.drv
/nix/store/pglhiyp1zdbmax4cglkpz98nspfgbnwr-gnumake-3.82.drv
/nix/store/q9l257jn9lndbi3r9ksnvf4dr8cwxzk7-gawk-4.1.0.drv
/nix/store/rgyrqxz1ilv90r01zxl0sq5nq0cq7v3v-binutils-2.23.1.drv
/nix/store/qzxhby795niy6wlagfpbja27dgsz43xk-gcc-wrapper-4.8.3.drv
/nix/store/sk590g7fv53m3zp0ycnxsc41snc2kdhp-gzip-1.6.drv
```console
$ nix-instantiate hello.nix
/nix/store/z77vn965a59irqnrrjvbspiyl2rph0jp-hello.drv
$ nix-store -q --references /nix/store/z77vn965a59irqnrrjvbspiyl2rph0jp-hello.drv
/nix/store/0q6pfasdma4as22kyaknk4kwx4h58480-hello-2.10.tar.gz
/nix/store/1zcs1y4n27lqs0gw4v038i303pb89rw6-coreutils-8.21.drv
/nix/store/2h4b30hlfw4fhqx10wwi71mpim4wr877-gnused-4.2.2.drv
/nix/store/39bgdjissw9gyi4y5j9wanf4dbjpbl07-gnutar-1.27.1.drv
/nix/store/7qa70nay0if4x291rsjr7h9lfl6pl7b1-builder.sh
/nix/store/g6a0shr58qvx2vi6815acgp9lnfh9yy8-gnugrep-2.14.drv
/nix/store/jdggv3q1sb15140qdx0apvyrps41m4lr-bash-4.2-p45.drv
/nix/store/pglhiyp1zdbmax4cglkpz98nspfgbnwr-gnumake-3.82.drv
/nix/store/q9l257jn9lndbi3r9ksnvf4dr8cwxzk7-gawk-4.1.0.drv
/nix/store/rgyrqxz1ilv90r01zxl0sq5nq0cq7v3v-binutils-2.23.1.drv
/nix/store/qzxhby795niy6wlagfpbja27dgsz43xk-gcc-wrapper-4.8.3.drv
/nix/store/sk590g7fv53m3zp0ycnxsc41snc2kdhp-gzip-1.6.drv
```
It has precisely the derivations referenced in the `derivation` function; nothing more, nothing less. Of course, we may not use some of them at all. However, given that our generic `mkDerivation` function always pulls such dependencies (think of it like [build-essential](https://packages.debian.org/unstable/build-essential) from Debian), we will already have these packages in the nix store for any future packages that need them.
@ -52,19 +54,23 @@ Nix handles runtime dependencies for us automatically. The technique it uses to
The snippet below shows the dependencies for `hello`.
$ nix-instantiate hello.nix
/nix/store/z77vn965a59irqnrrjvbspiyl2rph0jp-hello.drv
$ nix-store -r /nix/store/z77vn965a59irqnrrjvbspiyl2rph0jp-hello.drv
/nix/store/a42k52zwv6idmf50r9lps1nzwq9khvpf-hello
$ nix-store -q --references /nix/store/a42k52zwv6idmf50r9lps1nzwq9khvpf-hello
/nix/store/94n64qy99ja0vgbkf675nyk39g9b978n-glibc-2.19
/nix/store/8jm0wksask7cpf85miyakihyfch1y21q-gcc-4.8.3
/nix/store/a42k52zwv6idmf50r9lps1nzwq9khvpf-hello
```console
$ nix-instantiate hello.nix
/nix/store/z77vn965a59irqnrrjvbspiyl2rph0jp-hello.drv
$ nix-store -r /nix/store/z77vn965a59irqnrrjvbspiyl2rph0jp-hello.drv
/nix/store/a42k52zwv6idmf50r9lps1nzwq9khvpf-hello
$ nix-store -q --references /nix/store/a42k52zwv6idmf50r9lps1nzwq9khvpf-hello
/nix/store/94n64qy99ja0vgbkf675nyk39g9b978n-glibc-2.19
/nix/store/8jm0wksask7cpf85miyakihyfch1y21q-gcc-4.8.3
/nix/store/a42k52zwv6idmf50r9lps1nzwq9khvpf-hello
```
We see that `glibc` and `gcc` are runtime dependencies. Intuitively, `gcc` shouldn't be in this list! Displaying the printable strings in the `hello` binary shows that the out path of `gcc` does indeed appear:
$ strings result/bin/hello|grep gcc
/nix/store/94n64qy99ja0vgbkf675nyk39g9b978n-glibc-2.19/lib:/nix/store/8jm0wksask7cpf85miyakihyfch1y21q-gcc-4.8.3/lib64
```console
$ strings result/bin/hello|grep gcc
/nix/store/94n64qy99ja0vgbkf675nyk39g9b978n-glibc-2.19/lib:/nix/store/8jm0wksask7cpf85miyakihyfch1y21q-gcc-4.8.3/lib64
```
This is why Nix added `gcc`. But why is that path present in the first place? The answer is that it is the [ld rpath](http://en.wikipedia.org/wiki/Rpath): the list of directories where libraries can be found at runtime. In other distributions, this is usually not abused. But in Nix, we have to refer to particular versions of libraries, and thus the rpath has an important role.
@ -90,7 +96,9 @@ We will add a new phase to our autotools builder. The builder has six phases alr
Now we will add a new phase after the installation phase, which we call the "fixup" phase. At the end of the `builder.sh`, we append:
find $out -type f -exec patchelf --shrink-rpath '{}' \; -exec strip '{}' \; 2>/dev/null
```console
find $out -type f -exec patchelf --shrink-rpath '{}' \; -exec strip '{}' \; 2>/dev/null
```
That is, for each file we run `patchelf --shrink-rpath` and `strip`. Note that we used two new commands here, `find` and `patchelf`. These must be added to our derivation.
@ -98,20 +106,24 @@ That is, for each file we run `patchelf --shrink-rpath` and `strip`. Note that w
Now, we rebuild `hello.nix`...
$ nix-build hello.nix
[...]
$ nix-store -q --references result
/nix/store/94n64qy99ja0vgbkf675nyk39g9b978n-glibc-2.19
/nix/store/md4a3zv0ipqzsybhjb8ndjhhga1dj88x-hello
```console
$ nix-build hello.nix
[...]
$ nix-store -q --references result
/nix/store/94n64qy99ja0vgbkf675nyk39g9b978n-glibc-2.19
/nix/store/md4a3zv0ipqzsybhjb8ndjhhga1dj88x-hello
```
and we see that `glibc` is a runtime dependency. This is exactly what we wanted.
The package is self-contained. This means that we can copy its closure onto another machine and we will be able to run it. Remember, only a very few components under the `/nix/store` are required to [run nix](02-install-on-your-running.md). The `hello` binary will use the exact version of `glibc` library and interpreter referred to in the binary, rather than the system one:
$ ldd result/bin/hello
linux-vdso.so.1 (0x00007fff11294000)
libc.so.6 => /nix/store/94n64qy99ja0vgbkf675nyk39g9b978n-glibc-2.19/lib/libc.so.6 (0x00007f7ab7362000)
/nix/store/94n64qy99ja0vgbkf675nyk39g9b978n-glibc-2.19/lib/ld-linux-x86-64.so.2 (0x00007f7ab770f000)
```console
$ ldd result/bin/hello
linux-vdso.so.1 (0x00007fff11294000)
libc.so.6 => /nix/store/94n64qy99ja0vgbkf675nyk39g9b978n-glibc-2.19/lib/libc.so.6 (0x00007f7ab7362000)
/nix/store/94n64qy99ja0vgbkf675nyk39g9b978n-glibc-2.19/lib/ld-linux-x86-64.so.2 (0x00007f7ab770f000)
```
Of course, the executable will run fine as long as everything is under the `/nix/store` path.

170
pills/10-developing-with-nix-shell.md
View file

@ -14,11 +14,13 @@ Recall that in a nix environment, we don't have access to libraries or programs
We can call `nix-shell` on any Nix expression which returns a derivation, but the resulting `bash` shell's `PATH` does not have the utilities we want:
$ nix-shell hello.nix
[nix-shell]$ make
bash: make: command not found
[nix-shell]$ echo $baseInputs
/nix/store/jff4a6zqi0yrladx3kwy4v6844s3swpc-gnutar-1.27.1 [...]
```console
$ nix-shell hello.nix
[nix-shell]$ make
bash: make: command not found
[nix-shell]$ echo $baseInputs
/nix/store/jff4a6zqi0yrladx3kwy4v6844s3swpc-gnutar-1.27.1 [...]
```
This shell is rather useless. It would be reasonable to expect that the GNU `hello` build inputs are available in `PATH`, including GNU `make`, but this is not the case.
@ -26,8 +28,10 @@ However, we do have the environment variables that we set in the derivation, lik
This means that we can `source` our `builder.sh`, and it will build the derivation. You may get an error in the installation phase, because your user may not have the permission to write to `/nix/store`:
[nix-shell]$ source builder.sh
...
```console
[nix-shell]$ source builder.sh
...
```
The derivation didn't install, but it did build. Note the following:
@ -53,97 +57,107 @@ During our refactoring, we will wrap the build phases in functions to give more
Here is our modified `autotools.nix`. Noteworthy is the `setup = ./setup.sh;` attribute in the derivation, which adds `setup.sh` to the nix store and correspondingly adds a `$setup` environment variable in the builder.
pkgs: attrs:
let
defaultAttrs = {
builder = "${pkgs.bash}/bin/bash";
args = [ ./builder.sh ];
setup = ./setup.sh;
baseInputs = with pkgs; [
gnutar
gzip
gnumake
gcc
coreutils
gawk
gnused
gnugrep
binutils.bintools
patchelf
findutils
];
buildInputs = [ ];
system = builtins.currentSystem;
};
in
derivation (defaultAttrs // attrs)
```nix
pkgs: attrs:
let
defaultAttrs = {
builder = "${pkgs.bash}/bin/bash";
args = [ ./builder.sh ];
setup = ./setup.sh;
baseInputs = with pkgs; [
gnutar
gzip
gnumake
gcc
coreutils
gawk
gnused
gnugrep
binutils.bintools
patchelf
findutils
];
buildInputs = [ ];
system = builtins.currentSystem;
};
in
derivation (defaultAttrs // attrs)
```
Thanks to that, we can split `builder.sh` into `setup.sh` and `builder.sh`. What `builder.sh` does is `source` `$setup` and call the `genericBuild` function. Everything else is just some changes to the bash script.
Here is the modified `builder.sh`:
set -e
source $setup
genericBuild
```sh
set -e
source $setup
genericBuild
```
Here is the newly added `setup.sh`:
unset PATH
for p in $baseInputs $buildInputs; do
export PATH=$p/bin${PATH:+:}$PATH
```sh
unset PATH
for p in $baseInputs $buildInputs; do
export PATH=$p/bin${PATH:+:}$PATH
done
function unpackPhase() {
tar -xzf $src
for d in *; do
if [ -d "$d" ]; then
cd "$d"
break
fi
done
}
function unpackPhase() {
tar -xzf $src
function configurePhase() {
./configure --prefix=$out
}
for d in *; do
if [ -d "$d" ]; then
cd "$d"
break
fi
done
}
function buildPhase() {
make
}
function configurePhase() {
./configure --prefix=$out
}
function installPhase() {
make install
}
function buildPhase() {
make
}
function fixupPhase() {
find $out -type f -exec patchelf --shrink-rpath '{}' \; -exec strip '{}' \; 2>/dev/null
}
function installPhase() {
make install
}
function fixupPhase() {
find $out -type f -exec patchelf --shrink-rpath '{}' \; -exec strip '{}' \; 2>/dev/null
}
function genericBuild() {
unpackPhase
configurePhase
buildPhase
installPhase
fixupPhase
}
function genericBuild() {
unpackPhase
configurePhase
buildPhase
installPhase
fixupPhase
}
```
Finally, here is `hello.nix`:
let
pkgs = import <nixpkgs> { };
mkDerivation = import ./autotools.nix pkgs;
in
mkDerivation {
name = "hello";
src = ./hello-2.12.1.tar.gz;
}
```nix
let
pkgs = import <nixpkgs> { };
mkDerivation = import ./autotools.nix pkgs;
in
mkDerivation {
name = "hello";
src = ./hello-2.12.1.tar.gz;
}
```
Now back to nix-shell:
$ nix-shell hello.nix
[nix-shell]$ source $setup
[nix-shell]$
```console
$ nix-shell hello.nix
[nix-shell]$ source $setup
[nix-shell]$
```
Now, for example, you can run `unpackPhase` which unpacks `$src` and enters the directory. And you can run commands like `./configure`, `make`, and so forth manually, or run phases with their respective functions.

88
pills/11-garbage-collector.md
View file

@ -20,48 +20,56 @@ In summary, Nix maintains a list of GC roots. These roots can then be used to co
Before we begin we first run the [nix garbage collector](https://nixos.org/manual/nix/stable/command-ref/nix-collect-garbage.html) so that we have a clean setup for our experiments:
$ nix-collect-garbage
finding garbage collector roots...
[...]
deleting unused links...
note: currently hard linking saves -0.00 MiB
1169 store paths deleted, 228.43 MiB freed
```console
$ nix-collect-garbage
finding garbage collector roots...
[...]
deleting unused links...
note: currently hard linking saves -0.00 MiB
1169 store paths deleted, 228.43 MiB freed
```
If we run the garbage collector again it won't find anything new to delete, as we expect. After running the garbage collector, the nix store only contains paths with references from the GC roots.
We now install a new program, `bsd-games`, inspect its store path, and examine its GC root. The `nix-store -q --roots` command is used to query the GC roots that refer to a given derivation. In this case, our current user environment refers to `bsd-games`:
$ nix-env -iA nixpkgs.bsdgames
$ readlink -f `which fortune`
/nix/store/b3lxx3d3ggxcggvjw5n0m1ya1gcrmbyn-bsd-games-2.17/bin/fortune
$ nix-store -q --roots `which fortune`
/nix/var/nix/profiles/default-9-link
$ nix-env --list-generations
[...]
9 2014-08-20 12:44:14 (current)
```console
$ nix-env -iA nixpkgs.bsdgames
$ readlink -f `which fortune`
/nix/store/b3lxx3d3ggxcggvjw5n0m1ya1gcrmbyn-bsd-games-2.17/bin/fortune
$ nix-store -q --roots `which fortune`
/nix/var/nix/profiles/default-9-link
$ nix-env --list-generations
[...]
9 2014-08-20 12:44:14 (current)
```
Now we remove it and run the garbage collector, and note that `bsd-games` is still in the nix store:
$ nix-env -e bsd-games
uninstalling `bsd-games-2.17'
$ nix-collect-garbage
[...]
$ ls /nix/store/b3lxx3d3ggxcggvjw5n0m1ya1gcrmbyn-bsd-games-2.17
bin share
```console
$ nix-env -e bsd-games
uninstalling `bsd-games-2.17'
$ nix-collect-garbage
[...]
$ ls /nix/store/b3lxx3d3ggxcggvjw5n0m1ya1gcrmbyn-bsd-games-2.17
bin share
```
The old generation is still in the nix store because it is a GC root. As we will see below, all profiles and their generations are automatically GC roots.
Removing a GC root is simple. In our case, we delete the generation that refers to `bsd-games`, run the garbage collector, and note that `bsd-games` is no longer in the nix store:
$ rm /nix/var/nix/profiles/default-9-link
$ nix-env --list-generations
[...]
8 2014-07-28 10:23:24
10 2014-08-20 12:47:16 (current)
$ nix-collect-garbage
[...]
$ ls /nix/store/b3lxx3d3ggxcggvjw5n0m1ya1gcrmbyn-bsd-games-2.17
ls: cannot access /nix/store/b3lxx3d3ggxcggvjw5n0m1ya1gcrmbyn-bsd-games-2.17: No such file or directory
```console
$ rm /nix/var/nix/profiles/default-9-link
$ nix-env --list-generations
[...]
8 2014-07-28 10:23:24
10 2014-08-20 12:47:16 (current)
$ nix-collect-garbage
[...]
$ ls /nix/store/b3lxx3d3ggxcggvjw5n0m1ya1gcrmbyn-bsd-games-2.17
ls: cannot access /nix/store/b3lxx3d3ggxcggvjw5n0m1ya1gcrmbyn-bsd-games-2.17: No such file or directory
```
Note: `nix-env --list-generations` does not rely on any particular metadata. It is able to list generations based solely on the file names under the profiles directory.
@ -73,11 +81,13 @@ Recall that building the GNU `hello` package with `nix-build` produces a `result
In fact, `nix-build` automatically adds the `result` symlink as a GC root. Note that this is not the built derivation, but the symlink itself. These GC roots are added under `/nix/var/nix/gcroots/auto`.
$ ls -l /nix/var/nix/gcroots/auto/
total 8
drwxr-xr-x 2 nix nix 4096 Aug 20 10:24 ./
drwxr-xr-x 3 nix nix 4096 Jul 24 10:38 ../
lrwxrwxrwx 1 nix nix 16 Jul 31 10:51 xlgz5x2ppa0m72z5qfc78b8wlciwvgiz -> /home/nix/result/
```console
$ ls -l /nix/var/nix/gcroots/auto/
total 8
drwxr-xr-x 2 nix nix 4096 Aug 20 10:24 ./
drwxr-xr-x 3 nix nix 4096 Jul 24 10:38 ../
lrwxrwxrwx 1 nix nix 16 Jul 31 10:51 xlgz5x2ppa0m72z5qfc78b8wlciwvgiz -> /home/nix/result/
```
The name of the GC root symlink is not important to us at this time. What is important is that such a symlink exists and points to `/home/nix/result`. This is called an **indirect GC root**. A GC root is considered indirect if its specification is outside of `/nix/var/nix/gcroots`. In this case, this means that the target of the `result` symlink will not be garbage collected.
@ -109,10 +119,12 @@ Other systems typically "forget" everything about their previous state after an
The four steps are shown below:
$ nix-channel --update
$ nix-env -u --always
$ rm /nix/var/nix/gcroots/auto/*
$ nix-collect-garbage -d
```console
$ nix-channel --update
$ nix-env -u --always
$ rm /nix/var/nix/gcroots/auto/*
$ nix-collect-garbage -d
```
## Conclusion

190
pills/12-inputs-design-pattern.md
View file

@ -26,21 +26,25 @@ We have already packaged GNU `hello`. Next, we will package a graph-drawing prog
First, we download `graphviz` from [gitlab](https://gitlab.com/api/v4/projects/4207231/packages/generic/graphviz-releases/2.49.3/graphviz-2.49.3.tar.gz). The `graphviz.nix` expression is straightforward:
let
pkgs = import <nixpkgs> { };
mkDerivation = import ./autotools.nix pkgs;
in
mkDerivation {
name = "graphviz";
src = ./graphviz-2.49.3.tar.gz;
}
```nix
let
pkgs = import <nixpkgs> { };
mkDerivation = import ./autotools.nix pkgs;
in
mkDerivation {
name = "graphviz";
src = ./graphviz-2.49.3.tar.gz;
}
```
If we build the project with `nix-build graphviz.nix`, we will get runnable binaries under `result/bin`. Notice how we reused the same `autotools.nix` of `hello.nix.`
By default, `graphviz` does not compile with the ability to produce `png` files. Thus, the derivation above will build a binary supporting only the native output formats, as we see below:
$ echo 'graph test { a -- b }'|result/bin/dot -Tpng -o test.png
Format: "png" not recognized. Use one of: canon cmap [...]
```console
$ echo 'graph test { a -- b }'|result/bin/dot -Tpng -o test.png
Format: "png" not recognized. Use one of: canon cmap [...]
```
If we want to produce a `png` file with `graphviz`, we must add it to our derivation. The place to do so is in `autotools.nix`, where we created a `buildInputs` variable that gets concatenated to `baseInputs`. This is the exact reason for this variable: to allow users of `autotools.nix` to add additional inputs from package expressions.
@ -54,14 +58,16 @@ In classic POSIX systems, `pkg-config` just finds the `.pc` files of all install
As an alternative, we can inform `pkg-config` about the location of libraries via the `PKG_CONFIG_PATH` environment variable. We can populate this environment variable using the same trick we used for `PATH`: automatically filling the variables from `buildInputs`. This is the relevant snippet of `setup.sh`:
for p in $baseInputs $buildInputs; do
if [ -d $p/bin ]; then
```sh
for p in $baseInputs $buildInputs; do
if [ -d $p/bin ]; then
export PATH="$p/bin${PATH:+:}$PATH"
fi
if [ -d $p/lib/pkgconfig ]; then
fi
if [ -d $p/lib/pkgconfig ]; then
export PKG_CONFIG_PATH="$p/lib/pkgconfig${PKG_CONFIG_PATH:+:}$PKG_CONFIG_PATH"
fi
done
fi
done
```
Now if we add derivations to `buildInputs`, their `lib/pkgconfig` and `bin` paths are automatically added in `setup.sh`.
@ -69,19 +75,21 @@ Now if we add derivations to `buildInputs`, their `lib/pkgconfig` and `bin` path
Below, we finish the expression for `graphviz` with `gd` support. Note the use of the `with` expression in `buildInputs` to avoid repeating `pkgs`:
let
pkgs = import <nixpkgs> { };
mkDerivation = import ./autotools.nix pkgs;
in
mkDerivation {
name = "graphviz";
src = ./graphviz-2.49.3.tar.gz;
buildInputs = with pkgs; [
pkg-config
(pkgs.lib.getLib gd)
(pkgs.lib.getDev gd)
];
}
```nix
let
pkgs = import <nixpkgs> { };
mkDerivation = import ./autotools.nix pkgs;
in
mkDerivation {
name = "graphviz";
src = ./graphviz-2.49.3.tar.gz;
buildInputs = with pkgs; [
pkg-config
(pkgs.lib.getLib gd)
(pkgs.lib.getDev gd)
];
}
```
We add `pkg-config` to the derivation to make this tool available for the configure script. As `gd` is a package with [split outputs](https://nixos.org/manual/nixpkgs/stable/#sec-multiple-outputs-), we need to add both the library and development outputs.
@ -95,35 +103,43 @@ Using this technique we are able to abstract from the file names. Instead of ref
To begin, create a default.nix in the current directory:
{
hello = import ./hello.nix;
graphviz = import ./graphviz.nix;
}
```nix
{
hello = import ./hello.nix;
graphviz = import ./graphviz.nix;
}
```
This file is ready to use with `nix repl`:
$ nix repl
nix-repl> :l default.nix
Added 2 variables.
nix-repl> hello
«derivation /nix/store/dkib02g54fpdqgpskswgp6m7bd7mgx89-hello.drv»
nix-repl> graphviz
«derivation /nix/store/zqv520v9mk13is0w980c91z7q1vkhhil-graphviz.drv»
```console
$ nix repl
nix-repl> :l default.nix
Added 2 variables.
nix-repl> hello
«derivation /nix/store/dkib02g54fpdqgpskswgp6m7bd7mgx89-hello.drv»
nix-repl> graphviz
«derivation /nix/store/zqv520v9mk13is0w980c91z7q1vkhhil-graphviz.drv»
```
With `nix-build`, we can pass the -A option to access an attribute of the set from the given `.nix` expression:
$ nix-build default.nix -A hello
[...]
$ result/bin/hello
Hello, world!
```console
$ nix-build default.nix -A hello
[...]
$ result/bin/hello
Hello, world!
```
The `default.nix` file is special. When a directory contains a `default.nix` file, it is used as the implicit nix expression of the directory. This, for example, allows us to run `nix-build -A hello` without specifying `default.nix` explicitly.
We can now use `nix-env` to install the package into our user environment:
$ nix-env -f . -iA graphviz
[...]
$ dot -V
```console
$ nix-env -f . -iA graphviz
[...]
$ dot -V
```
Taking a closer look at the above command, we see the following options:
@ -157,20 +173,22 @@ The `./src` directory is also an input, but we wouldn't change the source from t
Our goal is to make package expressions independent of the repository. To achieve this, we use functions to declare inputs for a derivation. For example, with `graphviz.nix`, we make the following changes to make the derivation independent of the repository and customizable:
{ mkDerivation, lib, gdSupport ? true, gd, pkg-config }:
```nix
{ mkDerivation, lib, gdSupport ? true, gd, pkg-config }:
mkDerivation {
name = "graphviz";
src = ./graphviz-2.49.3.tar.gz;
buildInputs =
if gdSupport
then [
pkg-config
(lib.getLib gd)
(lib.getDev gd)
]
else [];
}
mkDerivation {
name = "graphviz";
src = ./graphviz-2.49.3.tar.gz;
buildInputs =
if gdSupport
then [
pkg-config
(lib.getLib gd)
(lib.getDev gd)
]
else [];
}
```
Recall that "`{...}: ...`" is the syntax for defining functions accepting an attribute set as argument; the above snippet just defines a function.
@ -178,31 +196,33 @@ We made `gd` and its dependencies optional. If `gdSupport` is true (which it is
Going back to back to `default.nix`, we modify our expression to utilize the inputs pattern:
let
pkgs = import <nixpkgs> { };
mkDerivation = import ./autotools.nix pkgs;
in
with pkgs;
{
hello = import ./hello.nix { inherit mkDerivation; };
graphviz = import ./graphviz.nix {
inherit
mkDerivation
lib
gd
pkg-config
;
};
graphvizCore = import ./graphviz.nix {
inherit
mkDerivation
lib
gd
pkg-config
;
gdSupport = false;
};
}
```nix
let
pkgs = import <nixpkgs> { };
mkDerivation = import ./autotools.nix pkgs;
in
with pkgs;
{
hello = import ./hello.nix { inherit mkDerivation; };
graphviz = import ./graphviz.nix {
inherit
mkDerivation
lib
gd
pkg-config
;
};
graphvizCore = import ./graphviz.nix {
inherit
mkDerivation
lib
gd
pkg-config
;
gdSupport = false;
};
}
```
We factorized the import of `nixpkgs` and `mkDerivation`, and also added a variant of `graphviz` with `gd` support disabled. The result is that both `hello.nix` (left as an exercise for the reader) and `graphviz.nix` are independent of the repository and customizable by passing specific inputs.

102
pills/13-callpackage-design-pattern.md
View file

@ -10,15 +10,19 @@ In the previous pill, we demonstrated how the `inputs` pattern decouples package
However, as with usual programming languages, there is some duplication of work: we declare parameter names and then we pass arguments, typically with the same name. For example, if we define a package derivation using the `inputs` pattern such as:
{ input1, input2, ... }:
...
```nix
{ input1, input2, ... }:
...
```
we would likely want to bundle that package derivation into a repository via a an attribute set defined as something like:
rec {
lib1 = import package1.nix { inherit input1 input2; };
program2 = import package2.nix { inherit inputX inputY lib1; };
}
```nix
rec {
lib1 = import package1.nix { inherit input1 input2; };
program2 = import package2.nix { inherit inputX inputY lib1; };
}
```
There are two things to note. First, that inputs often have the same name as attributes in the repository itself. Second, that (due to the `rec` keyword), the inputs to a package derivation may be other packages in the repository itself.
@ -26,10 +30,12 @@ Rather than passing the inputs twice, we would prefer to pass those inputs from
To achieve this, we will define a `callPackage` function with the following calling convention:
{
lib1 = callPackage package1.nix { };
program2 = callPackage package2.nix { someoverride = overriddenDerivation; };
}
```nix
{
lib1 = callPackage package1.nix { };
program2 = callPackage package2.nix { someoverride = overriddenDerivation; };
}
```
We want `callPackage` to be a function of two arguments, with the following behavior:
@ -45,9 +51,11 @@ In this section, we will build up the `callPackages` pattern from scratch. To st
Nix provides a builtin function to do this:
nix-repl> add = { a ? 3, b }: a+b
nix-repl> builtins.functionArgs add
{ a = true; b = false; }
```console
nix-repl> add = { a ? 3, b }: a+b
nix-repl> builtins.functionArgs add
{ a = true; b = false; }
```
In addition to returning the argument names, the attribute set returned by `functionArgs` indicates whether or not the argument has a default value. For our purposes, we are only interested in the argument names; we do not care about the default values right now.
@ -61,21 +69,25 @@ To do this, we need two things:
The former is easy: we just have to set our package derivation's inputs to be package names in a repository, such as `nixpkgs`. For the latter, Nix provides another builtin function:
nix-repl> values = { a = 3; b = 5; c = 10; }
nix-repl> builtins.intersectAttrs values (builtins.functionArgs add)
{ a = true; b = false; }
nix-repl> builtins.intersectAttrs (builtins.functionArgs add) values
{ a = 3; b = 5; }
```console
nix-repl> values = { a = 3; b = 5; c = 10; }
nix-repl> builtins.intersectAttrs values (builtins.functionArgs add)
{ a = true; b = false; }
nix-repl> builtins.intersectAttrs (builtins.functionArgs add) values
{ a = 3; b = 5; }
```
The `intersectAttrs` returns an attribute set whose names are the intersection of both arguments' attribute names, with the attribute values taken from the second argument.
This is all we need to do: we have obtained the argument names from a function, and populated these with an existing set of attributes. This is our simple implementation of `callPackage`:
nix-repl> callPackage = set: f: f (builtins.intersectAttrs (builtins.functionArgs f) set)
nix-repl> callPackage values add
8
nix-repl> with values; add { inherit a b; }
8
```console
nix-repl> callPackage = set: f: f (builtins.intersectAttrs (builtins.functionArgs f) set)
nix-repl> callPackage values add
8
nix-repl> with values; add { inherit a b; }
8
```
Let's dissect the above snippet:
@ -95,11 +107,13 @@ We achieved most of what we wanted: to automatically call functions given a set
The last missing piece is allowing users to override some of the parameters. We may not want to always call functions with values taken from the big set. Thus, we add a third parameter which takes a set of overrides:
nix-repl> callPackage = set: f: overrides: f ((builtins.intersectAttrs (builtins.functionArgs f) set) // overrides)
nix-repl> callPackage values add { }
8
nix-repl> callPackage values add { b = 12; }
15
```console
nix-repl> callPackage = set: f: overrides: f ((builtins.intersectAttrs (builtins.functionArgs f) set) // overrides)
nix-repl> callPackage values add { }
8
nix-repl> callPackage values add { b = 12; }
15
```
Apart from the increasing number of parentheses, it should be clear that we simply take a set union between the default arguments and the overriding set.
@ -107,23 +121,25 @@ Apart from the increasing number of parentheses, it should be clear that we simp
Given our `callPackages`, we can simplify the repository expression in `default.nix`:
```nix
let
nixpkgs = import <nixpkgs> { };
allPkgs = nixpkgs // pkgs;
callPackage =
path: overrides:
let
nixpkgs = import <nixpkgs> { };
allPkgs = nixpkgs // pkgs;
callPackage =
path: overrides:
let
f = import path;
in
f ((builtins.intersectAttrs (builtins.functionArgs f) allPkgs) // overrides);
pkgs = with nixpkgs; {
mkDerivation = import ./autotools.nix nixpkgs;
hello = callPackage ./hello.nix { };
graphviz = callPackage ./graphviz.nix { };
graphvizCore = callPackage ./graphviz.nix { gdSupport = false; };
};
f = import path;
in
pkgs
f ((builtins.intersectAttrs (builtins.functionArgs f) allPkgs) // overrides);
pkgs = with nixpkgs; {
mkDerivation = import ./autotools.nix nixpkgs;
hello = callPackage ./hello.nix { };
graphviz = callPackage ./graphviz.nix { };
graphvizCore = callPackage ./graphviz.nix { gdSupport = false; };
};
in
pkgs
```
Let's examine this in detail:

124
pills/14-override-design-pattern.md
View file

@ -12,7 +12,9 @@ In Nix, we mostly talk about **functions** that accept inputs in order to return
For example, let's say we have an initial derivation `drv` and we want to transform it into a `drv` with debugging information and custom patches:
debugVersion (applyPatches [ ./patch1.patch ./patch2.patch ] drv)
```nix
debugVersion (applyPatches [ ./patch1.patch ./patch2.patch ] drv)
```
The final result should be the original derivation with some changes. This is both interesting and very different from other packaging approaches, which is a consequence of using a functional language to describe packages.
@ -24,31 +26,39 @@ In [pill 12](12-inputs-design-pattern.md) we introduced the inputs design patter
In our repository we have a set of attributes that import the expressions of the packages and pass these arguments, getting back a derivation. Let's take for example the graphviz attribute:
graphviz = import ./graphviz.nix { inherit mkDerivation gd fontconfig libjpeg bzip2; };
```nix
graphviz = import ./graphviz.nix { inherit mkDerivation gd fontconfig libjpeg bzip2; };
```
If we wanted to produce a derivation of graphviz with a customized gd version, we would have to repeat most of the above plus specifying an alternative gd:
{
mygraphviz = import ./graphviz.nix {
inherit
mkDerivation
fontconfig
libjpeg
bzip2
;
gd = customgd;
};
}
```nix
{
mygraphviz = import ./graphviz.nix {
inherit
mkDerivation
fontconfig
libjpeg
bzip2
;
gd = customgd;
};
}
```
That's hard to maintain. Using `callPackage` would be easier:
mygraphviz = callPackage ./graphviz.nix { gd = customgd; };
```nix
mygraphviz = callPackage ./graphviz.nix { gd = customgd; };
```
But we may still be diverging from the original graphviz in the repository.
We would like to avoid specifying the nix expression again. Instead, we would like to reuse the original graphviz attribute in the repository and add our overrides like so:
mygraphviz = graphviz.override { gd = customgd; };
```nix
mygraphviz = graphviz.override { gd = customgd; };
```
The difference is obvious, as well as the advantages of this approach.
@ -62,14 +72,16 @@ Let's start by first creating a function "`makeOverridable`". This function will
We will put this function in a `lib.nix`:
{
makeOverridable =
f: origArgs:
let
origRes = f origArgs;
in
origRes // { override = newArgs: f (origArgs // newArgs); };
}
```nix
{
makeOverridable =
f: origArgs:
let
origRes = f origArgs;
in
origRes // { override = newArgs: f (origArgs // newArgs); };
}
```
`makeOverridable` takes a function and a set of original arguments. It returns the original returned set, plus a new `override` attribute.
@ -77,17 +89,19 @@ This `override` attribute is a function taking a set of new arguments, and retur
Let's try it with `nix repl`:
$ nix repl
nix-repl> :l lib.nix
Added 1 variables.
nix-repl> f = { a, b }: { result = a+b; }
nix-repl> f { a = 3; b = 5; }
{ result = 8; }
nix-repl> res = makeOverridable f { a = 3; b = 5; }
nix-repl> res
{ override = «lambda»; result = 8; }
nix-repl> res.override { a = 10; }
{ result = 15; }
```console
$ nix repl
nix-repl> :l lib.nix
Added 1 variables.
nix-repl> f = { a, b }: { result = a+b; }
nix-repl> f { a = 3; b = 5; }
{ result = 8; }
nix-repl> res = makeOverridable f { a = 3; b = 5; }
nix-repl> res
{ override = «lambda»; result = 8; }
nix-repl> res.override { a = 10; }
{ result = 15; }
```
Note that, as we specified above, the function `f` does not return the plain sum. Instead, it returns a set with the sum bound to the name `result`.
@ -99,28 +113,32 @@ This is a good start, but we can't override again! This is because the returned
The solution is simple: the `.override` function should make the result overridable again:
rec {
makeOverridable =
f: origArgs:
let
origRes = f origArgs;
in
origRes // { override = newArgs: makeOverridable f (origArgs // newArgs); };
}
```nix
rec {
makeOverridable =
f: origArgs:
let
origRes = f origArgs;
in
origRes // { override = newArgs: makeOverridable f (origArgs // newArgs); };
}
```
Please note the `rec` keyword. It's necessary so that we can refer to `makeOverridable` from `makeOverridable` itself.
Now let's try overriding twice:
nix-repl> :l lib.nix
Added 1 variables.
nix-repl> f = { a, b }: { result = a+b; }
nix-repl> res = makeOverridable f { a = 3; b = 5; }
nix-repl> res2 = res.override { a = 10; }
nix-repl> res2
{ override = «lambda»; result = 15; }
nix-repl> res2.override { b = 20; }
{ override = «lambda»; result = 30; }
```console
nix-repl> :l lib.nix
Added 1 variables.
nix-repl> f = { a, b }: { result = a+b; }
nix-repl> res = makeOverridable f { a = 3; b = 5; }
nix-repl> res2 = res.override { a = 10; }
nix-repl> res2
{ override = «lambda»; result = 15; }
nix-repl> res2.override { b = 20; }
{ override = «lambda»; result = 30; }
```
Success! The result is 30 (as expected) because `a` is overridden to 10 in the first override, and `b` is overridden to 20 in the second.
@ -132,7 +150,9 @@ The "`override`" pattern simplifies the way we customize packages starting from
We can dream of a custom, isolated `nix-shell` environment for testing graphviz with a custom gd:
debugVersion (graphviz.override { gd = customgd; })
```nix
debugVersion (graphviz.override { gd = customgd; })
```
Once a new version of the overridden package comes out in the repository, the customized package will make use of it automatically.

70
pills/15-nix-search-paths.md
View file

@ -22,21 +22,25 @@ For ease we will use `nix-instantiate --eval` to do our tests. I remind you, [ni
It's useless from a nix view point, but I think it's useful for your own understanding. Let's use `PATH` itself as `NIX_PATH`, and try to locate `ping` (or another binary if you don't have it).
$ nix-instantiate --eval -E '<ping>'
error: file `ping' was not found in the Nix search path (add it using $NIX_PATH or -I)
$ NIX_PATH=$PATH nix-instantiate --eval -E '<ping>'
/bin/ping
$ nix-instantiate -I /bin --eval -E '<ping>'
/bin/ping
```console
$ nix-instantiate --eval -E '<ping>'
error: file `ping' was not found in the Nix search path (add it using $NIX_PATH or -I)
$ NIX_PATH=$PATH nix-instantiate --eval -E '<ping>'
/bin/ping
$ nix-instantiate -I /bin --eval -E '<ping>'
/bin/ping
```
Great. At first attempt nix obviously said could not be found anywhere in the search path. Note that the -I option accepts a single directory. Paths added with -I take precedence over `NIX_PATH`.
The `NIX_PATH` also accepts a different yet very handy syntax: "`somename=somepath`". That is, instead of searching inside a directory for a name, we specify exactly the value of that name.
$ NIX_PATH="ping=/bin/ping" nix-instantiate --eval -E '<ping>'
/bin/ping
$ NIX_PATH="ping=/bin/foo" nix-instantiate --eval -E '<ping>'
error: file `ping' was not found in the Nix search path (add it using $N
```console
$ NIX_PATH="ping=/bin/ping" nix-instantiate --eval -E '<ping>'
/bin/ping
$ NIX_PATH="ping=/bin/foo" nix-instantiate --eval -E '<ping>'
error: file `ping' was not found in the Nix search path (add it using $N
```
Note in the second case how Nix checks whether the path exists or not.
@ -44,10 +48,12 @@ Note in the second case how Nix checks whether the path exists or not.
You are out of curiosity, right?
$ nix-instantiate --eval -E '<nixpkgs>'
/home/nix/.nix-defexpr/channels/nixpkgs
$ echo $NIX_PATH
nixpkgs=/home/nix/.nix-defexpr/channels/nixpkgs
```console
$ nix-instantiate --eval -E '<nixpkgs>'
/home/nix/.nix-defexpr/channels/nixpkgs
$ echo $NIX_PATH
nixpkgs=/home/nix/.nix-defexpr/channels/nixpkgs
```
You may have a different path, depending on how you added channels etc.. Anyway that's the whole point. The `<nixpkgs>` stranger that we used in our nix expressions, is referring to a path in the filesystem specified by `NIX_PATH`.
@ -59,9 +65,11 @@ You may wonder: then I can also specify a different [nixpkgs](https://github.com
Let's define a path for our repository, then! Let's say all the `default.nix`, `graphviz.nix` etc. are under `/home/nix/mypkgs`:
$ export NIX_PATH=mypkgs=/home/nix/mypkgs:$NIX_PATH
$ nix-instantiate --eval '<mypkgs>'
{ graphviz = <code>; graphvizCore = <code>; hello = <code>; mkDerivation = <code>; }
```console
$ export NIX_PATH=mypkgs=/home/nix/mypkgs:$NIX_PATH
$ nix-instantiate --eval '<mypkgs>'
{ graphviz = <code>; graphvizCore = <code>; hello = <code>; mkDerivation = <code>; }
```
Yes, `nix-build` also accepts paths with angular brackets. We first evaluate the whole repository (`default.nix`) and then pick the graphviz attribute.
@ -77,23 +85,29 @@ So if you run `nix-env -i graphviz` inside your repository, it will install the
In order to specify an alternative to `~/.nix-defexpr` it's possible to use the -f option:
$ nix-env -f '<mypkgs>' -i graphviz
warning: there are multiple derivations named `graphviz'; using the first one
replacing old `graphviz'
installing `graphviz'
```console
$ nix-env -f '<mypkgs>' -i graphviz
warning: there are multiple derivations named `graphviz'; using the first one
replacing old `graphviz'
installing `graphviz'
```
Oh why did it say there's another derivation named graphviz? Because both `graphviz` and `graphvizCore` attributes in our repository have the name "graphviz" for the derivation:
$ nix-env -f '<mypkgs>' -qaP
graphviz graphviz
graphvizCore graphviz
hello hello
```console
$ nix-env -f '<mypkgs>' -qaP
graphviz graphviz
graphvizCore graphviz
hello hello
```
By default `nix-env` parses all derivations and uses the derivation names to interpret the command line. So in this case "graphviz" matched two derivations. Alternatively, like for `nix-build`, one can use -A to specify an attribute name instead of a derivation name:
$ nix-env -f '<mypkgs>' -i -A graphviz
replacing old `graphviz'
installing `graphviz'
```console
$ nix-env -f '<mypkgs>' -i -A graphviz
replacing old `graphviz'
installing `graphviz'
```
This form, other than being more precise, it's also faster because `nix-env` does not have to parse all the derivations.

34
pills/16-nixpkgs-parameters.md
View file

@ -32,14 +32,18 @@ You will find this parameter in many other .nix expressions (e.g. release expres
`myrelease.nix`:
{ system ? builtins.currentSystem }:
```nix
{ system ? builtins.currentSystem }:
let pkgs = import <nixpkgs> { inherit system; };
...
let pkgs = import <nixpkgs> { inherit system; };
...
```
Why is it useful? With this parameter it's very easy to select a set of packages for a particular system. For example:
nix-build -A psmisc --argstr system i686-linux
```console
nix-build -A psmisc --argstr system i686-linux
```
This will build the psmisc derivation for i686-linux instead of x86_64-linux. This concept is very similar to multi-arch of Debian.
@ -55,13 +59,15 @@ After determining `config.nix`, it will be imported as a nix expression, and tha
The `config` is available in the resulting repository:
$ nix repl
nix-repl> pkgs = import <nixpkgs> {}
nix-repl> pkgs.config
{ }
nix-repl> pkgs = import <nixpkgs> { config = { foo = "bar"; }; }
nix-repl> pkgs.config
{ foo = "bar"; }
```console
$ nix repl
nix-repl> pkgs = import <nixpkgs> {}
nix-repl> pkgs.config
{ }
nix-repl> pkgs = import <nixpkgs> { config = { foo = "bar"; }; }
nix-repl> pkgs.config
{ foo = "bar"; }
```
What attributes go in `config` is a matter of convenience and conventions.
@ -79,9 +85,11 @@ In this case, nix does a trick:
For example you can nix-build the `.nix` file below:
{ pkgs ? import <nixpkgs> {} }:
```nix
{ pkgs ? import <nixpkgs> {} }:
pkgs.psmisc
pkgs.psmisc
```
Nix is able to call the function because the pkgs parameter has a default value. This allows you to pass a different value for pkgs using the `--arg` option.

84
pills/17-nixpkgs-overriding-packages.md
View file

@ -12,10 +12,12 @@ We put the override function in the returned attribute set of the original funct
Take for example graphviz. It has an input parameter xorg. If it's null, then graphviz will build without X support.
$ nix repl
nix-repl> :l <nixpkgs>
Added 4360 variables.
nix-repl> :b graphviz.override { withXorg = false; }
```console
$ nix repl
nix-repl> :l <nixpkgs>
Added 4360 variables.
nix-repl> :b graphviz.override { withXorg = false; }
```
This will build graphviz without X support, it's as simple as that.
@ -25,9 +27,11 @@ However, let's say a package `P` depends on graphviz, how do we make `P` depend
...you could do something like this:
pkgs = import <nixpkgs> {};
pkgs.graphviz = pkgs.graphviz.override { withXorg = false; };
build(pkgs.P)
```nix
pkgs = import <nixpkgs> {};
pkgs.graphviz = pkgs.graphviz.override { withXorg = false; };
build(pkgs.P)
```
Given `pkgs.P` depends on `pkgs.graphviz`, it's easy to build `P` with the replaced graphviz. In a pure functional language it's not that easy because you can assign to variables only once.
@ -37,24 +41,28 @@ The fixed point with lazy evaluation is crippling but about necessary in a langu
Follows the definition of fixed point in [nixpkgs](https://github.com/NixOS/nixpkgs/blob/f224a4f1b32b3e813783d22de54e231cd8ea2448/lib/fixed-points.nix#L19):
{
# Take a function and evaluate it with its own returned value.
fix =
f:
let
result = f result;
in
result;
}
```nix
{
# Take a function and evaluate it with its own returned value.
fix =
f:
let
result = f result;
in
result;
}
```
It's a function that accepts a function `f`, calls `f result` on the result just returned by `f result` and returns it. In other words it's `f(f(f(....`
At first sight, it's an infinite loop. With lazy evaluation it isn't, because the call is done only when needed.
nix-repl> fix = f: let result = f result; in result
nix-repl> pkgs = self: { a = 3; b = 4; c = self.a+self.b; }
nix-repl> fix pkgs
{ a = 3; b = 4; c = 7; }
```console
nix-repl> fix = f: let result = f result; in result
nix-repl> pkgs = self: { a = 3; b = 4; c = self.a+self.b; }
nix-repl> fix pkgs
{ a = 3; b = 4; c = 7; }
```
Without the `rec` keyword, we were able to refer to `a` and `b` of the same set.
@ -72,11 +80,13 @@ Won't go further with the explanation here. A good post about fixed point and Ni
Given that `self.a` and `self.b` refer to the passed set and not to the literal set in the function, we're able to override both `a` and `b` and get a new value for `c`:
nix-repl> overrides = { a = 1; b = 2; }
nix-repl> let newpkgs = pkgs (newpkgs // overrides); in newpkgs
{ a = 3; b = 4; c = 3; }
nix-repl> let newpkgs = pkgs (newpkgs // overrides); in newpkgs // overrides
{ a = 1; b = 2; c = 3; }
```console
nix-repl> overrides = { a = 1; b = 2; }
nix-repl> let newpkgs = pkgs (newpkgs // overrides); in newpkgs
{ a = 3; b = 4; c = 3; }
nix-repl> let newpkgs = pkgs (newpkgs // overrides); in newpkgs // overrides
{ a = 1; b = 2; c = 3; }
```
In the first case we computed pkgs with the overrides, in the second case we also included the overridden attributes in the result.
@ -88,19 +98,23 @@ To do this, `nixpkgs` offers `config.packageOverrides`. So `nixpkgs` returns a f
Create a `config.nix` file like this somewhere:
{
packageOverrides = pkgs: {
graphviz = pkgs.graphviz.override {
# disable xorg support
withXorg = false;
};
};
}
```nix
{
packageOverrides = pkgs: {
graphviz = pkgs.graphviz.override {
# disable xorg support
withXorg = false;
};
};
}
```
Now we can build e.g. asciidoc-full and it will automatically use the overridden graphviz:
nix-repl> pkgs = import <nixpkgs> { config = import ./config.nix; }
nix-repl> :b pkgs.asciidoc-full
```console
nix-repl> pkgs = import <nixpkgs> { config = import ./config.nix; }
nix-repl> :b pkgs.asciidoc-full
```
Note how we pass the `config` with `packageOverrides` when importing `nixpkgs`. Then `pkgs.asciidoc-full` is a derivation that has graphviz input (`pkgs.asciidoc` is the lighter version and doesn't use graphviz at all).

84
pills/18-nix-store-paths.md
View file

@ -10,17 +10,21 @@ The way store paths are computed is a little contrived, mostly due to historical
Let's start simple. You know nix allows relative paths to be used, such that the file or directory is stored in the nix store, that is `./myfile` gets stored into `/nix/store/.......` We want to understand how is the store path generated for such a file:
$ echo mycontent > myfile
```console
$ echo mycontent > myfile
```
I remind you, the simplest derivation you can write has a `name`, a `builder` and the `system`:
$ nix repl
nix-repl> derivation { system = "x86_64-linux"; builder = ./myfile; name = "foo"; }
«derivation /nix/store/y4h73bmrc9ii5bxg6i7ck6hsf5gqv8ck-foo.drv»
```console
$ nix repl
nix-repl> derivation { system = "x86_64-linux"; builder = ./myfile; name = "foo"; }
«derivation /nix/store/y4h73bmrc9ii5bxg6i7ck6hsf5gqv8ck-foo.drv»
```
Now inspect the .drv to see where is `./myfile` being stored:
```
```console
$ nix derivation show /nix/store/y4h73bmrc9ii5bxg6i7ck6hsf5gqv8ck-foo.drv
{
"/nix/store/y4h73bmrc9ii5bxg6i7ck6hsf5gqv8ck-foo.drv": {
@ -54,13 +58,17 @@ Great, how did nix decide to use `xv2iccirbrvklck36f1g7vldn5v58vck` ? Keep looki
The comments tell us to first compute the sha256 of the NAR serialization of the file. Can be done in two ways:
$ nix-hash --type sha256 myfile
2bfef67de873c54551d884fdab3055d84d573e654efa79db3c0d7b98883f9ee3
```console
$ nix-hash --type sha256 myfile
2bfef67de873c54551d884fdab3055d84d573e654efa79db3c0d7b98883f9ee3
```
Or:
$ nix-store --dump myfile|sha256sum
2bfef67de873c54551d884fdab3055d84d573e654efa79db3c0d7b98883f9ee3
```console
$ nix-store --dump myfile|sha256sum
2bfef67de873c54551d884fdab3055d84d573e654efa79db3c0d7b98883f9ee3
```
In general, Nix understands two contents: flat for regular files, or recursive for NAR serializations which can be anything.
@ -68,14 +76,18 @@ In general, Nix understands two contents: flat for regular files, or recursive f
Then nix uses a special string which includes the hash, the path type and the file name. We store this in another file:
$ echo -n "source:sha256:2bfef67de873c54551d884fdab3055d84d573e654efa79db3c0d7b98883f9ee3:/nix/store:myfile" > myfile.str
```console
$ echo -n "source:sha256:2bfef67de873c54551d884fdab3055d84d573e654efa79db3c0d7b98883f9ee3:/nix/store:myfile" > myfile.str
```
### Step 3, compute the final hash
Finally the comments tell us to compute the base-32 representation of the first 160 bits (truncation) of a sha256 of the above string:
$ nix-hash --type sha256 --truncate --base32 --flat myfile.str
xv2iccirbrvklck36f1g7vldn5v58vck
```console
$ nix-hash --type sha256 --truncate --base32 --flat myfile.str
xv2iccirbrvklck36f1g7vldn5v58vck
```
## Output paths
@ -85,16 +97,20 @@ It's computed in a similar way to source paths, except that the .drv is hashed a
At the time nix computes the out path, the .drv contains an empty string for each out path. So what we do is getting our .drv and replacing the out path with an empty string:
$ cp -f /nix/store/y4h73bmrc9ii5bxg6i7ck6hsf5gqv8ck-foo.drv myout.drv
$ sed -i 's,/nix/store/hs0yi5n5nw6micqhy8l1igkbhqdkzqa1-foo,,g' myout.drv
```console
$ cp -f /nix/store/y4h73bmrc9ii5bxg6i7ck6hsf5gqv8ck-foo.drv myout.drv
$ sed -i 's,/nix/store/hs0yi5n5nw6micqhy8l1igkbhqdkzqa1-foo,,g' myout.drv
```
The `myout.drv` is the .drv state in which nix is when computing the out path for our derivation:
$ sha256sum myout.drv
1bdc41b9649a0d59f270a92d69ce6b5af0bc82b46cb9d9441ebc6620665f40b5 myout.drv
$ echo -n "output:out:sha256:1bdc41b9649a0d59f270a92d69ce6b5af0bc82b46cb9d9441ebc6620665f40b5:/nix/store:foo" > myout.str
$ nix-hash --type sha256 --truncate --base32 --flat myout.str
hs0yi5n5nw6micqhy8l1igkbhqdkzqa1
```console
$ sha256sum myout.drv
1bdc41b9649a0d59f270a92d69ce6b5af0bc82b46cb9d9441ebc6620665f40b5 myout.drv
$ echo -n "output:out:sha256:1bdc41b9649a0d59f270a92d69ce6b5af0bc82b46cb9d9441ebc6620665f40b5:/nix/store:foo" > myout.str
$ nix-hash --type sha256 --truncate --base32 --flat myout.str
hs0yi5n5nw6micqhy8l1igkbhqdkzqa1
```
Then nix puts that out path in the .drv, and that's it.
@ -112,15 +128,17 @@ The builder must create the out path and make sure its hash is the same as the o
Let's say our builder should create a file whose contents is `mycontent`:
$ echo mycontent > myfile
$ sha256sum myfile
f3f3c4763037e059b4d834eaf68595bbc02ba19f6d2a500dce06d124e2cd99bb myfile
nix-repl> derivation { name = "bar"; system = "x86_64-linux"; builder = "none"; outputHashMode = "flat"; outputHashAlgo = "sha256"; outputHash = "f3f3c4763037e059b4d834eaf68595bbc02ba19f6d2a500dce06d124e2cd99bb"; }
«derivation /nix/store/ymsf5zcqr9wlkkqdjwhqllgwa97rff5i-bar.drv»
```console
$ echo mycontent > myfile
$ sha256sum myfile
f3f3c4763037e059b4d834eaf68595bbc02ba19f6d2a500dce06d124e2cd99bb myfile
nix-repl> derivation { name = "bar"; system = "x86_64-linux"; builder = "none"; outputHashMode = "flat"; outputHashAlgo = "sha256"; outputHash = "f3f3c4763037e059b4d834eaf68595bbc02ba19f6d2a500dce06d124e2cd99bb"; }
«derivation /nix/store/ymsf5zcqr9wlkkqdjwhqllgwa97rff5i-bar.drv»
```
Inspect the .drv and see that it also stored the fact that it's a fixed-output derivation with sha256 algorithm, compared to the previous examples:
```
```console
$ nix derivation show /nix/store/ymsf5zcqr9wlkkqdjwhqllgwa97rff5i-bar.drv
{
"/nix/store/ymsf5zcqr9wlkkqdjwhqllgwa97rff5i-bar.drv": {
@ -139,15 +157,19 @@ It doesn't matter which input derivations are being used, the final out path mus
What nix does is to create an intermediate string representation of the fixed-output content:
$ echo -n "fixed:out:sha256:f3f3c4763037e059b4d834eaf68595bbc02ba19f6d2a500dce06d124e2cd99bb:" > mycontent.str
$ sha256sum mycontent.str
423e6fdef56d53251c5939359c375bf21ea07aaa8d89ca5798fb374dbcfd7639 myfile.str
```console
$ echo -n "fixed:out:sha256:f3f3c4763037e059b4d834eaf68595bbc02ba19f6d2a500dce06d124e2cd99bb:" > mycontent.str
$ sha256sum mycontent.str
423e6fdef56d53251c5939359c375bf21ea07aaa8d89ca5798fb374dbcfd7639 myfile.str
```
Then proceed as it was a normal derivation output path:
$ echo -n "output:out:sha256:423e6fdef56d53251c5939359c375bf21ea07aaa8d89ca5798fb374dbcfd7639:/nix/store:bar" > myfile.str
$ nix-hash --type sha256 --truncate --base32 --flat myfile.str
a00d5f71k0vp5a6klkls0mvr1f7sx6ch
```console
$ echo -n "output:out:sha256:423e6fdef56d53251c5939359c375bf21ea07aaa8d89ca5798fb374dbcfd7639:/nix/store:bar" > myfile.str
$ nix-hash --type sha256 --truncate --base32 --flat myfile.str
a00d5f71k0vp5a6klkls0mvr1f7sx6ch
```
Hence, the store path only depends on the declared fixed-output hash.

116
pills/19-fundamentals-of-stdenv.md
View file

@ -10,32 +10,38 @@ The `stdenv` is not treated as a special derivation by Nix, but it's very import
First of all, `stdenv` is a derivation, and it's a very simple one:
$ nix-build '<nixpkgs>' -A stdenv
/nix/store/k4jklkcag4zq4xkqhkpy156mgfm34ipn-stdenv
$ ls -R result/
result/:
nix-support/ setup
```console
$ nix-build '<nixpkgs>' -A stdenv
/nix/store/k4jklkcag4zq4xkqhkpy156mgfm34ipn-stdenv
$ ls -R result/
result/:
nix-support/ setup
result/nix-support:
propagated-user-env-packages
result/nix-support:
propagated-user-env-packages
```
It has just two files: `/setup` and `/nix-support/propagated-user-env-packages`. Don't worry about the latter. It's empty, in fact. The important file is `/setup`.
How can this simple derivation pull in all of the toolchain and basic tools needed to compile packages? Let's look at the runtime dependencies:
$ nix-store -q --references result
/nix/store/3a45nb37s0ndljp68228snsqr3qsyp96-bzip2-1.0.6
/nix/store/a457ywa1haa0sgr9g7a1pgldrg3s798d-coreutils-8.24
/nix/store/zmd4jk4db5lgxb8l93mhkvr3x92g2sx2-bash-4.3-p39
/nix/store/47sfpm2qclpqvrzijizimk4md1739b1b-gcc-wrapper-4.9.3
...
```console
$ nix-store -q --references result
/nix/store/3a45nb37s0ndljp68228snsqr3qsyp96-bzip2-1.0.6
/nix/store/a457ywa1haa0sgr9g7a1pgldrg3s798d-coreutils-8.24
/nix/store/zmd4jk4db5lgxb8l93mhkvr3x92g2sx2-bash-4.3-p39
/nix/store/47sfpm2qclpqvrzijizimk4md1739b1b-gcc-wrapper-4.9.3
...
```
How can it be? The package must be referring to those other packages somehow. In fact, they are hardcoded in the `/setup` file:
$ head result/setup
export SHELL=/nix/store/zmd4jk4db5lgxb8l93mhkvr3x92g2sx2-bash-4.3-p39/bin/bash
initialPath="/nix/store/a457ywa1haa0sgr9g7a1pgldrg3s798d-coreutils-8.24 ..."
defaultNativeBuildInputs="/nix/store/sgwq15xg00xnm435gjicspm048rqg9y6-patchelf-0.8 ..."
```console
$ head result/setup
export SHELL=/nix/store/zmd4jk4db5lgxb8l93mhkvr3x92g2sx2-bash-4.3-p39/bin/bash
initialPath="/nix/store/a457ywa1haa0sgr9g7a1pgldrg3s798d-coreutils-8.24 ..."
defaultNativeBuildInputs="/nix/store/sgwq15xg00xnm435gjicspm048rqg9y6-patchelf-0.8 ..."
```
## The setup file
@ -53,15 +59,17 @@ Every phase has hooks to run commands before and after the phase has been execut
How to use this file? Like our old builder. To test it, we enter a fake empty derivation, source the `stdenv` `setup`, unpack the hello sources and build it:
$ nix-shell -E 'derivation { name = "fake"; builder = "fake"; system = "x86_64-linux"; }'
nix-shell$ unset PATH
nix-shell$ source /nix/store/k4jklkcag4zq4xkqhkpy156mgfm34ipn-stdenv/setup
nix-shell$ tar -xf hello-2.10.tar.gz
nix-shell$ cd hello-2.10
nix-shell$ configurePhase
...
nix-shell$ buildPhase
...
```console
$ nix-shell -E 'derivation { name = "fake"; builder = "fake"; system = "x86_64-linux"; }'
nix-shell$ unset PATH
nix-shell$ source /nix/store/k4jklkcag4zq4xkqhkpy156mgfm34ipn-stdenv/setup
nix-shell$ tar -xf hello-2.10.tar.gz
nix-shell$ cd hello-2.10
nix-shell$ configurePhase
...
nix-shell$ buildPhase
...
```
_I unset `PATH` to further show that the `stdenv` is sufficiently self-contained to build autotools packages that have no other dependencies._
@ -75,50 +83,58 @@ Note how `stdenv` is a derivation but it's also an attribute set which contains
Let's write a `hello.nix` expression using this newly discovered `stdenv`:
with import <nixpkgs> { };
stdenv.mkDerivation {
name = "hello";
src = ./hello-2.10.tar.gz;
}
```nix
with import <nixpkgs> { };
stdenv.mkDerivation {
name = "hello";
src = ./hello-2.10.tar.gz;
}
```
Don't be scared by the `with` expression. It pulls the `nixpkgs` repository into scope, so we can directly use `stdenv`. It looks very similar to the hello expression in [Pill 8](08-generic-builders.md).
It builds, and runs fine:
$ nix-build hello.nix
...
/nix/store/6y0mzdarm5qxfafvn2zm9nr01d1j0a72-hello
$ result/bin/hello
Hello, world!
```console
$ nix-build hello.nix
...
/nix/store/6y0mzdarm5qxfafvn2zm9nr01d1j0a72-hello
$ result/bin/hello
Hello, world!
```
## The stdenv.mkDerivation builder
Let's take a look at the builder used by `mkDerivation`. You can read the code [here in nixpkgs](https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/make-derivation.nix):
{
# ...
builder = attrs.realBuilder or shell;
args =
attrs.args or [
"-e"
(attrs.builder or ./default-builder.sh)
];
stdenv = result;
# ...
}
```nix
{
# ...
builder = attrs.realBuilder or shell;
args =
attrs.args or [
"-e"
(attrs.builder or ./default-builder.sh)
];
stdenv = result;
# ...
}
```
Also take a look at our old derivation wrapper in previous pills! The builder is bash (that shell variable), the argument to the builder (bash) is `default-builder.sh`, and then we add the environment variable `$stdenv` in the derivation which is the `stdenv` derivation.
You can open [default-builder.sh](https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/default-builder.sh) and see what it does:
source $stdenv/setup
genericBuild
```sh
source $stdenv/setup
genericBuild
```
It's what we did in [Pill 10](10-developing-with-nix-shell.md) to make the derivations `nix-shell` friendly. When entering the shell, the setup file only sets up the environment without building anything. When doing `nix-build`, it actually runs the build process.
To get a clear understanding of the environment variables, look at the .drv of the hello derivation:
```
```console
$ nix derivation show $(nix-instantiate hello.nix)
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
{

282
pills/20-basic-dependencies-and-hooks.md
View file

@ -14,86 +14,98 @@ Note: The complexity of the dependencies and hooks infrastructure has increased,
For the simplest dependencies where the current package directly needs another, we use the `buildInputs` attribute. This is exactly the pattern used in our builder in [Pill 8](08-generic-builders.html). To demo this, let's build GNU Hello, and then another package which provides a shell script that `exec`s it.
let
```nix
let
nixpkgs = import <nixpkgs> { };
nixpkgs = import <nixpkgs> { };
inherit (nixpkgs) stdenv fetchurl which;
inherit (nixpkgs) stdenv fetchurl which;
actualHello = stdenv.mkDerivation {
name = "hello-2.3";
actualHello = stdenv.mkDerivation {
name = "hello-2.3";
src = fetchurl {
url = "mirror://gnu/hello/hello-2.3.tar.bz2";
sha256 = "0c7vijq8y68bpr7g6dh1gny0bff8qq81vnp4ch8pjzvg56wb3js1";
};
};
src = fetchurl {
url = "mirror://gnu/hello/hello-2.3.tar.bz2";
sha256 = "0c7vijq8y68bpr7g6dh1gny0bff8qq81vnp4ch8pjzvg56wb3js1";
};
};
wrappedHello = stdenv.mkDerivation {
name = "hello-wrapper";
wrappedHello = stdenv.mkDerivation {
name = "hello-wrapper";
buildInputs = [
actualHello
which
];
buildInputs = [
actualHello
which
];
unpackPhase = "true";
unpackPhase = "true";
installPhase = ''
mkdir -p "$out/bin"
echo "#! ${stdenv.shell}" >> "$out/bin/hello"
echo "exec $(which hello)" >> "$out/bin/hello"
chmod 0755 "$out/bin/hello"
'';
};
in
wrappedHello
installPhase = ''
mkdir -p "$out/bin"
echo "#! ${stdenv.shell}" >> "$out/bin/hello"
echo "exec $(which hello)" >> "$out/bin/hello"
chmod 0755 "$out/bin/hello"
'';
};
in
wrappedHello
```
Notice that the wrappedHello derivation finds the `hello` binary from the `PATH`. This works because stdenv contains something like:
pkgs=""
for i in $buildInputs; do
findInputs $i
done
```sh
pkgs=""
for i in $buildInputs; do
findInputs $i
done
```
where `findInputs` is defined like:
findInputs() {
local pkg=$1
```sh
findInputs() {
local pkg=$1
## Don't need to repeat already processed package
case $pkgs in
*\ $pkg\ *)
return 0
;;
esac
## Don't need to repeat already processed package
case $pkgs in
*\ $pkg\ *)
return 0
;;
esac
pkgs="$pkgs $pkg "
pkgs="$pkgs $pkg "
## More goes here in reality that we can ignore for now.
}
## More goes here in reality that we can ignore for now.
}
```
then after this is run:
for i in $pkgs; do
addToEnv $i
done
```sh
for i in $pkgs; do
addToEnv $i
done
```
where `addToEnv` is defined like:
addToEnv() {
local pkg=$1
```sh
addToEnv() {
local pkg=$1
if test -d $1/bin; then
addToSearchPath _PATH $1/bin
fi
if test -d $1/bin; then
addToSearchPath _PATH $1/bin
fi
## More goes here in reality that we can ignore for now.
}
## More goes here in reality that we can ignore for now.
}
```
The `addToSearchPath` call adds `$1/bin` to `_PATH` if the former exists (code [here](https://github.com/NixOS/nixpkgs/blob/6675f0a52c0962042a1000c7f20e887d0d26ae25/pkgs/stdenv/generic/setup.sh#L60-L73)). Once all the packages in `buildInputs` have been processed, then content of `_PATH` is added to `PATH`, as follows:
PATH="${_PATH-}${_PATH:+${PATH:+:}}$PATH"
```sh
PATH="${_PATH-}${_PATH:+${PATH:+:}}$PATH"
```
With the real `hello` on the `PATH`, the `installPhase` should hopefully make sense.
@ -101,92 +113,100 @@ With the real `hello` on the `PATH`, the `installPhase` should hopefully make se
The `buildInputs` covers direct dependencies, but what about indirect dependencies where one package needs a second package which needs a third? Nix itself handles this just fine, understanding various dependency closures as covered in previous builds. But what about the conveniences that `buildInputs` provides, namely accumulating in `pkgs` environment variable and inclusion of `pkg/bin` directories on the `PATH`? For this, stdenv provides the `propagatedBuildInputs`:
let
```nix
let
nixpkgs = import <nixpkgs> { };
nixpkgs = import <nixpkgs> { };
inherit (nixpkgs) stdenv fetchurl which;
inherit (nixpkgs) stdenv fetchurl which;
actualHello = stdenv.mkDerivation {
name = "hello-2.3";
actualHello = stdenv.mkDerivation {
name = "hello-2.3";
src = fetchurl {
url = "mirror://gnu/hello/hello-2.3.tar.bz2";
sha256 = "0c7vijq8y68bpr7g6dh1gny0bff8qq81vnp4ch8pjzvg56wb3js1";
};
};
src = fetchurl {
url = "mirror://gnu/hello/hello-2.3.tar.bz2";
sha256 = "0c7vijq8y68bpr7g6dh1gny0bff8qq81vnp4ch8pjzvg56wb3js1";
};
};
intermediary = stdenv.mkDerivation {
name = "middle-man";
intermediary = stdenv.mkDerivation {
name = "middle-man";
propagatedBuildInputs = [ actualHello ];
propagatedBuildInputs = [ actualHello ];
unpackPhase = "true";
unpackPhase = "true";
installPhase = ''
mkdir -p "$out"
'';
};
installPhase = ''
mkdir -p "$out"
'';
};
wrappedHello = stdenv.mkDerivation {
name = "hello-wrapper";
wrappedHello = stdenv.mkDerivation {
name = "hello-wrapper";
buildInputs = [
intermediary
which
];
buildInputs = [
intermediary
which
];
unpackPhase = "true";
unpackPhase = "true";
installPhase = ''
mkdir -p "$out/bin"
echo "#! ${stdenv.shell}" >> "$out/bin/hello"
echo "exec $(which hello)" >> "$out/bin/hello"
chmod 0755 "$out/bin/hello"
'';
};
in
wrappedHello
installPhase = ''
mkdir -p "$out/bin"
echo "#! ${stdenv.shell}" >> "$out/bin/hello"
echo "exec $(which hello)" >> "$out/bin/hello"
chmod 0755 "$out/bin/hello"
'';
};
in
wrappedHello
```
See how the intermediate package has a `propagatedBuildInputs` dependency, but the wrapper only needs a `buildInputs` dependency on the intermediary.
How does this work? You might think we do something in Nix, but actually it's done not at eval time but at build time in bash. let's look at part of the `fixupPhase` of stdenv:
fixupPhase() {
```sh
fixupPhase() {
## Elided
## Elided
if test -n "$propagatedBuildInputs"; then
mkdir -p "$out/nix-support"
echo "$propagatedBuildInputs" > "$out/nix-support/propagated-build-inputs"
fi
if test -n "$propagatedBuildInputs"; then
mkdir -p "$out/nix-support"
echo "$propagatedBuildInputs" > "$out/nix-support/propagated-build-inputs"
fi
## Elided
## Elided
}
}
```
This dumps the propagated build inputs in a so-named file in `$out/nix-support/`. Then, back in `findInputs` look at the lines at the bottom we elided before:
findInputs() {
local pkg=$1
```sh
findInputs() {
local pkg=$1
## More goes here in reality that we can ignore for now.
## More goes here in reality that we can ignore for now.
if test -f $pkg/nix-support/propagated-build-inputs; then
for i in $(cat $pkg/nix-support/propagated-build-inputs); do
findInputs $i
done
fi
}
if test -f $pkg/nix-support/propagated-build-inputs; then
for i in $(cat $pkg/nix-support/propagated-build-inputs); do
findInputs $i
done
fi
}
```
See how `findInputs` is actually recursive, looking at the propagated build inputs of each dependency, and those dependencies' propagated build inputs, etc.
We actually simplified the `findInputs` call site from before; `propagatedBuildInputs` is also looped over in reality:
pkgs=""
for i in $buildInputs $propagatedBuildInputs; do
findInputs $i
done
```sh
pkgs=""
for i in $buildInputs $propagatedBuildInputs; do
findInputs $i
done
```
This demonstrates an important point. For the _current_ package alone, it doesn't matter whether a dependency is propagated or not. It will be processed the same way: called with `findInputs` and `addToEnv`. (The packages discovered by `findInputs`, which are also accumulated in `pkgs` and passed to `addToEnv`, are also the same in both cases.) Downstream however, it certainly does matter because only the propagated immediate dependencies are put in the `$out/nix-support/propagated-build-inputs`.
@ -196,18 +216,20 @@ As we mentioned above, sometimes dependencies need to influence the packages tha
Setup hooks are the basic building block we have for this. In nixpkgs, a "hook" is basically a bash callback, and a setup hook is no exception. Let's look at the last part of `findInputs` we haven't covered:
findInputs() {
local pkg=$1
```sh
findInputs() {
local pkg=$1
## More goes here in reality that we can ignore for now.
## More goes here in reality that we can ignore for now.
if test -f $pkg/nix-support/setup-hook; then
source $pkg/nix-support/setup-hook
fi
if test -f $pkg/nix-support/setup-hook; then
source $pkg/nix-support/setup-hook
fi
## More goes here in reality that we can ignore for now.
## More goes here in reality that we can ignore for now.
}
}
```
If a package includes the path `pkg/nix-support/setup-hook`, it will be sourced by any stdenv-based build including that as a dependency.
@ -221,26 +243,30 @@ As a first step, we can move that logic to a setup hook on the C compiler; indee
The other half of `addToEnv` is:
addToEnv() {
local pkg=$1
```sh
addToEnv() {
local pkg=$1
## More goes here in reality that we can ignore for now.
## More goes here in reality that we can ignore for now.
# Run the package-specific hooks set by the setup-hook scripts.
for i in "${envHooks[@]}"; do
$i $pkg
done
}
# Run the package-specific hooks set by the setup-hook scripts.
for i in "${envHooks[@]}"; do
$i $pkg
done
}
```
Functions listed in `envHooks` are applied to every package passed to `addToEnv`. One can write a setup hook like:
anEnvHook() {
local pkg=$1
```sh
anEnvHook() {
local pkg=$1
echo "I'm depending on "$pkg""
}
echo "I'm depending on \"$pkg\""
}
envHooks+=(anEnvHook)
envHooks+=(anEnvHook)
```
and if one dependency has that setup hook then all of them will be so `echo`ed. Allowing dependencies to learn about their _sibling_ dependencies is exactly what compilers need.