diff --git a/pills/08-generic-builders.xml b/pills/08-generic-builders.xml index a75a3bf..7f8a2b3 100644 --- a/pills/08-generic-builders.xml +++ b/pills/08-generic-builders.xml @@ -4,5 +4,320 @@ version="5.0" xml:id="generic-builders"> -generic builders + Generic builders + + + Welcome to the 8th Nix pill. In the previous + 7th pill we successfully built a + derivation. We wrote a builder script that compiled a C file and installed + the binary under the nix store. + + + + In this post, we will generalize the builder script, write a Nix expression + for GNU hello world + and create a wrapper around the derivation built-in function. + + +
+ Packaging GNU hello world + + + In the previous pill we packaged a simple .c file, which was being + compiled with a raw gcc call. That's not a good example of project. Many + use autotools, and since we're going to generalize our builder, better do + it with the most used build system. + + + + GNU hello world, + despite its name, is a simple yet complete project using autotools. + Fetch the latest tarball here: + http://ftp.gnu.org/gnu/hello/hello-2.9.tar.gz. + + + + Let's create a builder script for GNU hello world: + + + + + + And the derivation hello.nix: + + + + + + Now build it with nix-build hello.nix and you can + launch result/bin/hello. Nothing easier, but do we + have to create a builder.sh for each package? Do we always have to pass + the dependencies to the derivation function? + + + + Please note the --prefix=$out we were talking about in + the previous pill. + +
+ +
+ A generic builder + + + Let's a create a generic builder.sh for autotools + projects: + + + + + + What do we do here? + + + + + + Exit the build on any error with set -e. + + + + + First unset PATH, because it's initially set to a + non-existant path. + + + + + We'll see this below in detail, however for each path in + $buildInputs, we append bin to + PATH. + + + + + Unpack the source. + + + + + Find a directory where the source has been unpacked and + cd into it. + + + + + Once we're set up, compile and install. + + + + + + As you can see, there's no reference to "hello" in the builder anymore. + It still does several assumptions, but it's certainly more generic. + + + + Now let's rewrite hello.nix: + + + + + + All clear, except that buildInputs. However it's easier than any black + magic you are thinking in this moment. + + + + Nix is able to convert a list to a string. It first converts the elements + to strings, and then concatenates them separated by a space: + + + + + + Recall that derivations can be converted to a string, hence: + + + + + + Simple! The buildInputs variable is a string with out paths separated by + space, perfect for bash usage in a for loop. + +
+ +
+ A more convenient derivation function + + + We managed to write a builder that can be used for multiple autotools + projects. But in the hello.nix expression we are specifying tools that + are common to more projects; we don't want to pass them everytime. + + + + A natural approach would be to create a function that accepts an + attribute set, similar to the one used by the derivation function, and + merge it with another attribute set containing values common to many + projects. + + + + Create autotools.nix: + + + + + + Ok now we have to remember a little about + Nix functions. The whole nix + expression of this autotools.nix file will evaluate + to a function. This function accepts a parameter pkgs, then + returns a function which accepts a parameter attrs. + + + + The body of the function is simple, yet at first sight it might be hard + to grasp: + + + + + + First drop in the scope the magic pkgs attribute set. + + + + + Within a let expression we define an helper variable, + defaultAttrs, which serves as a set of common attributes + used in derivations. + + + + + Finally we create the derivation with that strange expression, + (defaultAttrs // attrs). + + + + + + The + // operator + is an operator between two sets. The result is the union of the two sets. + In case of conflicts between attribute names, the value on the right set + is preferred. + + + + So we use defaultAttrs as base set, and add (or override) the + attributes from attrs. + + + + A couple of examples ought to be enough to clear out the behavior of the + operator: + + + + + + Complete the new builder.sh by adding + $baseInputs in the for loop together with + $buildInputs. As you noticed, we passed that new variable in + the derivation. Instead of merging buildInputs with the base ones, we + prefer to preserve buildInputs as seen by the caller, so we keep them + separated. Just a matter of choice. + + + + Then we rewrite hello.nix as follows: + + + + + + Finally! We got a very simple description of a package! A couple of + remarks that you may find useful to keep understanding the nix language: + + + + + + We assigned to pkgs the import that we did in the previous expressions + in the "with", don't be afraid. It's that straightforward. + + + + + The mkDerivation variable is a nice example of partial application, + look at it as (import ./autotools.nix) pkgs. + First we import the expression, then we apply the pkgs + parameter. That will give us a function that accepts the attribute + set attrs. + + + + + We create the derivation specifying only name and src. If the project + eventually needed other dependencies to be in PATH, then we would + simply add those to buildInputs (not specified in hello.nix because + empty). + + + + + + Note we didn't use any other library. Special C flags may be needed to + find include files of other libraries at compile time, and ld flags at + link time. + +
+ +
+ Conclusion + + + Nix gives us the bare metal tools for creating derivations, setting up a + build environment and storing the result in the nix store. + + + + Out of this we managed to create a generic builder for autotools projects, + and a function mkDerivation that composes by default the + common components used in autotools projects instead of repeating them + in all the packages we would write. + + + + We are feeling the way a Nix system grows up: it's about creating and + composing derivations with the Nix language. + + + + Analogy: in C you create objects + in the heap, and then you compose them inside new objects. Pointers are + used to refer to other objects. + + + + In Nix you create derivations stored in the nix store, and then you + compose them by creating new derivations. Store paths are used to refer + to other derivations. + +
+ +
+ Next pill + + + ...we will talk a little about runtime dependencies. Is the GNU hello + world package self-contained? What are its runtime dependencies? We only + specified build dependencies by means of using other derivations in the + "hello" derivation. + +
diff --git a/pills/08/autotools-nix.txt b/pills/08/autotools-nix.txt new file mode 100644 index 0000000..e224c4e --- /dev/null +++ b/pills/08/autotools-nix.txt @@ -0,0 +1,11 @@ +pkgs: attrs: + with pkgs; + let defaultAttrs = { + builder = "${bash}/bin/bash"; + args = [ ./builder.sh ]; + baseInputs = [ gnutar gzip gnumake gcc binutils coreutils gawk gnused gnugrep ]; + buildInputs = []; + system = builtins.currentSystem; + }; + in + derivation (defaultAttrs // attrs) diff --git a/pills/08/generic-builder.txt b/pills/08/generic-builder.txt new file mode 100644 index 0000000..0d1c764 --- /dev/null +++ b/pills/08/generic-builder.txt @@ -0,0 +1,18 @@ +set -e +unset PATH +for p in $buildInputs; do + export PATH=$p/bin${PATH:+:}$PATH +done + +tar -xf $src + +for d in *; do + if [ -d "$d" ]; then + cd "$d" + break + fi +done + +./configure --prefix=$out +make +make install diff --git a/pills/08/hello-builder.txt b/pills/08/hello-builder.txt new file mode 100644 index 0000000..a24d0d5 --- /dev/null +++ b/pills/08/hello-builder.txt @@ -0,0 +1,7 @@ +hello_builder.sh +export PATH="$gnutar/bin:$gcc/bin:$gnumake/bin:$coreutils/bin:$gawk/bin:$gzip/bin:$gnugrep/bin:$gnused/bin:$binutils/bin" +tar -xzf $src +cd hello-2.9 +./configure --prefix=$out +make +make install diff --git a/pills/08/hello-nix-rev-1.txt b/pills/08/hello-nix-rev-1.txt new file mode 100644 index 0000000..4646387 --- /dev/null +++ b/pills/08/hello-nix-rev-1.txt @@ -0,0 +1,9 @@ +with (import {}); +derivation { + name = "hello"; + builder = "${bash}/bin/bash"; + args = [ ./builder.sh ]; + buildInputs = [ gnutar gzip gnumake gcc binutils coreutils gawk gnused gnugrep ]; + src = ./hello-2.9.tar.gz; + system = builtins.currentSystem; +} diff --git a/pills/08/hello-nix-rev-2.txt b/pills/08/hello-nix-rev-2.txt new file mode 100644 index 0000000..cfb07c5 --- /dev/null +++ b/pills/08/hello-nix-rev-2.txt @@ -0,0 +1,7 @@ +let + pkgs = import {}; + mkDerivation = import ./autotools.nix pkgs; +in mkDerivation { + name = "hello"; + src = ./hello-2.9.tar.gz; +} diff --git a/pills/08/hello-nix.txt b/pills/08/hello-nix.txt new file mode 100644 index 0000000..d934d5b --- /dev/null +++ b/pills/08/hello-nix.txt @@ -0,0 +1,9 @@ +with (import {}); +derivation { + name = "hello"; + builder = "${bash}/bin/bash"; + args = [ ./hello_builder.sh ]; + inherit gnutar gzip gnumake gcc binutils coreutils gawk gnused gnugrep; + src = ./hello-2.9.tar.gz; + system = builtins.currentSystem; +} diff --git a/pills/08/set-union.txt b/pills/08/set-union.txt new file mode 100644 index 0000000..392e6a7 --- /dev/null +++ b/pills/08/set-union.txt @@ -0,0 +1,4 @@ +nix-repl> { a = "b"; } // { c = "d"; } +{ a = "b"; c = "d"; } +nix-repl> { a = "b"; } // { a = "c"; } +{ a = "c"; } diff --git a/pills/08/to-string-nixpkgs.txt b/pills/08/to-string-nixpkgs.txt new file mode 100644 index 0000000..f09ee03 --- /dev/null +++ b/pills/08/to-string-nixpkgs.txt @@ -0,0 +1,6 @@ +nix-repl> :l +Added 3950 variables. +nix-repl> builtins.toString gnugrep +"/nix/store/g5gdylclfh6d224kqh9sja290pk186xd-gnugrep-2.14" +nix-repl> builtins.toString [ gnugrep gnused ] +"/nix/store/g5gdylclfh6d224kqh9sja290pk186xd-gnugrep-2.14 /nix/store/krgdc4sknzpw8iyk9p20lhqfd52kjmg0-gnused-4.2.2" diff --git a/pills/08/to-string.txt b/pills/08/to-string.txt new file mode 100644 index 0000000..0e126a3 --- /dev/null +++ b/pills/08/to-string.txt @@ -0,0 +1,4 @@ +nix-repl> builtins.toString 123 +"123" +nix-repl> builtins.toString [ 123 456 ] +"123 456"