mirror of
https://github.com/NixOS/nix-pills
synced 2024-09-19 04:00:13 -04:00
Port pill #8
This commit is contained in:
parent
06530261a2
commit
c6af654c89
|
@ -4,5 +4,320 @@
|
||||||
version="5.0"
|
version="5.0"
|
||||||
xml:id="generic-builders">
|
xml:id="generic-builders">
|
||||||
|
|
||||||
<title>generic builders</title>
|
<title>Generic builders</title>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Welcome to the 8th Nix pill. In the previous
|
||||||
|
<link linkend="working-derivation">7th pill</link> we successfully built a
|
||||||
|
derivation. We wrote a builder script that compiled a C file and installed
|
||||||
|
the binary under the nix store.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
In this post, we will generalize the builder script, write a Nix expression
|
||||||
|
for <link
|
||||||
|
xlink:href="http://www.gnu.org/software/hello/">GNU hello world</link>
|
||||||
|
and create a wrapper around the derivation built-in function.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<section>
|
||||||
|
<title>Packaging GNU hello world</title>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
In the previous pill we packaged a simple .c file, which was being
|
||||||
|
compiled with a raw gcc call. That's not a good example of project. Many
|
||||||
|
use autotools, and since we're going to generalize our builder, better do
|
||||||
|
it with the most used build system.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
<link xlink:href="http://www.gnu.org/software/hello/">GNU hello world</link>,
|
||||||
|
despite its name, is a simple yet complete project using autotools.
|
||||||
|
Fetch the latest tarball here:
|
||||||
|
<link xlink:href="http://ftp.gnu.org/gnu/hello/hello-2.9.tar.gz">http://ftp.gnu.org/gnu/hello/hello-2.9.tar.gz</link>.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Let's create a builder script for GNU hello world:
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<screen><xi:include href="./08/hello-builder.txt" parse="text" /></screen>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
And the derivation hello.nix:
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<screen><xi:include href="./08/hello-nix.txt" parse="text" /></screen>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Now build it with <command>nix-build hello.nix</command> and you can
|
||||||
|
launch <filename>result/bin/hello</filename>. Nothing easier, but do we
|
||||||
|
have to create a builder.sh for each package? Do we always have to pass
|
||||||
|
the dependencies to the <literal>derivation</literal> function?
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Please note the <command>--prefix=$out</command> we were talking about in
|
||||||
|
the <link linkend="working-derivation">previous pill</link>.
|
||||||
|
</para>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section>
|
||||||
|
<title>A generic builder</title>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Let's a create a generic <filename>builder.sh</filename> for autotools
|
||||||
|
projects:
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<screen><xi:include href="./08/generic-builder.txt" parse="text" /></screen>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
What do we do here?
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<orderedlist>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
Exit the build on any error with <command>set -e</command>.
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
First <command>unset PATH</command>, because it's initially set to a
|
||||||
|
non-existant path.
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
We'll see this below in detail, however for each path in
|
||||||
|
<code>$buildInputs</code>, we append <code>bin</code> to
|
||||||
|
<code>PATH</code>.
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
Unpack the source.
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
Find a directory where the source has been unpacked and
|
||||||
|
<command>cd</command> into it.
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
Once we're set up, compile and install.
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
|
</orderedlist>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
As you can see, there's no reference to "hello" in the builder anymore.
|
||||||
|
It still does several assumptions, but it's certainly more generic.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Now let's rewrite <filename>hello.nix</filename>:
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<screen><xi:include href="./08/hello-nix-rev-1.txt" parse="text" /></screen>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
All clear, except that buildInputs. However it's easier than any black
|
||||||
|
magic you are thinking in this moment.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Nix is able to convert a list to a string. It first converts the elements
|
||||||
|
to strings, and then concatenates them separated by a space:
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<screen><xi:include href="./08/to-string.txt" parse="text" /></screen>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Recall that derivations can be converted to a string, hence:
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<screen><xi:include href="./08/to-string-nixpkgs.txt" parse="text" /></screen>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Simple! The buildInputs variable is a string with out paths separated by
|
||||||
|
space, perfect for bash usage in a for loop.
|
||||||
|
</para>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section>
|
||||||
|
<title>A more convenient derivation function</title>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
We managed to write a builder that can be used for multiple autotools
|
||||||
|
projects. But in the hello.nix expression we are specifying tools that
|
||||||
|
are common to more projects; we don't want to pass them everytime.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
A natural approach would be to create a function that accepts an
|
||||||
|
attribute set, similar to the one used by the derivation function, and
|
||||||
|
merge it with another attribute set containing values common to many
|
||||||
|
projects.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Create <filename>autotools.nix</filename>:
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<screen><xi:include href="./08/autotools-nix.txt" parse="text" /></screen>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Ok now we have to remember a little about
|
||||||
|
<link linkend="functions-and-imports">Nix functions</link>. The whole nix
|
||||||
|
expression of this <filename>autotools.nix</filename> file will evaluate
|
||||||
|
to a function. This function accepts a parameter <code>pkgs</code>, then
|
||||||
|
returns a function which accepts a parameter <code>attrs</code>.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
The body of the function is simple, yet at first sight it might be hard
|
||||||
|
to grasp:
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<orderedlist>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
First drop in the scope the magic <code>pkgs</code> attribute set.
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
Within a let expression we define an helper variable,
|
||||||
|
<code>defaultAttrs</code>, which serves as a set of common attributes
|
||||||
|
used in derivations.
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
Finally we create the derivation with that strange expression,
|
||||||
|
(<code>defaultAttrs // attrs</code>).
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
|
</orderedlist>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
The
|
||||||
|
<link xlink:href="http://nixos.org/nix/manual/#idm47361539098656">// operator</link>
|
||||||
|
is an operator between two sets. The result is the union of the two sets.
|
||||||
|
In case of conflicts between attribute names, the value on the right set
|
||||||
|
is preferred.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
So we use <code>defaultAttrs</code> as base set, and add (or override) the
|
||||||
|
attributes from <code>attrs</code>.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
A couple of examples ought to be enough to clear out the behavior of the
|
||||||
|
operator:
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<screen><xi:include href="./08/set-union.txt" parse="text" /></screen>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Complete the new <filename>builder.sh</filename> by adding
|
||||||
|
<code>$baseInputs</code> in the <code>for</code> loop together with
|
||||||
|
<code>$buildInputs</code>. As you noticed, we passed that new variable in
|
||||||
|
the derivation. Instead of merging buildInputs with the base ones, we
|
||||||
|
prefer to preserve buildInputs as seen by the caller, so we keep them
|
||||||
|
separated. Just a matter of choice.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Then we rewrite <filename>hello.nix</filename> as follows:
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<screen><xi:include href="./08/hello-nix-rev-2.txt" parse="text" /></screen>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Finally! We got a very simple description of a package! A couple of
|
||||||
|
remarks that you may find useful to keep understanding the nix language:
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<itemizedlist>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
We assigned to pkgs the import that we did in the previous expressions
|
||||||
|
in the "with", don't be afraid. It's that straightforward.
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
The mkDerivation variable is a nice example of partial application,
|
||||||
|
look at it as (<code>import ./autotools.nix</code>) <code>pkgs</code>.
|
||||||
|
First we import the expression, then we apply the <code>pkgs</code>
|
||||||
|
parameter. That will give us a function that accepts the attribute
|
||||||
|
set <code>attrs</code>.
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
We create the derivation specifying only name and src. If the project
|
||||||
|
eventually needed other dependencies to be in PATH, then we would
|
||||||
|
simply add those to buildInputs (not specified in hello.nix because
|
||||||
|
empty).
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
|
</itemizedlist>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Note we didn't use any other library. Special C flags may be needed to
|
||||||
|
find include files of other libraries at compile time, and ld flags at
|
||||||
|
link time.
|
||||||
|
</para>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section>
|
||||||
|
<title>Conclusion</title>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Nix gives us the bare metal tools for creating derivations, setting up a
|
||||||
|
build environment and storing the result in the nix store.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Out of this we managed to create a generic builder for autotools projects,
|
||||||
|
and a function <code>mkDerivation</code> that composes by default the
|
||||||
|
common components used in autotools projects instead of repeating them
|
||||||
|
in all the packages we would write.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
We are feeling the way a Nix system grows up: it's about creating and
|
||||||
|
composing derivations with the Nix language.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
<emphasis role="underline">Analogy</emphasis>: in C you create objects
|
||||||
|
in the heap, and then you compose them inside new objects. Pointers are
|
||||||
|
used to refer to other objects.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
In Nix you create derivations stored in the nix store, and then you
|
||||||
|
compose them by creating new derivations. Store paths are used to refer
|
||||||
|
to other derivations.
|
||||||
|
</para>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section>
|
||||||
|
<title>Next pill</title>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
...we will talk a little about runtime dependencies. Is the GNU hello
|
||||||
|
world package self-contained? What are its runtime dependencies? We only
|
||||||
|
specified build dependencies by means of using other derivations in the
|
||||||
|
"hello" derivation.
|
||||||
|
</para>
|
||||||
|
</section>
|
||||||
</chapter>
|
</chapter>
|
||||||
|
|
11
pills/08/autotools-nix.txt
Normal file
11
pills/08/autotools-nix.txt
Normal file
|
@ -0,0 +1,11 @@
|
||||||
|
pkgs: attrs:
|
||||||
|
with pkgs;
|
||||||
|
let defaultAttrs = {
|
||||||
|
builder = "${bash}/bin/bash";
|
||||||
|
args = [ ./builder.sh ];
|
||||||
|
baseInputs = [ gnutar gzip gnumake gcc binutils coreutils gawk gnused gnugrep ];
|
||||||
|
buildInputs = [];
|
||||||
|
system = builtins.currentSystem;
|
||||||
|
};
|
||||||
|
in
|
||||||
|
derivation (defaultAttrs // attrs)
|
18
pills/08/generic-builder.txt
Normal file
18
pills/08/generic-builder.txt
Normal file
|
@ -0,0 +1,18 @@
|
||||||
|
set -e
|
||||||
|
unset PATH
|
||||||
|
for p in $buildInputs; do
|
||||||
|
export PATH=$p/bin${PATH:+:}$PATH
|
||||||
|
done
|
||||||
|
|
||||||
|
tar -xf $src
|
||||||
|
|
||||||
|
for d in *; do
|
||||||
|
if [ -d "$d" ]; then
|
||||||
|
cd "$d"
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
./configure --prefix=$out
|
||||||
|
make
|
||||||
|
make install
|
7
pills/08/hello-builder.txt
Normal file
7
pills/08/hello-builder.txt
Normal file
|
@ -0,0 +1,7 @@
|
||||||
|
hello_builder.sh
|
||||||
|
export PATH="$gnutar/bin:$gcc/bin:$gnumake/bin:$coreutils/bin:$gawk/bin:$gzip/bin:$gnugrep/bin:$gnused/bin:$binutils/bin"
|
||||||
|
tar -xzf $src
|
||||||
|
cd hello-2.9
|
||||||
|
./configure --prefix=$out
|
||||||
|
make
|
||||||
|
make install
|
9
pills/08/hello-nix-rev-1.txt
Normal file
9
pills/08/hello-nix-rev-1.txt
Normal file
|
@ -0,0 +1,9 @@
|
||||||
|
with (import <nixpkgs> {});
|
||||||
|
derivation {
|
||||||
|
name = "hello";
|
||||||
|
builder = "${bash}/bin/bash";
|
||||||
|
args = [ ./builder.sh ];
|
||||||
|
buildInputs = [ gnutar gzip gnumake gcc binutils coreutils gawk gnused gnugrep ];
|
||||||
|
src = ./hello-2.9.tar.gz;
|
||||||
|
system = builtins.currentSystem;
|
||||||
|
}
|
7
pills/08/hello-nix-rev-2.txt
Normal file
7
pills/08/hello-nix-rev-2.txt
Normal file
|
@ -0,0 +1,7 @@
|
||||||
|
let
|
||||||
|
pkgs = import <nixpkgs> {};
|
||||||
|
mkDerivation = import ./autotools.nix pkgs;
|
||||||
|
in mkDerivation {
|
||||||
|
name = "hello";
|
||||||
|
src = ./hello-2.9.tar.gz;
|
||||||
|
}
|
9
pills/08/hello-nix.txt
Normal file
9
pills/08/hello-nix.txt
Normal file
|
@ -0,0 +1,9 @@
|
||||||
|
with (import <nixpkgs> {});
|
||||||
|
derivation {
|
||||||
|
name = "hello";
|
||||||
|
builder = "${bash}/bin/bash";
|
||||||
|
args = [ ./hello_builder.sh ];
|
||||||
|
inherit gnutar gzip gnumake gcc binutils coreutils gawk gnused gnugrep;
|
||||||
|
src = ./hello-2.9.tar.gz;
|
||||||
|
system = builtins.currentSystem;
|
||||||
|
}
|
4
pills/08/set-union.txt
Normal file
4
pills/08/set-union.txt
Normal file
|
@ -0,0 +1,4 @@
|
||||||
|
nix-repl> { a = "b"; } // { c = "d"; }
|
||||||
|
{ a = "b"; c = "d"; }
|
||||||
|
nix-repl> { a = "b"; } // { a = "c"; }
|
||||||
|
{ a = "c"; }
|
6
pills/08/to-string-nixpkgs.txt
Normal file
6
pills/08/to-string-nixpkgs.txt
Normal file
|
@ -0,0 +1,6 @@
|
||||||
|
nix-repl> :l <nixpkgs>
|
||||||
|
Added 3950 variables.
|
||||||
|
nix-repl> builtins.toString gnugrep
|
||||||
|
"/nix/store/g5gdylclfh6d224kqh9sja290pk186xd-gnugrep-2.14"
|
||||||
|
nix-repl> builtins.toString [ gnugrep gnused ]
|
||||||
|
"/nix/store/g5gdylclfh6d224kqh9sja290pk186xd-gnugrep-2.14 /nix/store/krgdc4sknzpw8iyk9p20lhqfd52kjmg0-gnused-4.2.2"
|
4
pills/08/to-string.txt
Normal file
4
pills/08/to-string.txt
Normal file
|
@ -0,0 +1,4 @@
|
||||||
|
nix-repl> builtins.toString 123
|
||||||
|
"123"
|
||||||
|
nix-repl> builtins.toString [ 123 456 ]
|
||||||
|
"123 456"
|
Loading…
Reference in a new issue