mirror of
https://github.com/NixOS/nix-pills
synced 2024-09-19 04:00:13 -04:00
bcea6b35ed
In #89 it has been reported that the third step can't be done as `nix-repl` doesn't evaluate anymore on 18.09 and unstable as in both cases an evaluation error will be thrown. First of all this may confuse new users who want to learn Nix, but don't know the ecosystem sufficiently to understand why the install failed. As recent NixOS versions (unstable and 18.09) use Nix 2.0 by default and unstable doesn't evaluate with Nix 1.x anymore it should be a safe thing to do now. This patch covers two aspects: * Using Nix 2.0: I replayed the installation steps with `nix-env` to ensure that the steps and explanations provided in the first three chapters are still valid. * Replacing `nix-repl` references: most of the cases it was sufficient to replace `nix-repl` with the newly introduced command `nix repl`. In chapter three `nix-repl` was used to demonstrate the installation of a package with `nix-env`. I decided to use `nix-index` as demo package as I figured this tool to be extremely helpful to locate packages by output files. The explanation that Nix is not only a tool for package/derivation management, but a functional language as well was moved to chapter four where the basics of the language were actually covered. This change is just a first step towards an updated series, in the future we may want to use even more Nix 2.0 features (such as `nix-build` vs. `nix build` with `nix log`). Fixes #89 Possibly supersedes #71
327 lines
14 KiB
XML
327 lines
14 KiB
XML
<chapter xmlns="http://docbook.org/ns/docbook"
|
|
xmlns:xlink="http://www.w3.org/1999/xlink"
|
|
xmlns:xi="http://www.w3.org/2001/XInclude"
|
|
version="5.0"
|
|
xml:id="working-derivation">
|
|
|
|
<title>Working Derivation</title>
|
|
|
|
<section>
|
|
<title>Introduction</title>
|
|
<para>
|
|
Welcome to the seventh nix pill. In the previous
|
|
<link linkend="our-first-derivation">sixth pill</link> we introduced the
|
|
notion of derivation in the Nix language — how to define a raw derivation
|
|
and how to (try to) build it.
|
|
</para>
|
|
<para>
|
|
In this post we continue along the path, by creating a derivation that
|
|
actually builds something. Then, we try to package a real program: we
|
|
compile a simple C file and create a derivation out of it, given a blessed
|
|
toolchain.
|
|
</para>
|
|
|
|
<para>
|
|
I remind you how to enter the Nix environment:
|
|
<command>source ~/.nix-profile/etc/profile.d/nix.sh</command>
|
|
</para>
|
|
</section>
|
|
|
|
<section>
|
|
<title>Using a script as a builder</title>
|
|
|
|
<para>
|
|
What's the easiest way to run a sequence of commands for building
|
|
something? A bash script. We write a custom bash script, and we want it to
|
|
be our builder. Given a <filename>builder.sh</filename>, we want the
|
|
derivation to run <command>bash builder.sh</command>.
|
|
</para>
|
|
|
|
<para>
|
|
We don't use hash bangs in <filename>builder.sh</filename>, because at the
|
|
time we are writing it we do not know the path to
|
|
<application>bash</application> in the nix store. Yes, even bash is in the
|
|
nix store, everything is there.
|
|
</para>
|
|
|
|
<para>
|
|
We don't even use <application>/usr/bin/env</application>, because then we
|
|
lose the cool stateless property of Nix. Not to mention that
|
|
<envar>PATH</envar> gets cleared when building, so it wouldn't find
|
|
<application>bash</application> anyway.
|
|
</para>
|
|
|
|
<para>
|
|
In summary, we want the builder to be <application>bash</application>, and
|
|
pass it an argument, <literal>builder.sh</literal>. Turns out the
|
|
<function>derivation</function> function accepts an optional
|
|
<parameter>args</parameter> attribute which is used to pass arguments to
|
|
the builder executable.
|
|
</para>
|
|
|
|
<para>
|
|
First of all, let's write our <filename>builder.sh</filename> in the
|
|
current directory:
|
|
|
|
<programlisting><xi:include href="./07/builder.sh.txt" parse="text" /></programlisting>
|
|
|
|
As we covered in the previous pill, Nix computes the output path of the
|
|
derivation. The resulting <literal>.drv</literal> file contains a list of
|
|
environment variables passed to the builder. One of these is
|
|
<varname>$out</varname>.
|
|
</para>
|
|
<para>
|
|
What we have to do is create something in the path
|
|
<varname>$out</varname>, be it a file or a directory. In this case we are
|
|
creating a file.
|
|
</para>
|
|
|
|
<para>
|
|
In addition, we print out the environment variables during the build
|
|
process. We cannot use <application>env</application> for this, because
|
|
<application>env</application> is part of
|
|
<application>coreutils</application> and we don't have a dependency to it
|
|
yet. We only have <application>bash</application> for now.
|
|
</para>
|
|
|
|
<para>
|
|
Like for coreutils in the previous pill, we get a blessed bash for free
|
|
from our magic nixpkgs stuff:
|
|
|
|
<xi:include href="./07/bash.xml" />
|
|
|
|
So with the usual trick, we can refer to
|
|
<application>bin/bash</application> and create our derivation:
|
|
|
|
<xi:include href="./07/simple-derivation.xml" />
|
|
|
|
We did it! The contents of
|
|
<filename>/nix/store/w024zci0x1hh1wj6gjq0jagkc1sgrf5r-<emphasis>foo</emphasis></filename>
|
|
is really foo. We've built our first derivation.
|
|
</para>
|
|
<para>
|
|
Note that we used <code>./builder.sh</code> and not
|
|
<code>"./builder.sh"</code>. This way, it is parsed as a path, and Nix
|
|
performs some magic which we will cover later. Try using the string
|
|
version and you will find that it cannot find
|
|
<filename>builder.sh</filename>. This is because it tries to find it
|
|
relative to the temporary build directory.
|
|
</para>
|
|
</section>
|
|
|
|
<section>
|
|
<title>The builder environment</title>
|
|
<para>
|
|
Let's inspect those environment variables printed during the build process.
|
|
<itemizedlist>
|
|
<listitem><para>
|
|
<envar>$HOME</envar> is not your home directory, and
|
|
<filename>/homeless-shelter</filename> doesn't exist at all. We force
|
|
packages not to depend on <envar>$HOME</envar> during the build
|
|
process.
|
|
</para></listitem>
|
|
<listitem><para>
|
|
<envar>$PATH</envar> plays the same game as <envar>$HOME</envar>
|
|
</para></listitem>
|
|
<listitem><para>
|
|
<envar>$NIX_BUILD_CORES</envar> and <envar>$NIX_STORE</envar> are
|
|
<link xlink:href="http://nixos.org/nix/manual/#sec-conf-file">nix
|
|
configuration options</link>
|
|
</para></listitem>
|
|
<listitem><para>
|
|
<envar>$PWD</envar> and <envar>$TMP</envar> clearly show that nix
|
|
created a temporary build directory
|
|
</para></listitem>
|
|
<listitem><para>
|
|
Then <envar>$builder</envar>, <envar>$name</envar>,
|
|
<envar>$out</envar>, and <envar>$system</envar> are variables set due
|
|
to the .drv file's contents.
|
|
</para></listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
<para>
|
|
And that's how we were able to use <envar>$out</envar> in our derivation
|
|
and put stuff in it. It's like Nix reserved a slot in the nix store for
|
|
us, and we must fill it.
|
|
</para>
|
|
<para>
|
|
<!-- Not really sure what this paragraph is meant to mean, particularly
|
|
wrt the insistence that this is not make's DESTDIR. I've left it as is but
|
|
it should probably be written more clearly. -->
|
|
In terms of autotools, <envar>$out</envar> will be the
|
|
<option>--prefix</option> path. Yes, not the make
|
|
<option>DESTDIR</option>, but the <option>--prefix</option>. That's the
|
|
essence of stateless packaging. You don't install the package in a global
|
|
common path under <filename>/</filename>, you install it in a local
|
|
isolated path under your nix store slot.
|
|
</para>
|
|
</section>
|
|
<section>
|
|
<title>The .drv contents</title>
|
|
<para>
|
|
We added something else to the derivation this time: the args attribute.
|
|
Let's see how this changed the .drv compared to the previous pill:
|
|
<xi:include href="./07/foo.drv.xml" />
|
|
|
|
Much like the usual .drv, except that there's a list of arguments in there
|
|
passed to the builder (<application>bash</application>) with
|
|
<filename>builder.sh</filename>… In the nix store..? Nix automatically
|
|
copies files or directories needed for the build into the store to ensure
|
|
that they are not changed during the build process and that the deployment
|
|
is stateless and independent of the building machine.
|
|
<filename>builder.sh</filename> is not only in the arguments passed to the
|
|
builder, it's also in the input derivations.
|
|
</para>
|
|
<para>
|
|
Given that <filename>builder.sh</filename> is a plain file, it has no .drv
|
|
associated with it. The store path is computed based on the filename and
|
|
on the hash of its contents. Store paths are covered in detail in <link
|
|
linkend="nix-store-paths">a later pill</link>.
|
|
</para>
|
|
</section>
|
|
<section>
|
|
<title>Packaging a simple C program</title>
|
|
<para>
|
|
Start off by writing a simple C program called <filename>simple.c</filename>:
|
|
|
|
<programlisting><xi:include href="./07/simple.c.txt" parse="text" /></programlisting>
|
|
|
|
And its <filename>simple_builder.sh</filename>:
|
|
|
|
<programlisting><xi:include href="./07/simple_builder.sh.txt" parse="text" /></programlisting>
|
|
|
|
Don't worry too much about where those variables come from yet; let's
|
|
write the derivation and build it:
|
|
|
|
<xi:include href="./07/c-program-derivation.xml" />
|
|
|
|
Now you can run
|
|
<filename>/nix/store/ni66p4jfqksbmsl616llx3fbs1d232d4-simple/simple</filename>
|
|
in your shell.
|
|
</para>
|
|
</section>
|
|
<section>
|
|
<title>Explanation</title>
|
|
<para>
|
|
We added two new attributes to the derivation call, <varname>gcc</varname>
|
|
and <varname>coreutils</varname>. In <code>gcc = gcc;</code>, the name on
|
|
the left is the name in the derivation set, and the name on the right
|
|
refers to the gcc derivation from nixpkgs. The same applies for coreutils.
|
|
</para>
|
|
<para>
|
|
We also added the <varname>src</varname> attribute, nothing magical — it's
|
|
just a name, to which the path <filename>./simple.c</filename> is
|
|
assigned. Like <filename>simple-builder.sh</filename>,
|
|
<filename>simple.c</filename> will be added to the store.
|
|
</para>
|
|
<para>
|
|
The trick: every attribute in the set passed to
|
|
<function>derivation</function> will be converted to a string and passed
|
|
to the builder as an environment variable. This is how the builder gains
|
|
access to <application>coreutils</application> and
|
|
<application>gcc</application>: when converted to strings, the derivations
|
|
evaluate to their output paths, and appending <literal>/bin</literal> to
|
|
these leads us to their binaries.
|
|
</para>
|
|
<para>
|
|
The same goes for the <varname>src</varname> variable. <envar>$src</envar>
|
|
is the path to <filename>simple.c</filename> in the nix store. As an
|
|
exercise, pretty print the .drv file. You'll see
|
|
<filename>simple_builder.sh</filename> and <filename>simple.c</filename>
|
|
listed in the input derivations, along with
|
|
<application>bash</application>, <application>gcc</application> and
|
|
<application>coreutils</application> .drv files. The newly added
|
|
environment variables described above will also appear.
|
|
</para>
|
|
<para>
|
|
In <filename>simple_builder.sh</filename> we set the <envar>PATH</envar>
|
|
for <application>gcc</application> and
|
|
<application>coreutils</application> binaries, so that our build script
|
|
can find the necessary utilities like <application>mkdir</application> and
|
|
<application>gcc</application>.
|
|
</para>
|
|
<para>
|
|
We then create <envar>$out</envar> as a directory and place the binary
|
|
inside it. Note that <application>gcc</application> is found via the
|
|
<envar>PATH</envar> environment variable, but it could equivalently be
|
|
referenced explicitly using <code>$gcc/bin/gcc</code>.
|
|
</para>
|
|
</section>
|
|
<section>
|
|
<title>Enough of <literal>nix repl</literal></title>
|
|
<para>
|
|
Drop out of <application>nix repl</application> and write a file
|
|
<filename>simple.nix</filename>:
|
|
|
|
<programlisting><xi:include href="./07/simple.txt" parse="text" /></programlisting>
|
|
|
|
Now you can build it with <command>nix-build simple.nix</command>. This
|
|
will create a symlink <filename>result</filename> in the current
|
|
directory, pointing to the out path of the derivation.
|
|
</para>
|
|
<para>
|
|
<application>nix-build</application> does two jobs:
|
|
<itemizedlist>
|
|
<listitem><para>
|
|
<link xlink:href="http://nixos.org/nix/manual/#sec-nix-instantiate">
|
|
<application>nix-instantiate</application>
|
|
</link>: parse and evaluate <filename>simple.nix</filename> and return
|
|
the .drv file corresponding to the parsed derivation set
|
|
</para></listitem>
|
|
<listitem><para>
|
|
<link xlink:href="http://nixos.org/nix/manual/#rsec-nix-store-realise">
|
|
<command>nix-store -r</command>
|
|
</link>: realise the .drv file, which actually builds it.
|
|
</para></listitem>
|
|
</itemizedlist>
|
|
Finally, it creates the symlink.
|
|
</para>
|
|
<para>
|
|
In the first line of <filename>simple.nix</filename>, we have an
|
|
<function>import</function> function call nested in a <code>with</code>
|
|
statement. Recall that <function>import</function> accepts one argument, a
|
|
nix file to load. In this case, the contents of the file evaluated to a
|
|
function.
|
|
</para>
|
|
<para>
|
|
Afterwards, we call the function with the empty set. We saw this already
|
|
in <link linkend="functions-and-imports">the fifth pill</link>. To
|
|
reiterate: <code>import <nixpkgs> {}</code> is calling two functions,
|
|
not one. Reading it as <code>(import <nixpkgs>) {}</code> makes this
|
|
clearer.
|
|
</para>
|
|
<para>
|
|
The value returned by the nixpkgs function is a set. More specifically,
|
|
it's a set of derivations. Using the <code>with</code> expression we bring
|
|
them into scope. This is the same as what <command>:l</command> does in
|
|
<application>nix repl</application>, so we can easily access derivations
|
|
such as <varname>bash</varname>, <varname>gcc</varname>, and
|
|
<varname>coreutils</varname>.
|
|
</para>
|
|
<para>
|
|
Then we meet the
|
|
<link xlink:href="https://nixos.org/nix/manual/#idm140737318075200"><code>inherit</code> keyword</link>.
|
|
<code>inherit foo;</code> is equivalent to <code>foo = foo;</code>;
|
|
<code>inherit foo bar;</code> is equivalent to <code>foo = foo; bar = bar;</code>.
|
|
</para>
|
|
<para>
|
|
This syntax only makes sense inside sets. There's no magic involved, it's
|
|
simply a convenience to avoid repeating the same name for both the
|
|
attribute name and the value in scope.
|
|
</para>
|
|
</section>
|
|
<section>
|
|
<title>Next pill</title>
|
|
<para>
|
|
We will generalize the builder. You may have noticed that we wrote two
|
|
separate <filename>builder.sh</filename> scripts in this post. We would
|
|
like to have a generic builder script instead, especially since each build
|
|
script goes in the nix store: a bit of a waste.
|
|
</para>
|
|
<para>
|
|
<emphasis>Is it really that hard to package stuff in Nix? No</emphasis>,
|
|
here we're studying the fundamentals of Nix.
|
|
</para>
|
|
</section>
|
|
</chapter>
|