mirror of
https://github.com/NixOS/nix-pills
synced 2024-09-19 04:00:13 -04:00
497 lines
16 KiB
XML
497 lines
16 KiB
XML
<chapter xmlns="http://docbook.org/ns/docbook"
|
|
xmlns:xlink="http://www.w3.org/1999/xlink"
|
|
xmlns:xi="http://www.w3.org/2001/XInclude" version="5.0"
|
|
xml:id="our-first-derivation">
|
|
|
|
<title>Our First Derivation</title>
|
|
|
|
<para>
|
|
Welcome to the sixth Nix pill. In the previous <link
|
|
linkend="functions-and-imports">fifth pill</link> we introduced functions
|
|
and imports. Functions and imports are very simple concepts that allow for
|
|
building complex abstractions and composition of modules to build a flexible
|
|
Nix system.
|
|
</para>
|
|
|
|
<para>
|
|
In this post we finally arrived to writing a derivation. Derivations are the
|
|
building blocks of a Nix system, from a file system view point. The Nix
|
|
language is used to describe such derivations.
|
|
</para>
|
|
|
|
<para>
|
|
I remind you how to enter the Nix environment: <code>source
|
|
~/.nix-profile/etc/profile.d/nix.sh</code>
|
|
</para>
|
|
|
|
|
|
<section>
|
|
<title>The derivation function</title>
|
|
|
|
<para>
|
|
The <link
|
|
xlink:href="https://nixos.org/manual/nix/stable/expressions/derivations.html">derivation
|
|
built-in function</link> is used to create derivations. I invite you to read
|
|
the link in the Nix manual about the derivation built-in. A derivation from
|
|
a Nix language view point is simply a set, with some attributes. Therefore
|
|
you can pass the derivation around with variables like anything else.
|
|
</para>
|
|
|
|
<para>
|
|
That's where the real power comes in.
|
|
</para>
|
|
|
|
<para>
|
|
The <code>derivation</code> function receives a set as first argument. This
|
|
set requires at least the following three attributes:
|
|
</para>
|
|
|
|
<itemizedlist mark='bullet'>
|
|
<listitem>
|
|
<para>
|
|
name: the name of the derivation. In the nix store the format is
|
|
hash-name, that's the name.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
system: is the name of the system in which the derivation can be built.
|
|
For example, x86_64-linux.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
builder: it is the binary program that builds the derivation.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>
|
|
First of all, what's the name of our system as seen by nix?
|
|
</para>
|
|
|
|
<screen><xi:include href="./06/current-system.txt" parse="text" /></screen>
|
|
|
|
<para>
|
|
Let's try to fake the name of the system:
|
|
</para>
|
|
|
|
<screen><xi:include href="./06/fake-system.txt" parse="text" /></screen>
|
|
|
|
<para>
|
|
Oh oh, what's that? Did it build the derivation? No it didn't, but it
|
|
<emphasis role="strong">did create the .drv file</emphasis>. <literal>nix repl</literal> does
|
|
not build derivations unless you tell to do so.
|
|
</para>
|
|
|
|
</section>
|
|
|
|
<section>
|
|
<title>Digression about .drv files</title>
|
|
|
|
<para>
|
|
What's that <filename>.drv</filename> file? It is the specification of how
|
|
to build the derivation, without all the Nix language fuzz.
|
|
</para>
|
|
|
|
<para>
|
|
Before continuing, some analogies with the C language:
|
|
</para>
|
|
|
|
<itemizedlist mark='bullet'>
|
|
<listitem>
|
|
<para>
|
|
<filename>.nix</filename> files are like <filename>.c</filename> files
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
<filename>.drv</filename> files are intermediate files like
|
|
<filename>.o</filename> files. The <filename>.drv</filename> describes
|
|
how to build a derivation, it's the bare minimum information.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
out paths are then the product of the build
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>
|
|
Both drv paths and out paths are stored in the nix store as you can see.
|
|
</para>
|
|
|
|
<para>
|
|
What's in that <filename>.drv</filename> file? You can read it, but it's
|
|
better to pretty print it:
|
|
</para>
|
|
|
|
<xi:include href="./06/show-derivation.xml" />
|
|
|
|
|
|
<para>
|
|
Ok we can see there's an out path, but it does not exist yet. We never told
|
|
Nix to build it, but we know beforehand where the build output will be. Why?
|
|
</para>
|
|
|
|
<para>
|
|
Think, if Nix ever built the derivation just because we accessed it in Nix,
|
|
we would have to wait a long time if it was, say, Firefox. That's why Nix
|
|
let us know the path beforehand and keep evaluating the Nix expressions, but
|
|
it's still empty because no build was ever made.
|
|
</para>
|
|
|
|
<para>
|
|
<emphasis role="underline">Important</emphasis>: the hash of the out path is
|
|
based solely on the input derivations in the current version of Nix, not on
|
|
the contents of the build product. It's possible however to have <link
|
|
xlink:href="https://en.wikipedia.org/wiki/Content-addressable_storage">content-addressable</link>
|
|
derivations for e.g. tarballs as we'll see later on.
|
|
</para>
|
|
|
|
<para>
|
|
Many things are empty in that <filename>.drv</filename>, however I'll write a
|
|
summary of the <link
|
|
xlink:href="http://nixos.org/~eelco/pubs/phd-thesis.pdf">.drv format</link>
|
|
for you:
|
|
</para>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>
|
|
The output paths (there can be multiple ones). By default nix creates one
|
|
out path called "out".
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
The list of input derivations. It's empty because we are not referring
|
|
to any other derivation. Otherwise, there would be a list of other .drv
|
|
files.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
The system and the builder executable (yes, it's a fake one).
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Then a list of environment variables passed to the builder.
|
|
</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<para>
|
|
That's it, the minimum necessary information to build our derivation.
|
|
</para>
|
|
|
|
<para>
|
|
<emphasis role="underline">Important note</emphasis>: the environment
|
|
variables passed to the builder are just those you see in the .drv plus some
|
|
other Nix related configuration (number of cores, temp dir, ...). The
|
|
builder will not inherit any variable from your running shell, otherwise
|
|
builds would suffer from <link
|
|
xlink:href="https://wiki.debian.org/ReproducibleBuilds">non-determinism</link>.
|
|
</para>
|
|
|
|
<para>
|
|
Back to our fake derivation.
|
|
</para>
|
|
|
|
<para>
|
|
Let's build our really fake derivation:
|
|
</para>
|
|
|
|
<screen><xi:include href="./06/build-derivation.txt" parse="text" /></screen>
|
|
|
|
<para>
|
|
The <code>:b</code> is a <literal>nix repl</literal> specific command to build a derivation.
|
|
You can see more commands with <code>:?</code> . So in the output you can
|
|
see that it takes the <filename>.drv</filename> as information on how to
|
|
build the derivation. Then it says it's trying to produce our out path.
|
|
Finally the error we were waiting for: that derivation can't be built on our
|
|
system.
|
|
</para>
|
|
|
|
<para>
|
|
We're doing the build inside <literal>nix repl</literal>, but what if we don't want to use
|
|
<literal>nix repl</literal>? You can <emphasis role="strong">realise</emphasis> a
|
|
<filename>.drv</filename> with:
|
|
</para>
|
|
|
|
<screen><xi:include href="./06/realise-derivation.txt" parse="text"
|
|
/></screen>
|
|
|
|
<para>
|
|
You will get the same output as before.
|
|
</para>
|
|
|
|
<para>
|
|
Let's fix the system attribute:
|
|
</para>
|
|
|
|
<screen><xi:include href="./06/fix-attribute.txt" parse="text" /></screen>
|
|
|
|
<para>
|
|
A step forward: of course, that <code>mybuilder</code> executable does not
|
|
really exist. Stop for a moment.
|
|
</para>
|
|
|
|
|
|
</section>
|
|
<section>
|
|
<title>What's in a derivation set</title>
|
|
|
|
<para>
|
|
It is useful to start by inspecting the return value from the derivation function.
|
|
In this case, the returned value is a plain set:
|
|
</para>
|
|
|
|
<screen><xi:include href="./06/inspect-values.txt" parse="text" /></screen>
|
|
|
|
<para>
|
|
You can guess what <code>builtins.isAttrs</code> does, it returns true if
|
|
the argument is a set. While <code>builtins.attrNames</code> returns a list
|
|
of keys of the given set. Some kind of reflection, you might say.
|
|
</para>
|
|
|
|
<para>
|
|
Start from drvAttrs:
|
|
</para>
|
|
|
|
<screen><xi:include href="./06/drvattrs.txt" parse="text" /></screen>
|
|
|
|
<para>
|
|
That's basically the input we gave to the derivation function. Also
|
|
<code>d.name</code>, <code>d.system</code> and <code>d.builder</code>
|
|
attributes are straight the ones we gave as input.
|
|
</para>
|
|
|
|
<screen><xi:include href="./06/check-drvattrs.txt" parse="text" /></screen>
|
|
|
|
<para>
|
|
So out is just the derivation itself, it seems weird but the reason is that
|
|
we only have one output from the derivation. That's also the reason why
|
|
<code>d.all</code> is a singleton. We'll see multiple outputs later.
|
|
</para>
|
|
|
|
<para>
|
|
The <code>d.drvPath</code> is the path of the <filename>.drv</filename>
|
|
file: <filename>/nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-<emphasis
|
|
role="strong">myname.drv</emphasis></filename>.
|
|
</para>
|
|
|
|
<para>
|
|
Something interesting is the <code>type</code> attribute. It's
|
|
<code>"derivation"</code>. Nix does add a little of magic to sets with type
|
|
derivation, but not that much. To let you understand, you can create
|
|
yourself a set with that type, it's a simple set:
|
|
</para>
|
|
|
|
<screen><xi:include href="./06/type-derivation.txt" parse="text" /></screen>
|
|
|
|
<para>
|
|
Of course it has no other information, so Nix doesn't know what to say :-)
|
|
But you get it, the <code>type = "derivation"</code> is just a convention
|
|
for Nix and for us to understand the set is a derivation.
|
|
</para>
|
|
|
|
<para>
|
|
When writing packages, we are interested in the outputs. The other metadata
|
|
is needed for Nix to know how to create the drv path and the out path.
|
|
</para>
|
|
|
|
<para>
|
|
The outPath attribute is the build path in the nix store:
|
|
<filename>/nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-<emphasis
|
|
role="strong">myname</emphasis></filename>.
|
|
</para>
|
|
|
|
|
|
</section>
|
|
<section>
|
|
<title>Referring to other derivations</title>
|
|
|
|
<para>
|
|
Just like dependencies in other package managers, how do we refer to other
|
|
packages? How do we refer to other derivations in terms of files on the
|
|
disk? We use the outPath. The outPath tells where the files are of that
|
|
derivation. To make it more convenient, Nix is able to do a conversion from
|
|
a derivation set to a string.
|
|
</para>
|
|
|
|
<screen><xi:include href="./06/outpath.txt" parse="text" /></screen>
|
|
|
|
<para>
|
|
Nix does the "set to string conversion" as long as there is the
|
|
<code>outPath</code> attribute (much like a toString method in other
|
|
languages):
|
|
</para>
|
|
|
|
<screen><xi:include href="./06/tostring.txt" parse="text" /></screen>
|
|
|
|
<para>
|
|
Say we want to use binaries from coreutils (ignore the nixpkgs etc.):
|
|
</para>
|
|
|
|
<screen><xi:include href="./06/coreutils.txt" parse="text" /></screen>
|
|
|
|
<para>
|
|
Apart the nixpkgs stuff, just think we added to the scope a series of
|
|
variables. One of them is coreutils. It is the derivation of the coreutils
|
|
package you all know of from other Linux distributions. It contains basic
|
|
binaries for GNU/Linux systems (you may have multiple derivations of
|
|
coreutils in the nix store, no worries):
|
|
</para>
|
|
|
|
|
|
<screen><xi:include href="./06/list-coreutils.txt" parse="text" /></screen>
|
|
|
|
<para>
|
|
I remind you, inside strings it's possible to interpolate Nix expressions
|
|
with <code>${...}</code>:
|
|
</para>
|
|
|
|
|
|
<screen><xi:include href="./06/interpolate.txt" parse="text" /></screen>
|
|
|
|
<para>
|
|
That's very convenient, because then we could refer to e.g. the bin/true
|
|
binary like this:
|
|
</para>
|
|
|
|
|
|
<screen><xi:include href="./06/reference.txt" parse="text" /></screen>
|
|
|
|
</section>
|
|
<section>
|
|
<title>An almost working derivation</title>
|
|
|
|
<para>
|
|
In the previous attempt we used a fake builder, <code>mybuilder</code> which
|
|
obviously does not exist. But we can use for example bin/true, which always
|
|
exits with 0 (success).
|
|
</para>
|
|
|
|
<screen><xi:include href="./06/test-build.txt" parse="text" /></screen>
|
|
|
|
<para>
|
|
Another step forward, it executed the builder (bin/true), but the builder
|
|
did not create the out path of course, it just exited with 0.
|
|
</para>
|
|
|
|
|
|
<para>
|
|
<emphasis role="underline">Obvious note</emphasis>: everytime we change the
|
|
derivation, a new hash is created.
|
|
</para>
|
|
|
|
<para>
|
|
Let's examine the new <filename>.drv</filename> now that we referred to
|
|
another derivation:
|
|
</para>
|
|
|
|
<xi:include href="./06/examine-build.xml" />
|
|
|
|
<para>
|
|
Aha! Nix added a dependency to our myname.drv, it's the coreutils.drv.
|
|
Before doing our build, Nix should build the coreutils.drv. But since
|
|
coreutils is already in our nix store, no build is needed, it's already
|
|
there with out path
|
|
<filename>/nix/store/qrxs7sabhqcr3j9ai0j0cp58zfnny0jz-<emphasis
|
|
role="strong">coreutils-8.29</emphasis></filename>.
|
|
</para>
|
|
|
|
</section>
|
|
<section>
|
|
<title>When is the derivation built</title>
|
|
|
|
<para>
|
|
Nix does not build derivations <emphasis role="strong">during
|
|
evaluation</emphasis> of Nix expressions. In fact, that's why we have to do
|
|
":b drv" in <literal>nix repl</literal>, or use nix-store -r in the first place.
|
|
</para>
|
|
|
|
<para>
|
|
An important separation is made in Nix:
|
|
</para>
|
|
|
|
<itemizedlist mark='bullet'>
|
|
<listitem>
|
|
<para>
|
|
<emphasis role="strong">Instantiate/Evaluation time</emphasis>: the Nix
|
|
expression is parsed, interpreted and finally returns a derivation set.
|
|
During evaluation, you can refer to other derivations because Nix will
|
|
create .drv files and we will know out paths beforehand. This is
|
|
achieved with <link
|
|
xlink:href="https://nixos.org/manual/nix/stable/command-ref/nix-instantiate.html">nix-instantiate</link>.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
<emphasis role="strong">Realise/Build time</emphasis>: the .drv from the
|
|
derivation set is built, first building .drv inputs (build
|
|
dependencies). This is achieved with <link
|
|
xlink:href="https://nixos.org/manual/nix/stable/command-ref/nix-store.html#operation---realise">nix-store
|
|
-r</link>.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>
|
|
Think of it as of compile time and link time like with C/C++ projects. You
|
|
first compile all source files to object files. Then link object files in a
|
|
single executable.
|
|
</para>
|
|
|
|
<para>
|
|
In Nix, first the Nix expression (usually in a .nix file) is compiled to
|
|
.drv, then each .drv is built and the product is installed in the relative
|
|
out paths.
|
|
</para>
|
|
|
|
</section>
|
|
<section>
|
|
<title>Conclusion</title>
|
|
|
|
<para>
|
|
Is it that complicated to create a package for Nix? No it's not.
|
|
</para>
|
|
|
|
<para>
|
|
We're walking through the fundamentals of Nix derivations, to understand how
|
|
they work, how they are represented. Packaging in Nix is certainly easier
|
|
than that, but we're not there yet in this post. More Nix pills are needed.
|
|
</para>
|
|
|
|
<para>
|
|
With the derivation function we provide a set of information on how to build
|
|
a package, and we get back the information about where the package was
|
|
built. Nix converts a set to a string when there's an outPath, that's very
|
|
convenient. With that, it's easy to refer to other derivations.
|
|
</para>
|
|
|
|
<para>
|
|
When Nix builds a derivation, it first creates a .drv file from a derivation
|
|
expression, and uses it to build the output. It does so recursively for all
|
|
the dependencies (inputs). It "executes" the .drv files like a machine. Not
|
|
much magic after all.
|
|
</para>
|
|
|
|
|
|
</section>
|
|
<section>
|
|
<title>Next pill</title>
|
|
|
|
<para>
|
|
...we will finally write our first <emphasis
|
|
role="strong">working</emphasis> derivation. Yes, this post is about "our
|
|
first derivation", but I never said it was a working one ;)
|
|
</para>
|
|
|
|
</section>
|
|
|
|
</chapter>
|