mirror of
https://github.com/NixOS/nix-pills
synced 2024-09-19 04:00:13 -04:00
Transcribe pill 7
This commit is contained in:
parent
40ec99b1de
commit
d20e2c1746
|
@ -4,5 +4,324 @@
|
|||
version="5.0"
|
||||
xml:id="working-derivation">
|
||||
|
||||
<title>working derivation</title>
|
||||
<title>working derivation</title>
|
||||
|
||||
<section>
|
||||
<title>Introduction</title>
|
||||
<para>
|
||||
Welcome to the seventh nix pill. In the previous
|
||||
<link linkend="our-first-derivation">sixth pill</link> we introduced the
|
||||
notion of derivation in the Nix language — how to define a raw derivation
|
||||
and how to (try to) build it.
|
||||
</para>
|
||||
<para>
|
||||
In this post we continue along the path, by creating a derivation that
|
||||
actually builds something. Then, we try to package a real program: we
|
||||
compile a simple C file and create a derivation out of it, given a blessed
|
||||
toolchain.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
I remind you how to enter the Nix environment:
|
||||
<command>source ~/.nix-profile/etc/profile.d/nix.sh</command>
|
||||
</para>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<title>Using a script as a builder</title>
|
||||
|
||||
<para>
|
||||
What's the easiest way to run a sequence of commands for building
|
||||
something? A bash script. We write a custom bash script, and we want it to
|
||||
be our builder. Given a <filename>builder.sh</filename>, we want the
|
||||
derivation to run <command>bash builder.sh</command>.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
We don't use hash bangs in <filename>builder.sh</filename>, because at the
|
||||
time we are writing it we do not know the path to
|
||||
<application>bash</application> in the nix store. Yes, even bash is in the
|
||||
nix store, everything is there.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
We don't even use <application>/usr/bin/env</application>, because then we
|
||||
lose the cool stateless property of Nix. Not to mention that
|
||||
<envar>PATH</envar> gets cleared when building, so it wouldn't find
|
||||
<application>bash</application> anyway.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
In summary, we want the builder to be <application>bash</application>, and
|
||||
pass it an argument, <literal>builder.sh</literal>. Turns out the
|
||||
<function>derivation</function> function accepts an optional
|
||||
<parameter>args</parameter> attribute which is used to pass arguments to
|
||||
the builder executable.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
First of all, let's write our <filename>builder.sh</filename> in the
|
||||
current directory:
|
||||
|
||||
<programlisting><xi:include href="./07/builder.sh.txt" parse="text" /></programlisting>
|
||||
|
||||
As we covered in the previous pill, Nix computes the output path of the
|
||||
derivation. The resulting <literal>.drv</literal> file contains a list of
|
||||
environment variables passed to the builder. One of these is
|
||||
<varname>$out</varname>.
|
||||
</para>
|
||||
<para>
|
||||
What we have to do is create something in the path
|
||||
<varname>$out</varname>, be it a file or a directory. In this case we are
|
||||
creating a file.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
In addition, we print out the environment variables during the build
|
||||
process. We cannot use <application>env</application> for this, because
|
||||
<application>env</application> is part of
|
||||
<application>coreutils</application> and we don't have a dependency to it
|
||||
yet. We only have <application>bash</application> for now.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
Like for coreutils in the previous pill, we get a blessed bash for free
|
||||
from our magic nixpkgs stuff:
|
||||
|
||||
<xi:include href="./07/bash.xml" />
|
||||
|
||||
So with the usual trick, we can refer to
|
||||
<application>bin/bash</application> and create our derivation:
|
||||
|
||||
<xi:include href="./07/simple-derivation.xml" />
|
||||
|
||||
We did it! The contents of
|
||||
<filename>/nix/store/w024zci0x1hh1wj6gjq0jagkc1sgrf5r-<emphasis>foo</emphasis></filename>
|
||||
is really foo. We've built our first derivation.
|
||||
</para>
|
||||
<para>
|
||||
Note that we used <code>./builder.sh</code> and not
|
||||
<code>"./builder.sh"</code>. This way, it is parsed as a path, and Nix
|
||||
performs some magic which we will cover later. Try using the string
|
||||
version and you will find that it cannot find
|
||||
<filename>builder.sh</filename>. This is because it tries to find it
|
||||
relative to the temporary build directory.
|
||||
</para>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<title>The builder environment</title>
|
||||
<para>
|
||||
Let's inspect those environment variables printed during the build process.
|
||||
<itemizedlist>
|
||||
<listitem><para>
|
||||
<envar>$HOME</envar> is not your home directory, and
|
||||
<filename>/homeless-shelter</filename> doesn't exist at all. We force
|
||||
packages not to depend on <envar>$HOME</envar> during the build
|
||||
process.
|
||||
</para></listitem>
|
||||
<listitem><para>
|
||||
<envar>$PATH</envar> plays the same game as <envar>$HOME</envar>
|
||||
</para></listitem>
|
||||
<listitem><para>
|
||||
<envar>$NIX_BUILD_CORES</envar> and <envar>$NIX_STORE</envar> are
|
||||
<link xlink:href="http://nixos.org/nix/manual/#sec-conf-file">nix
|
||||
configuration options</link>
|
||||
</para></listitem>
|
||||
<listitem><para>
|
||||
<envar>$PWD</envar> and <envar>$TMP</envar> clearly show that nix
|
||||
created a temporary build directory
|
||||
</para></listitem>
|
||||
<listitem><para>
|
||||
Then <envar>$builder</envar>, <envar>$name</envar>,
|
||||
<envar>$out</envar>, and <envar>$system</envar> are variables set due
|
||||
to the .drv file's contents.
|
||||
</para></listitem>
|
||||
</itemizedlist>
|
||||
</para>
|
||||
<para>
|
||||
And that's how we were able to use <envar>$out</envar> in our derivation
|
||||
and put stuff in it. It's like Nix reserved a slot in the nix store for
|
||||
us, and we must fill it.
|
||||
</para>
|
||||
<para>
|
||||
<!-- Not really sure what this paragraph is meant to mean, particularly
|
||||
wrt the insistence that this is not make's DESTDIR. I've left it as is but
|
||||
it should probably be written more clearly. -->
|
||||
In terms of autotools, <envar>$out</envar> will be the
|
||||
<option>--prefix</option> path. Yes, not the make
|
||||
<option>DESTDIR</option>, but the <option>--prefix</option>. That's the
|
||||
essence of stateless packaging. You don't install the package in a global
|
||||
common path under <filename>/</filename>, you install it in a local
|
||||
isolated path under your nix store slot.
|
||||
</para>
|
||||
</section>
|
||||
<section>
|
||||
<title>The .drv contents</title>
|
||||
<para>
|
||||
We added something else to the derivation this time: the args attribute.
|
||||
Let's see how this changed the .drv compared to the previous pill:
|
||||
<!-- pp-aterm is no longer in nixpkgs, see nixos/nixpkgs#25371. This bit should be changed to reflect this. -->
|
||||
<xi:include href="./07/foo.drv.xml" />
|
||||
|
||||
Much like the usual .drv, except that there's a list of arguments in there
|
||||
passed to the builder (<application>bash</application>) with
|
||||
<filename>builder.sh</filename>… In the nix store..? Nix automatically
|
||||
copies files or directories needed for the build into the store to ensure
|
||||
that they are not changed during the build process and that the deployment
|
||||
is stateless and independent of the building machine.
|
||||
<filename>builder.sh</filename> is not only in the arguments passed to the
|
||||
builder, it's also in the input derivations.
|
||||
</para>
|
||||
<para>
|
||||
Given that <filename>builder.sh</filename> is a plain file, it has no .drv
|
||||
associated with it. The store path is computed based on the filename and
|
||||
on the hash of its contents. Store paths are covered in detail in <link
|
||||
linkend="nix-store-paths">a later pill</link>.
|
||||
</para>
|
||||
</section>
|
||||
<section>
|
||||
<title>Packaging a simple C program</title>
|
||||
<para>
|
||||
Start off by writing a simple C program:
|
||||
|
||||
<programlisting><xi:include href="./07/simple.c.txt" parse="text" /></programlisting>
|
||||
|
||||
And its <filename>simple_builder.sh</filename>:
|
||||
|
||||
<programlisting><xi:include href="./07/simple_builder.sh.txt" parse="text" /></programlisting>
|
||||
|
||||
Don't worry too much about where those variables come from yet; let's
|
||||
write the derivation and build it:
|
||||
|
||||
<xi:include href="./07/c-program-derivation.xml" />
|
||||
|
||||
Now you can run
|
||||
<filename>/nix/store/ni66p4jfqksbmsl616llx3fbs1d232d4-simple/simple</filename>
|
||||
in your shell.
|
||||
</para>
|
||||
</section>
|
||||
<section>
|
||||
<title>Explanation</title>
|
||||
<para>
|
||||
We added two new attributes to the derivation call, <varname>gcc</varname>
|
||||
and <varname>coreutils</varname>. In <code>gcc = gcc;</code>, the name on
|
||||
the left is the name in the derivation set, and the name on the right
|
||||
refers to the gcc derivation from nixpkgs. The same applies for coreutils.
|
||||
</para>
|
||||
<para>
|
||||
We also added the <varname>src</varname> attribute, nothing magical — it's
|
||||
just a name, to which the path <filename>./simple.c</filename> is
|
||||
assigned. Like <filename>simple-builder.sh</filename>,
|
||||
<filename>simple.c</filename> will be added to the store.
|
||||
</para>
|
||||
<para>
|
||||
The trick: every attribute in the set passed to
|
||||
<function>derivation</function> will be converted to a string and passed
|
||||
to the builder as an environment variable. This is how the builder gains
|
||||
access to <application>coreutils</application> and
|
||||
<application>gcc</application>: when converted to strings, the derivations
|
||||
evaluate to their output paths, and appending <literal>/bin</literal> to
|
||||
these leads us to their binaries.
|
||||
</para>
|
||||
<para>
|
||||
The same goes for the <varname>src</varname> variable. <envar>$src</envar>
|
||||
is the path to <filename>simple.c</filename> in the nix store. As an
|
||||
exercise, pretty print the .drv file. You'll see
|
||||
<filename>simple_builder.sh</filename> and <filename>simple.c</filename>
|
||||
listed in the input derivations, along with
|
||||
<application>bash</application>, <application>gcc</application> and
|
||||
<application>coreutils</application> .drv files. The newly added
|
||||
environment variables described above will also appear.
|
||||
</para>
|
||||
<para>
|
||||
In <filename>simple_builder.sh</filename> we set the <envar>PATH</envar>
|
||||
for <application>gcc</application> and
|
||||
<application>coreutils</application> binaries, so that our build script
|
||||
can find the necessary utilities like <application>mkdir</application> and
|
||||
<application>gcc</application>.
|
||||
</para>
|
||||
<para>
|
||||
We then create <envar>$out</envar> as a directory and place the binary
|
||||
inside it. Note that <application>gcc</application> is found via the
|
||||
<envar>PATH</envar> environment variable, but it could equivalently be
|
||||
referenced explicitly using <code>$gcc/bin/gcc</code>.
|
||||
</para>
|
||||
</section>
|
||||
<section>
|
||||
<title>Enough of nix-repl</title>
|
||||
<para>
|
||||
Drop out of <application>nix-repl</application> and write a file
|
||||
<filename>simple.nix</filename>:
|
||||
|
||||
<programlisting><xi:include href="./07/simple.txt" parse="text" /></programlisting>
|
||||
|
||||
Now you can build it with <command>nix-build simple.nix</command>. This
|
||||
will create a symlink <filename>result</filename> in the current
|
||||
directory, pointing to the out path of the derivation.
|
||||
</para>
|
||||
<para>
|
||||
<application>nix-build</application> does two jobs:
|
||||
<itemizedlist>
|
||||
<listitem><para>
|
||||
<link xlink:href="http://nixos.org/nix/manual/#sec-nix-instantiate">
|
||||
<application>nix-instantiate</application>
|
||||
</link>: parse and evaluate <filename>simple.nix</filename> and return
|
||||
the .drv file corresponding to the parsed derivation set
|
||||
</para></listitem>
|
||||
<listitem><para>
|
||||
<link xlink:href="http://nixos.org/nix/manual/#rsec-nix-store-realise">
|
||||
<command>nix-store -r</command>
|
||||
</link>: realise the .drv file, which actually builds it.
|
||||
</para></listitem>
|
||||
</itemizedlist>
|
||||
Finally, it creates the symlink.
|
||||
</para>
|
||||
<para>
|
||||
In the first line of <filename>simple.nix</filename>, we have an
|
||||
<function>import</function> function call nested in a <code>with</code>
|
||||
statement. Recall that <function>import</function> accepts one argument, a
|
||||
nix file to load. In this case, the contents of the file evaluated to a
|
||||
function.
|
||||
</para>
|
||||
<para>
|
||||
Afterwards, we call the function with the empty set. We saw this already
|
||||
in <link linkend="functions-and-imports">the fifth pill</link>. To
|
||||
reiterate: <code>import <nixpkgs> {}</code> is calling two functions,
|
||||
not one. Reading it as <code>(import <nixpkgs> {}</code> makes this
|
||||
clearer.
|
||||
</para>
|
||||
<para>
|
||||
The value returned by the nixpkgs function is a set. More specifically,
|
||||
it's a set of derivations. Using the <code>with</code> expression we bring
|
||||
them into scope. This is the same as what <command>:l</command> does in
|
||||
<application>nix-repl</application>, so we can easily access derivations
|
||||
such as <varname>bash</varname>, <varname>gcc</varname>, and
|
||||
<varname>coreutils</varname>.
|
||||
</para>
|
||||
<para>
|
||||
Then we meet the
|
||||
<link xlink:href="https://nixos.org/nix/manual/#idm140737318075200"><code>inherit</code> keyword</link>.
|
||||
<code>inherit foo;</code> is equivalent to <code>foo = foo;</code>;
|
||||
<code>inherit foo bar;</code> is equivalent to <code>foo = foo; bar = bar;</code>.
|
||||
</para>
|
||||
<para>
|
||||
This syntax only makes sense inside sets. There's no magic involved, it's
|
||||
simply a convenience to avoid repeating the same name for both the
|
||||
attribute name and the value in scope.
|
||||
</para>
|
||||
</section>
|
||||
<section>
|
||||
<title>Next pill</title>
|
||||
<para>
|
||||
We will generalize the builder. You may have noticed that we wrote two
|
||||
separate <filename>builder.sh</filename> scripts in this post. We would
|
||||
like to have a generic builder script instead, especially since each build
|
||||
script goes in the nix store: a bit of a waste.
|
||||
</para>
|
||||
<para>
|
||||
<emphasis>Is it really that hard to package stuff in Nix? No</emphasis>,
|
||||
here we're studying the fundamentals of Nix.
|
||||
</para>
|
||||
</section>
|
||||
</chapter>
|
||||
|
|
5
pills/07/bash.xml
Normal file
5
pills/07/bash.xml
Normal file
|
@ -0,0 +1,5 @@
|
|||
<screen xmlns="http://docbook.org/ns/docbook"><prompt>nix-repl> </prompt><userinput>:l <nixpkgs></userinput>
|
||||
<computeroutput>Added 3950 variables.</computeroutput>
|
||||
<prompt>nix-repl> </prompt><userinput>"${bash}"</userinput>
|
||||
<computeroutput>"/nix/store/ihmkc7z2wqk3bbipfnlh0yjrlfkkgnv6-<emphasis>bash-4.2-p45</emphasis>"</computeroutput>
|
||||
</screen>
|
2
pills/07/builder.sh.txt
Normal file
2
pills/07/builder.sh.txt
Normal file
|
@ -0,0 +1,2 @@
|
|||
declare -xp
|
||||
echo foo > $out
|
7
pills/07/c-program-derivation.xml
Normal file
7
pills/07/c-program-derivation.xml
Normal file
|
@ -0,0 +1,7 @@
|
|||
<screen xmlns="http://docbook.org/ns/docbook"><prompt>nix-repl> </prompt><userinput>:l <nixpkgs></userinput>
|
||||
<prompt>nix-repl> </prompt><userinput>simple = derivation { name = "simple"; builder = "${bash}/bin/bash"; args = [ ./simple_builder.sh ]; gcc = gcc; coreutils = coreutils; src = ./simple.c; system = builtins.currentSystem;</userinput>
|
||||
<prompt>nix-repl> </prompt><userinput>:b simple</userinput>
|
||||
<computeroutput>this derivation produced the following outputs:
|
||||
|
||||
out -> /nix/store/ni66p4jfqksbmsl616llx3fbs1d232d4-simple
|
||||
</computeroutput></screen>
|
14
pills/07/foo.drv.xml
Normal file
14
pills/07/foo.drv.xml
Normal file
|
@ -0,0 +1,14 @@
|
|||
<screen xmlns="http://docbook.org/ns/docbook"><prompt>$ </prompt><userinput>pp-aterm -i /nix/store/g6jj1mjzq68i66rbqyb3gpx3k0x606af-<emphasis>foo.drv</emphasis></userinput>
|
||||
<computeroutput>Derive(
|
||||
[("out", "/nix/store/w024zci0x1hh1wj6gjq0jagkc1sgrf5r-foo", "", "")]
|
||||
, [("/nix/store/jdggv3q1sb15140qdx0apvyrps41m4lr-bash-4.2-p45.drv", ["out"])]
|
||||
, ["/nix/store/5d1i99yd1fy4wkyx85iz5bvh78j2j96r-builder.sh"]
|
||||
, "x86_64-linux"
|
||||
, "/nix/store/ihmkc7z2wqk3bbipfnlh0yjrlfkkgnv6-bash-4.2-p45/bin/bash"
|
||||
, ["/nix/store/5d1i99yd1fy4wkyx85iz5bvh78j2j96r-builder.sh"]
|
||||
, [ ("builder", "/nix/store/ihmkc7z2wqk3bbipfnlh0yjrlfkkgnv6-bash-4.2-p45/bin/bash")
|
||||
, ("name", "foo")
|
||||
, ("out", "/nix/store/w024zci0x1hh1wj6gjq0jagkc1sgrf5r-foo")
|
||||
, ("system", "x86_64-linux")
|
||||
]
|
||||
)</computeroutput></screen>
|
10
pills/07/simple-derivation.xml
Normal file
10
pills/07/simple-derivation.xml
Normal file
|
@ -0,0 +1,10 @@
|
|||
<screen xmlns="http://docbook.org/ns/docbook"><prompt>nix-repl> </prompt><userinput>d = derivation { name = "foo"; builder = "${bash}/bin/bash"; args = [ ./builder.sh ]; system = builtins.currentSystem; }</userinput>
|
||||
<prompt>nix-repl> </prompt><userinput>:b d</userinput>
|
||||
<computeroutput>these derivations will be built:
|
||||
/nix/store/ybnysdh5k6cjznhg4afjgbhr6czbwb4s-<emphasis>foo.drv</emphasis>
|
||||
building path(s) `/nix/store/72v14vk4li47n8sx3z2ibd802ihpqyvx-<emphasis>foo</emphasis>'
|
||||
these derivations will be built:
|
||||
/nix/store/ibwr68l3rjlx02kgz61dkkkrlpgljfxd-simple.drv
|
||||
[...]<!-- Not sure if this should be marked up somehow -->
|
||||
this derivation produced the following outputs:
|
||||
out -> /nix/store/w024zci0x1hh1wj6gjq0jagkc1sgrf5r-<emphasis>foo</emphasis></computeroutput></screen>
|
3
pills/07/simple.c.txt
Normal file
3
pills/07/simple.c.txt
Normal file
|
@ -0,0 +1,3 @@
|
|||
void main() {
|
||||
puts("Simple!");
|
||||
}
|
9
pills/07/simple.txt
Normal file
9
pills/07/simple.txt
Normal file
|
@ -0,0 +1,9 @@
|
|||
with (import <nixpkgs> {});
|
||||
derivation {
|
||||
name = "simple";
|
||||
builder = "${bash}/bin/bash";
|
||||
args = [ ./simple_builder.sh ];
|
||||
inherit gcc coreutils;
|
||||
src = ./simple.c;
|
||||
system = builtins.currentSystem;
|
||||
}
|
3
pills/07/simple_builder.sh.txt
Normal file
3
pills/07/simple_builder.sh.txt
Normal file
|
@ -0,0 +1,3 @@
|
|||
export PATH="$coreutils/bin:$gcc/bin"
|
||||
mkdir $out
|
||||
gcc -o $out/simple $src
|
Loading…
Reference in a new issue