mirror of
https://github.com/NixOS/nix-pills
synced 2024-09-19 04:00:13 -04:00
181 lines
11 KiB
XML
181 lines
11 KiB
XML
<chapter xmlns="http://docbook.org/ns/docbook"
|
|
xmlns:xlink="http://www.w3.org/1999/xlink"
|
|
xmlns:xi="http://www.w3.org/2001/XInclude"
|
|
version="5.0"
|
|
xml:id="basic-dependencies-and-hooks">
|
|
|
|
<title>Basic Dependencies and Hooks</title>
|
|
|
|
<para>
|
|
Welcome to the 20th Nix pill.
|
|
In the previous <link linkend="fundamentals-of-stdenv">19th</link> pill we introduced Nixpkgs' stdenv, including <filename>setup.sh</filename> script, <filename>default-builder.sh</filename> helper script, and <literal>stdenv.mkDerivation</literal> builder.
|
|
We focused on how stdenv is put together, and how it's used, and a bit about the phases of <function>genericBuild</function>.
|
|
</para>
|
|
|
|
<para>
|
|
This time, we'll focus on the interaction of packages built with <literal>stdenv.mkDerivation</literal>.
|
|
Packages need to depend on each other, of course.
|
|
For this we have <varname>buildInputs</varname> and <varname>propagatedBuildInputs</varname> attributes.
|
|
We've also found that dependencies sometimes need to influence their dependents in ways the dependents can't or shouldn't predict.
|
|
For this we have <firstterm>setup hooks</firstterm> and <firstterm>env hooks</firstterm>.
|
|
Together, these 4 concepts support almost all build-time package interactions.
|
|
</para>
|
|
|
|
<note><para>
|
|
The complexity of the dependencies and hooks infrastructure has increased, over time, to support cross compilation.
|
|
Once you learn the core concepts, you will be able to understand the extra complexity.
|
|
As a starting point, you might want to refer to nixpkgs commit <link xlink:href="https://github.com/nixos/nixpkgs/tree/6675f0a52c0962042a1000c7f20e887d0d26ae25">6675f0a5</link>, the last version of stdenv without cross-compilation complexity.
|
|
</para></note>
|
|
|
|
<section>
|
|
<title>The <varname>buildInputs</varname> Attribute</title>
|
|
|
|
<para>
|
|
For the simplest dependencies where the current package directly needs another, we use the <varname>buildInputs</varname> attribute.
|
|
This is exactly the pattern in taught with our builder in <link linkend="generic-builders">Pill 8</link>.
|
|
To demo this, let's build GNU Hello, and then another package which provides a shell script that <command>exec</command>s it.
|
|
<screen><xi:include href="./20/two-hellos.nix" parse="text" /></screen>
|
|
</para>
|
|
|
|
<para>
|
|
Notice that the wrappedHello derivation finds the <command>hello</command> binary from the <envar>PATH</envar>.
|
|
This works because stdenv contains something like:
|
|
<screen><xi:include href="./20/build-inputs-0.bash" parse="text" /></screen>
|
|
where <function>findInputs</function> is defined like:
|
|
<screen><xi:include href="./20/build-inputs-1.bash" parse="text" /></screen>
|
|
then after this is run:
|
|
<screen><xi:include href="./20/build-inputs-2.bash" parse="text" /></screen>
|
|
where <function>addToEnv</function> is defined like:
|
|
<screen><xi:include href="./20/build-inputs-3.bash" parse="text" /></screen>
|
|
|
|
The <function>addToSearchPath</function> call adds <literal>$1/bin</literal> to <envar>_PATH</envar> if the former exists (code <link xlink:href="https://github.com/NixOS/nixpkgs/blob/6675f0a52c0962042a1000c7f20e887d0d26ae25/pkgs/stdenv/generic/setup.sh#L60-L73">here</link>).
|
|
Once all the packages in <varname>buildInputs</varname> have been processed, then content of <envar>_PATH</envar> is added to <envar>PATH</envar>, as follows:
|
|
<screen><xi:include href="./20/build-inputs-4.bash" parse="text" /></screen>
|
|
|
|
With the real <command>hello</command> on the <envar>PATH</envar>, the <function>installPhase</function> should hopefully make sense.
|
|
</para>
|
|
</section>
|
|
|
|
<section>
|
|
<title>The <varname>propagatedBuildInputs</varname> Attribute</title>
|
|
|
|
<para>
|
|
The <varname>buildInputs</varname> covers direct dependencies, but what about indirect dependencies where one package needs a second package which needs a third?
|
|
Nix itself handles this just fine, understanding various dependency <firstterm>closures</firstterm> as covered in previous builds.
|
|
But what about the conveniences that <varname>buildInputs</varname> provides, namely accumulating in <envar>pkgs</envar> environment variable and inclusion of <filename><replaceable>pkg</replaceable>/bin</filename> directories on the <envar>PATH</envar>?
|
|
For this, stdenv provides the <varname>propagatedBuildInputs</varname>:
|
|
<screen><xi:include href="./20/three-hellos.nix" parse="text" /></screen>
|
|
See how the intermediate package has a <varname>propagatedBuildInputs</varname> dependency, but the wrapper only needs a <varname>buildInputs</varname> dependency on the intermediary.
|
|
</para>
|
|
|
|
<para>
|
|
How does this work?
|
|
You might think we do something in Nix, but actually it's done not at eval time but at build time in bash.
|
|
let's look at part of the <function>fixupPhase</function> of stdenv:
|
|
|
|
<screen><xi:include href="./20/propagated-build-inputs-0.bash" parse="text" /></screen>
|
|
|
|
This dumps the propagated build inputs in a so-named file in <filename>$out/nix-support/</filename>.
|
|
Then, back in <function>findInputs</function> look at the lines at the bottom we elided before:
|
|
|
|
<screen><xi:include href="./20/propagated-build-inputs-1.bash" parse="text" /></screen>
|
|
|
|
See how <function>findInputs</function> is actually recursive, looking at the propagated build inputs of each dependency, and those dependencies' propagated build inputs, etc.
|
|
</para>
|
|
|
|
<para>
|
|
We actually simplified the <function>findInputs</function> call site from before; <envar>propagatedBuildInputs</envar> is also looped over in reality:
|
|
<screen><xi:include href="./20/propagated-build-inputs-2.bash" parse="text" /></screen>
|
|
This demonstrates an important point. For the <emphasis>current</emphasis> package alone, it doesn't matter whether a dependency is propagated or not.
|
|
It will be processed the same way: called with <function>findInputs</function> and <function>addToEnv</function>.
|
|
(The packages discovered by <function>findInputs</function>, which are also accumulated in <envar>pkgs</envar> and passed to <function>addToEnv</function>, are also the same in both cases.)
|
|
Downstream however, it certainly does matter because only the propagated immediate dependencies are put in the <filename>$out/nix-support/propagated-build-inputs</filename>.
|
|
</para>
|
|
</section>
|
|
|
|
<section>
|
|
<title>Setup Hooks</title>
|
|
|
|
<para>
|
|
As we mentioned above, sometimes dependencies need to influence the packages that use them in ways other than just <emphasis>being</emphasis> a dependency.
|
|
|
|
<footnote>
|
|
<para>
|
|
We can now be precise and consider what <function>addToEnv</function> does alone the minimal treatment of a dependency:
|
|
i.e. a package that is <emphasis>just</emphasis> a dependency would <emphasis>only</emphasis> have <function>addToEnv</function> applied to it.
|
|
</para>
|
|
</footnote>
|
|
|
|
<varname>propagatedBuildInputs</varname> can actually be seen as an example of this:
|
|
packages using that are effectively "injecting" those dependencies as extra <varname>buildInputs</varname> in their downstream dependents.
|
|
|
|
But in general, a dependency might affect the packages it depends on in arbitrary ways.
|
|
<emphasis>Arbitrary</emphasis> is the key word here.
|
|
We could teach <filename>setup.sh</filename> things about upstream packages like <filename><replaceable>pkg</replaceable>/nix-support/propagated-build-inputs</filename>, but not arbitrary interactions.
|
|
</para>
|
|
|
|
<para>
|
|
<firstterm>Setup hooks</firstterm> are the basic building block we have for this.
|
|
In nixpkgs, a "hook" is basically a bash callback, and a setup hook is no exception.
|
|
Let's look at the last part of <function>findInputs</function> we haven't covered:
|
|
<screen><xi:include href="./20/setup-hooks-0.bash" parse="text" /></screen>
|
|
If a package includes the path <filename><replaceable>pkg</replaceable>/nix-support/setup-hook</filename>, it will be sourced by any stdenv-based build including that as a dependency.
|
|
</para>
|
|
|
|
<para>
|
|
This is strictly more general than any of the other mechanisms introduced in this chapter.
|
|
For example, try writing a setup hook that has the same effect as a <emphasis>propagatedBuildInputs</emphasis> entry.
|
|
One can almost think of this as an escape hatch around Nix's normal isolation guarantees, and the principle that dependencies are immutable and inert.
|
|
We're not actually doing something unsafe or modifying dependencies, but we are allowing arbitrary ad-hoc behavior.
|
|
For this reason, setup-hooks should only be used as a last resort.
|
|
</para>
|
|
</section>
|
|
|
|
<section>
|
|
<title>Environment Hooks</title>
|
|
|
|
<para>
|
|
As a final convenience, we have environment hooks.
|
|
Recall in <link linkend="inputs-design-pattern">Pill 12</link> how we created <envar>NIX_CFLAGS_COMPILE</envar> for <literal>-I</literal> flags and <envar>NIX_LDFLAGS</envar> for <literal>-L</literal> flags, in a similar manner to how we prepared the <envar>PATH</envar>.
|
|
One point of ugliness was how anti-modular this was.
|
|
It makes sense to build the <envar>PATH</envar> in generic builder, because the <envar>PATH</envar> is used by the shell, and the generic builder is intrinsically tied to the shell.
|
|
But <literal>-I</literal> and <literal>-L</literal> flags are only relevant to the C compiler.
|
|
The stdenv isn't wedded to including a C compiler (though it does by default), and there are other compilers too which may take completely different flags.
|
|
</para>
|
|
|
|
<para>
|
|
As a first step, we can move that logic to a setup hook on the C compiler;
|
|
indeed that's just what we do in CC Wrapper.
|
|
<footnote>
|
|
<para>
|
|
It was called <link xlink:href="https://github.com/NixOS/nixpkgs/tree/6675f0a52c0962042a1000c7f20e887d0d26ae25/pkgs/build-support/gcc-wrapper">GCC Wrapper</link> in the version of nixpkgs suggested for following along in this pill; Darwin and Clang support hadn't yet motivated the rename.
|
|
</para>
|
|
</footnote>
|
|
But this pattern comes up fairly often, so somebody decided to add some helper support to reduce boilerplate.
|
|
</para>
|
|
|
|
<para>
|
|
The other half of <function>addToEnv</function> is:
|
|
<screen><xi:include href="./20/env-hooks-0.bash" parse="text" /></screen>
|
|
Functions listed in <varname>envHooks</varname> are applied to every package passed to <function>addToEnv</function>.
|
|
One can write a setup hook like:
|
|
<screen><xi:include href="./20/env-hooks-1.bash" parse="text" /></screen>
|
|
and if one dependency has that setup hook then all of them will be so <command>echo</command>ed.
|
|
Allowing dependencies to learn about their <emphasis>sibling</emphasis> dependencies is exactly what compilers need.
|
|
</para>
|
|
|
|
</section>
|
|
|
|
<section>
|
|
<title>Next pill...</title>
|
|
<para>
|
|
...I'm not sure!
|
|
We could talk about the additional dependency types and hooks which cross compilation necessitates, building on our knowledge here to cover stdenv as it works today.
|
|
We could talk about how nixpkgs is bootstrapped.
|
|
Or we could talk about how <varname>localSystem</varname> and <varname>crossSystem</varname> are elaborated into the <varname>buildPlatform</varname>, <varname>hostPlatform</varname>, and <varname>targetPlatform</varname> each bootstrapping stage receives.
|
|
Let us know which most interests you!
|
|
</para>
|
|
</section>
|
|
|
|
</chapter>
|