1
0
Fork 0
mirror of https://github.com/NixOS/nix.dev.git synced 2024-10-18 00:06:26 -04:00

Compare commits

...

62 commits

Author SHA1 Message Date
Valentin Gagarin 397de85817
Merge 7f237c263d into a2c8326dcf 2024-02-07 05:00:11 +01:00
Daniel Sidhion a2c8326dcf
Merge pull request #905 from NixOS/readme-deploy-test
add more preview instructions to README
2024-02-06 19:33:16 -08:00
Daniel Sidhion 1d327199cf
Merge pull request #900 from fricklerhandwerk/add-linktext
add linktext to summary
2024-02-06 19:31:07 -08:00
Valentin Gagarin 53cbd112d7
fix up language in the packaging tutorial (#887)
* fix up language in the packaging tutorial

- make many sentences shorter
- add links
- expand on finding things
- use more domain-specific headings
- add next steps


Co-authored-by: Henrik <i97henka@gmail.com>
Co-authored-by: Olivia Crain <olivia@olivia.dev>
Co-authored-by: Silvan Mosberger <github@infinisil.com>
2024-02-07 02:36:18 +01:00
Valentin Gagarin cafff4801c add more preview instructions to README 2024-02-07 01:03:36 +01:00
Valentin Gagarin 1f6a1219ae
actually fix mutable shortlinks (#904)
now tested with:

    nix build && nix-shell -p netlify-cli --run "cd result; netlify dev"
2024-02-06 23:55:59 +00:00
Valentin Gagarin 2720a4121e
fix mutable shortlinks (#903)
they were missing a response status code.

use 302 since the target URL is decidedly not permanent:
https://datatracker.ietf.org/doc/html/rfc9110#name-302-found
2024-02-06 21:42:51 +01:00
Valentin Gagarin 502a862077 add linktext to summary 2024-02-06 07:47:28 +01:00
Valentin Gagarin e0e2f95866
endorse CC-BY-SA for independent work (#894) 2024-02-06 05:03:01 +01:00
Daniel Sidhion e23ae33951
Merge pull request #896 from fricklerhandwerk/team-handbook
update docs team page
2024-02-05 18:24:20 -08:00
Silvan Mosberger ab49acd74a
Merge pull request #830 from fricklerhandwerk/nix-manual-shortlinks
add mutable shortlinks
2024-02-05 18:08:56 +01:00
Valentin Gagarin b6bdfa86f5
collapse unnecessary reference links (#898) 2024-02-05 12:42:53 +00:00
Valentin Gagarin 4be58f2eaf update docs team page
- add link to inaugural announcement
- the GitHub team is the most up to date in terms of membership
- fix link to meeting notes overview
2024-02-05 13:31:27 +01:00
Valentin Gagarin 22bedeaba0
front page: Who is Nix for? (#872)
* front page: Who is Nix for?

For at least two years there were ongoing debates about the target
audience of the Nix ecosystem. Nix maintainers did not manage to
converge on a statement [0]. The marketing team is explicitly focusing
on software developers [1], and the documentation team was supposed to
align with that from the very beginning [2].

What was missing so far was what we mean by "software developers".
Arguably, it encompasses a particular mind set about how to deal with
computers and enough time to learn things.

And while for documentation it matters most *what* the software is doing
rather than *who* it is made for (or by), the ecosystem is a community
effort, and people matter [3]. So far, we haven't really identified the
(eventual) boundaries of this community (at least I don't know of any
serious attempt), which also plays into the definition of a target
audience. Yet, in practice, not every contribution, not every question
or comment is treated equally, and this has reasons in our implicit
assumptions about who belongs.

This is an attempt to draw such a boundary that is not arbitrary, and
neither to narrow nor too wide. The goal is, as always, to help
Nix beginners set realistic expectations for their journey.

The list of occupations/interests who may benefit from Nix is based on
the 2022 [4] and 2023 [5] community surveys.

[0]: https://github.com/NixOS/nix/pull/7156
[1]: 76d42a052d/community/teams/marketing.tt (L89)
[2]: https://discourse.nixos.org/t/2022-06-15-documentation-team-meeting-notes-1/20004
[3]: https://discourse.nixos.org/t/zurich-23-05-zhf-hackathon-and-workshop-report/29093#ux-workshop-21
[4]: https://discourse.nixos.org/t/2022-nix-survey-results/18983
[5]: https://discourse.nixos.org/t/nix-community-survey-2023-results/33124

Co-authored-by: Daniel Sidhion <DanielSidhion@users.noreply.github.com>
2024-02-02 01:29:40 +01:00
Silvan Mosberger 54666bff02
Merge pull request #876 from NixOS/information-flow
update information flow diagram
2024-01-31 23:22:27 +01:00
Valentin Gagarin 06e89b89d4 update information flow diagram
we probably don't need it here in particular, but originally when the documentation team was new, it was useful to understand and communicate what's going on. it can probably stay here until we either find a better place or feel like it's in the way.

until then, a small update for clarity and consistency.
2024-01-31 23:19:34 +01:00
Silvan Mosberger a505a3b066
Merge pull request #871 from fricklerhandwerk/email-subscription
remove email subscription form
2024-01-31 22:48:25 +01:00
Silvan Mosberger 5bc6e4046b
Merge pull request #870 from MasonM/patch-1
Fix unquoted URL in "Nix language basics"
2024-01-31 22:32:31 +01:00
Valentin Gagarin c9ed694ea8 NixOS tutorials: add next steps 2024-01-31 02:39:45 +01:00
Valentin Gagarin 747c3c7060 reorder NixOS tutorials
it makes more sense to start with the most innocent ones, where
everything runs in a VM.
2024-01-31 02:39:45 +01:00
nbelakovski 596d9db532
Update packaging-existing-software.md (#888)
As it was written, I thought we'd have to do more work to supply fetchzip, but the next invokation of nix-build seems to call fetchzip without any definition errors so I assume the error is in the documentation.
2024-01-30 20:18:50 +01:00
Valentin Gagarin a295a4dae1
fix broken reference (#889) 2024-01-30 12:32:03 +00:00
Valentin Gagarin 41ad39d8c2
clean up the Reference side bar (#885)
- use more neutral phrasing
- add nix-community, a lot of stuff going on there
- put the Glossary on top
2024-01-29 05:54:05 +01:00
Valentin Gagarin a11222eaec
add the NixOS Foundation YouTube channel to the videos list (#884) 2024-01-29 05:53:41 +01:00
Valentin Gagarin 9c12897c8f
fix up language in python environment guide (#882) 2024-01-29 01:44:42 +00:00
Valentin Gagarin e6c1fb1b14
move automatic environments up, add next steps (#881)
automatically entering the shell is the next logical step, especially if
we have turned it on and off multiple times in a row.
2024-01-29 02:42:49 +01:00
Valentin Gagarin a1c683aabb
move shell.nix tutorial up (#879)
it doesn't make sense that we continue with reproducibility becuase we
haven't seen lookup paths yet.
2024-01-29 02:42:40 +01:00
Valentin Gagarin cfa14f780a
update the FAQ on packaging (#883)
* update the FAQ on packaging

Co-authored-by: Silvan Mosberger <github@infinisil.com>
2024-01-29 01:41:51 +00:00
Valentin Gagarin cf94730f93
show nix-shell --run (#886)
* show nix-shell --run

it's quite useful and not necessarily obvious without reading reference documentation

Co-authored-by: Silvan Mosberger <github@infinisil.com>
2024-01-29 01:41:04 +00:00
Silvan Mosberger 5f1eea5bc2
Merge pull request #880 from fricklerhandwerk/wording
reword shell.nix tutorial introduction
2024-01-29 01:17:43 +01:00
Silvan Mosberger 12649edcde
Merge pull request #878 from fricklerhandwerk/highlight
highlight recommendation to use search.nixos.org
2024-01-29 01:13:27 +01:00
Valentin Gagarin be612d5d12 reword shell.nix tutorial introduction
make shorter sentences and drop the key message about sharing
declarative environments
2024-01-28 20:00:12 +01:00
Valentin Gagarin b0a8a39fce highlight recommendation to use search.nixos.org
this way it's easier to find and probably also remember
2024-01-28 19:52:26 +01:00
Valentin Gagarin 75eceb50cd
small wording improvements (#877) 2024-01-28 18:30:34 +00:00
Valentin Gagarin 41f580f488 remove email subscription form
historically it still points to Cachix and not to a community-owned
instance. at the moment there is no regular community newsletter, which
would certainly include updates about documentation. we can add it back
once such a thing exists.
2024-01-26 21:54:21 +01:00
Mason Malone 6f56eda69a
Fix unquoted URL in "Nix language basics"
I'm a newbie, so I might be missing something, but the use of an unquoted URL here seems like it goes against the "Best Practice" page (https://nix.dev/guides/best-practices#urls), which says to always quote URLs.
2024-01-25 17:57:50 -08:00
Silvan Mosberger 46a7927691
Merge pull request #868 from Moraxyc/patch-1
fix: typo
2024-01-22 06:24:26 +01:00
Moraxyc 22b3a861cb
fix: typo
add ;
2024-01-22 12:13:35 +08:00
Silvan Mosberger 4c987d1441
Merge pull request #842 from fricklerhandwerk/unsubscribe
reorganise CODEOWNERS
2024-01-15 21:05:48 +01:00
Valentin Gagarin f28a554a99 CODEOWNERS: reorganise
- Assign subscriptions explicitly to some paths
- Use new documentation-reviewers GitHub team
2024-01-15 20:22:08 +01:00
Valentin Gagarin d6b29c39d9 add mutable shortlinks
this allows mapping cleanly from the existing mutable links on nixos.org
2024-01-15 13:49:14 +01:00
Valentin Gagarin 0ff3ceca01
use upstream page redirects for Nix manual (#829) 2024-01-15 13:44:42 +01:00
asymmetric 82d2ea7258
Merge pull request #860 from NixOS/dependabot/github_actions/cachix/install-nix-action-25
chore(deps): bump cachix/install-nix-action from 24 to 25
2024-01-15 11:20:44 +01:00
asymmetric 578a9a9bb8
Merge pull request #861 from NixOS/dependabot/github_actions/cachix/cachix-action-14
chore(deps): bump cachix/cachix-action from 13 to 14
2024-01-15 11:20:25 +01:00
dependabot[bot] 56125c9e01
chore(deps): bump cachix/install-nix-action from 24 to 25
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 24 to 25.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](https://github.com/cachix/install-nix-action/compare/v24...v25)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-15 10:17:43 +00:00
dependabot[bot] 5ccd31c74a
chore(deps): bump cachix/cachix-action from 13 to 14
Bumps [cachix/cachix-action](https://github.com/cachix/cachix-action) from 13 to 14.
- [Release notes](https://github.com/cachix/cachix-action/releases)
- [Commits](https://github.com/cachix/cachix-action/compare/v13...v14)

---
updated-dependencies:
- dependency-name: cachix/cachix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-15 10:17:42 +00:00
Valentin Gagarin 4f9862fdb9
disable linkcheck for GitHub anchor links (#859)
apparently it got (a lot) worse and GitHub doesn't render README files
statically any more.

this obviates the need of additional instructions.
2024-01-15 10:16:39 +00:00
Valentin Gagarin 0669665b42
simplify config.guess line (#858) 2024-01-07 10:08:50 +00:00
Alexander Groleau 143801d9ae
Merge pull request #854 from fricklerhandwerk/fix-noop-faq
remove no-op command in troubleshooting advice
2024-01-05 19:35:28 -08:00
Silvan Mosberger acb5c7acc5
Merge pull request #856 from henrik-ch/crosscompile-hostplatform-det
fix for issue with host platform determination
2024-01-05 23:04:14 +01:00
Henrik Karlsson 35909232d3 fix for issue with host platform determination
cross compilation tutorial.
2024-01-05 21:55:58 +00:00
Valentin Gagarin 15dfbcf6c3 remove no-op command in troubleshooting advice
`nix-store --init` doesn't actually do anything [0] and is not even
documented in the Nix manual

[0]: 9651034dc2/src/nix-store/nix-store.cc (L736-L744)
2024-01-05 18:50:50 +01:00
asymmetric 6a7d821003
Merge pull request #850 from greg0ire/patch-1
Refer to the tags page
2023-12-28 18:09:32 +01:00
Grégoire Paris 02f47d8a46
Refer to the tags page
The releases page is empty, and that might confuse people.
2023-12-28 15:02:05 +01:00
Silvan Mosberger f3799fbe19
Merge pull request #849 from carlthome/patch-1 2023-12-26 14:47:57 +01:00
Carl Thomé c5d7eab39d
nix-generate-config -> nixos-generate-config 2023-12-26 14:12:53 +01:00
amirreza8002 36f1df3008
updated channel version (#848)
22.11 to 23.11
2023-12-26 10:28:34 +00:00
Silvan Mosberger 0466cf0b0d
Merge pull request #846 from NixOS/fileset.gitTracked-update
File sets: Update note on `gitTracked`
2023-12-22 21:31:28 +01:00
Silvan Mosberger 840f7ff50a File sets: Improve a sentence
It was confusing before
2023-12-22 21:29:02 +01:00
Silvan Mosberger 4ebcad8b0d File sets: Update note on gitTracked
Now that https://github.com/NixOS/nixpkgs/pull/273893 is merged, the
previous note isn't accurate anymore.
2023-12-22 21:28:58 +01:00
Valentin Gagarin 7f237c263d WIP: reword, reformat, fix links 2023-05-04 11:26:10 +02:00
Valentin Gagarin adb335f39f add guide on post build hook
originally written under LGPL 2.1 for the Nix reference manual.
authors as recorded in the Nix repository:

    git log --pretty="Co-authored-by: %an <%ae>" -- doc/manual/advanced-topics/post-build-hook.xml doc/manual/src/advanced-topics/post-build-hook.md | sort | uniq

relicenced with authors' permission to CC-BY-SA 4.0: https://github.com/NixOS/nix.dev/issues/463

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Graham Christensen <graham@grahamc.com>
Co-authored-by: Joe Hermaszewski <git@monoid.al>
Co-authored-by: Kevin Stock <kevin@kevinstock.org>
Co-authored-by: endgame <endgame@users.noreply.github.com>
Co-authored-by: regnat <rg@regnat.ovh>
2023-05-04 11:07:25 +02:00
43 changed files with 645 additions and 338 deletions

6
.github/CODEOWNERS vendored
View file

@ -11,11 +11,15 @@
# This also holds true for GitHub teams. Since almost none of our teams have write
# permissions, you need to list all members of the team with commit access individually.
* @NixOS/documentation-team
* @NixOS/documentation-reviewers
/.github/workflows/ @asymmetric
/source/tutorials/file-sets.md @infinisil
source/guides/recipes/dependency-management.md @infinisil
source/guides/recipes/python-environment.md @alejandrosame
source/tutorials/module-system @infinisil @asymmetric @proofconstruction
source/tutorials/packaging-existing-software.md @proofconstruction
/source/tutorials/integration-testing-using-virtual-machines.md @olafklingt
/source/tutorials/nixos-configuration-on-vm.md @olafklingt

View file

@ -10,10 +10,10 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v24
- uses: cachix/install-nix-action@v25
with:
nix_path: nixpkgs=channel:nixos-unstable
- uses: cachix/cachix-action@v13
- uses: cachix/cachix-action@v14
with:
name: nix-dev
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'

View file

@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: cachix/install-nix-action@v24
- uses: cachix/install-nix-action@v25
with:
nix_path: nixpkgs=channel:nixos-unstable
- name: Checking EditorConfig

View file

@ -11,10 +11,10 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v24
- uses: cachix/install-nix-action@v25
with:
nix_path: nixpkgs=channel:nixos-unstable
- uses: cachix/cachix-action@v13
- uses: cachix/cachix-action@v14
with:
name: nix-dev
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
@ -26,10 +26,10 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v24
- uses: cachix/install-nix-action@v25
with:
nix_path: nixpkgs=channel:nixos-unstable
- uses: cachix/cachix-action@v13
- uses: cachix/cachix-action@v14
with:
name: nix-dev
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'

View file

@ -8,6 +8,6 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: cachix/install-nix-action@v24
- uses: cachix/install-nix-action@v25
- name: Vale
run: nix-shell --run "vale maintainers source"

View file

@ -99,23 +99,3 @@ We would be glad to incorporate your insights.
## Contributor guides
Please read [Contributing Documentation](https://nix.dev/contributing/documentation).
## GitHub heading anchors fails linkcheck
Due to a [Sphinx bug][linkcheck gh bug], linkcheck fails when it verifies the
existence of GitHub heading anchors on rendered Markdown documents.
Until the bug is resolved, add the `user-content-` prefix to GitHub links
containing heading anchors.
For example, instead of
```
https://github.com/nix-community/nixos-generators#cross-compiling
```
use
```
https://github.com/nix-community/nixos-generators#user-content-cross-compiling
```

View file

@ -7,10 +7,18 @@ Official documentation for getting things done with Nix.
## Contributing
Content is written in MyST, a superset of CommonMark. For its syntax, see the [MyST docs](https://myst-parser.readthedocs.io/en/latest/syntax/typography.html#syntax-core).
For contents and style see [contribution guide](CONTRIBUTING.md).
## Local preview
Run `nix-shell --run devmode` and open a browser at <http://localhost:5500>.
As you make changes your browser should auto-reload within a few seconds.
For contents and style see [contribution guide](CONTRIBUTING.md).
To manually test [redirects](./_redirects):
Content is written in MyST, a superset of CommonMark. For its syntax, see the [MyST docs](https://myst-parser.readthedocs.io/en/latest/syntax/typography.html#syntax-core).
```console
nix-shell -p netlify-cli --run "cd result; netlify dev"
```

View file

@ -1,3 +1,7 @@
# format documentation:
# - https://docs.netlify.com/routing/redirects/#syntax-for-the-redirects-file
# - https://docs.netlify.com/routing/redirects/redirect-options/
/tutorials/install-nix /install-nix 301
/anti-patterns/language /recipes/best-practices 301
/faq /recipes/faq 301

View file

@ -27,25 +27,39 @@ let
'';
installPhase =
let
# Various versions of the Nix manuals, grep for (nix-manual)=
# FIXME: This requires human interaction to update!
# See ./CONTRIBUTING.md for details.
releases = [
"2.19"
"2.18"
"2.13"
];
# Various versions of the Nix manuals, grep for (nix-manual)= to find where they are displayed
# FIXME: This requires human interaction to update! See ./CONTRIBUTING.md for details.
releases = {
latest = "2.19";
rolling = "2.18";
stable = "2.18";
prev-stable = "2.13";
};
inputName = version: pkgs.lib.strings.replaceStrings [ "." ] [ "-" ] version;
src = version: (import inputs."nix_${inputName version}").default.doc;
src = version: inputs."nix_${inputName version}";
manual = version: (import (src version)).default.doc;
copy = version: ''
cp -R ${src version}/share/doc/nix/manual/* $out/manual/nix/${version}
cp -Rf ${manual version}/share/doc/nix/manual/* $out/manual/nix/${version}
'';
# add upstream page redirects of the form `<from> <to> <status>`, excluding comment lines and empty
redirects = version: ''
sed '/^#/d;/^$/d;s#^\(.*\) \(.*\) #/manual/nix/${version}\1 /manual/nix/${version}\2 #g' ${src version}/doc/manual/_redirects >> $out/_redirects
'';
shortlink = release: version: ''
echo /manual/nix/${release} /manual/nix/${version}/ 302 >> $out/_redirects
'';
versions = with pkgs.lib; lists.unique (attrsets.attrValues releases);
in
with pkgs.lib.attrsets;
with pkgs.lib.strings;
''
mkdir -p $out/manual/nix/{${concatStringsSep "," releases}}
${concatStringsSep "\n" (map copy releases)}
mkdir -p $out
cp -R build/html/* $out/
# NOTE: the comma in the shell expansion makes it also work for singleton lists
mkdir -p $out/manual/nix/{${concatStringsSep "," versions},}
${concatStringsSep "\n" (map copy versions)}
${concatStringsSep "\n" (map redirects versions)}
${concatStringsSep "\n" (mapAttrsToList shortlink releases)}
'';
};

View file

@ -1,12 +1,12 @@
# Nix documentation team
The Nix documentation team was formed to flatten the infamous learning curve.
The Nix documentation team was formed to [flatten the infamous learning curve](https://discourse.nixos.org/t/documentation-team-flattening-the-learning-curve/20003).
## Goals
- ease learning, increase onboarding success and user retention
- improve organization of knowledge
- lead, guide, and support related community efforts
- Ease learning, increase onboarding success and user retention
- Improve organisation of knowledge
- Lead, guide, and support related community efforts
## Motivation
@ -16,13 +16,15 @@ fix numerous little problems and help people get their contributions merged.
It requires significant time or resources to do this consistently.
The team is built around that limitation, and therefore organized as a praxicracy:
you are in charge if and only if you get the work done.
You are in charge if and only if you get the work done.
The teams reason to exist is to make that principle discoverable and reproducible by laying groundwork and setting examples.
## Members
The team member list is maintained on the [Nix community documentation team page](https://nixos.org/community/teams/documentation).
Current members are listed on the GitHub team [`@NixOS/documentation-team`](https://github.com/orgs/NixOS/teams/documentation-team).
Ping [`@NixOS/documentation-reviewers`](https://github.com/orgs/NixOS/teams/documentation-reviewers) for documentation reviews.
## Responsibilities
@ -91,7 +93,7 @@ Contact [@fricklerhandwerk] to get a calendar invitation.
- [Meeting notes scratchpad][collaborative scratchpad]
- [GitHub project board]
[Previous meeting notes](https://discourse.nixos.org/search?q=documentation%20team%20meeting%20%23dev%3Adocumentation%20order%3Alatest)
[Previous meeting notes](https://discourse.nixos.org/search?q=documentation%20team%20meeting%20notes%20%23%20%23dev%3Adocumentation%20in%3Atitle%20order%3Alatest_topic)
### Meeting protocol

View file

@ -42,7 +42,7 @@ To better navigate the material and judge its relevance, every entry should prov
- https://nixos.org/guides/nix-pills/nix-search-paths.html
- https://nixos.org/guides/nix-pills/nix-store-paths.html
- [Nix Pills Chapter 18. Nix Store Paths](https://nixos.org/guides/nix-pills/nix-store-paths.html)
Store paths and how they are computed, `nix-store`, `nix-hash`, `nix derivation show`, `nix repl`.
- Reading time: 6 min

View file

@ -1,66 +0,0 @@
# How to contribute to documentation in the Nix ecosystem
If you want to help improving documentation, consider the following guidelines developed by the [Nix documentation team](./README.md).
Different audiences require different levels of sophistication, and both readers' and contributors' available time and expertise varies widely.
Use the following diagram to decide where to best put the results of your efforts, depending on how much time you can spend on refining them to get accepted.
The diagram shows a pipeline for contributions to learning materials and documentation.
Arrows point towards
- decreasing rate of change
- increasing public visibility
- increasing amount of scrutiny.
Communication platforms are distinguished by mode of interaction:
Streams can be appended to:
- The amount of information in them grows without bounds.
- Only recent items are relevant.
Artifacts can be changed:
- The amount of information in them is strictly bounded.
- All items should be up to date.
```mermaid
flowchart
Matrix --> |filter| Discourse
events[meetings and events] --> |report| Discourse
external[external sources] --> |collect| Discourse
Discourse --> |design| RFCs
Discourse --> |change| code[source code]
Discourse --> |learn| Wiki
Wiki --> |refine| manuals --> |guide| dev[nix.dev] --> |overview| www[nixos.org]
RFCs --> code --> |document| manuals
subgraph stream
Matrix
events
external
Discourse
RFCs
end
subgraph artifact
Wiki
code
manuals
dev
www
end
```
## Contributing to this guide
Do you have suggestions how to ease contributing to documentation in the Nix ecosystem?
Please open a pull request to update this document with your proposals.
## Links to this guide
In case this guide moves, the following documents link here and should be updated:
- [Discourse: Summer of Nix documentation stream](https://discourse.nixos.org/t/summer-of-nix-documentation-stream/20351)
- [Discourse: How to contribute to documentation](https://discourse.nixos.org/t/how-to-contribute-to-documentation/21028)
- [NixOS Wiki: Contributing to Nix documentation](https://nixos.wiki/wiki/Contributing_to_Nix_documentation)

View file

@ -0,0 +1,53 @@
# Information flow in the Nix ecosystem
The diagram shows a simplified model of the information flow in the Nix ecosystem.
Its purpose is to make dependencies of the documentation types more explicit.
Note that it does not show issues and pull requests on GitHub, which host a wealth of ephemeral information.
Nodes are places or resources, edges denote an "informed by" relation, and edge labels are units of information.
Read the example as "A is informed by X and placed in Y":
```mermaid
flowchart LR
X --> |A| Y
```
Information resources communication platforms are distinguished by mode of interaction:
*Streams* can be appended to:
- The amount of information in them grows without bounds.
- Only recent items are relevant.
*Artifacts* can be changed:
- The amount of information in them is strictly bounded.
- All items should be up to date.
```mermaid
flowchart
Matrix --> |summary| Discourse
events[meetings and events] --> |meeting notes, report| Discourse
external[external sources] --> |link| Discourse
Discourse --> |draft| RFCs --> |implementation| code
RFCs --> |explanation| docs
code --> |reference| manuals --> |guide| docs[nix.dev]
Discourse --> |tutorial| docs
subgraph stream
Matrix
events
external
Discourse
RFCs
end
subgraph artifact
code
manuals
docs
end
```
Arrows point roughly towards
- decreasing rate of change
- increasing public visibility

View file

@ -2,9 +2,7 @@
## Join the team meetings
Participate in the regular team meetings, see [meeting information and protocol] for details.
[meeting information and protocol]: ./README.md#team-meetings
Participate in the regular team meetings, see [meeting information and protocol](./README.md#team-meetings) for details.
If you can't participate, notify other team members.
@ -18,15 +16,13 @@ Follow relevant activities on these communication channels:
- Discourse
Set the notification level on the [Development > Documentation] Discourse category to "watching".
[Development > Documentation]: https://discourse.nixos.org/c/dev/documentation/25
Set the notification level on the [Development > Documentation](https://discourse.nixos.org/c/dev/documentation/25) Discourse category to "watching".
- Matrix
Join the [Nix\* Documentation] Matrix room and enable relevant notifications.
Join the public [Nix\* Documentation](https://matrix.to/#/#docs:nixos.org) Matrix room and enable relevant notifications.
[Nix\* Documentation]: https://matrix.to/#/#docs:nixos.org
There is also a [private Matrix room](https://matrix.to/#/!DersLzqcNaaZvpnxKk:matrix.org) for exceptional, internal issues.
## Take ownership
@ -48,8 +44,6 @@ Make sure to reserve time for reviewing pull requests that fall within your area
Guide new and potential contributors:
- Help answering questions on the [Discourse Help category]
- Help answering questions on the [Discourse Help category](https://discourse.nixos.org/c/learn/9)
- Encourage question authors to raise or upvote issues on GitHub, if needed
- Assist them with making pull requests to solve the underlying problems, if possible
[Discourse Help category]: https://discourse.nixos.org/c/learn/9

View file

@ -5,10 +5,10 @@
"homepage": "https://nixos.org/",
"owner": "nixos",
"repo": "nix",
"rev": "56eb17906822f14bb348017315cb1ca643da900a",
"sha256": "1k0x63l9crkrhdgdclcx8mp6r9mcp4brs67l6hsii1ag8gy2gmj4",
"rev": "25f2dfc6e41d8c30e7abc443a7b262e34e49253b",
"sha256": "0lbp6yq92wpzi7kihbbma8d1c6q33msw95y0kmirfiikd8yvpvs7",
"type": "tarball",
"url": "https://github.com/nixos/nix/archive/56eb17906822f14bb348017315cb1ca643da900a.tar.gz",
"url": "https://github.com/nixos/nix/archive/25f2dfc6e41d8c30e7abc443a7b262e34e49253b.tar.gz",
"url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
},
"nix_2-18": {
@ -17,10 +17,10 @@
"homepage": "https://nixos.org/",
"owner": "nixos",
"repo": "nix",
"rev": "f5f4de6a550327b4b1a06123c2e450f1b92c73b6",
"sha256": "1g23sxsh9aj7sard5d3ijqfjqbh4fvlvyr73sj3dcdzcqiys5naq",
"rev": "60eb80593f3a18aebc7672ad7007cb23c14db061",
"sha256": "0nyssab6skn9qd7mz4v0y3ycnhck7is6agm0i26l1anrgs90x37l",
"type": "tarball",
"url": "https://github.com/nixos/nix/archive/f5f4de6a550327b4b1a06123c2e450f1b92c73b6.tar.gz",
"url": "https://github.com/nixos/nix/archive/60eb80593f3a18aebc7672ad7007cb23c14db061.tar.gz",
"url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
},
"nix_2-19": {
@ -29,10 +29,10 @@
"homepage": "https://nixos.org/",
"owner": "nixos",
"repo": "nix",
"rev": "33bacbe220b49ceaaeb817354592a0102fbae087",
"sha256": "17fjz3mxqcszxfzpj48f8rqzdqavnnpibpi0ssw7vxmyybrqsqly",
"rev": "dc09e6193bffcab37d3d43107eae9464395ab51d",
"sha256": "0n117h0jz3aih41mg0kns8v6rjhmrg7r6p8azl55p32zrj02zyvm",
"type": "tarball",
"url": "https://github.com/nixos/nix/archive/33bacbe220b49ceaaeb817354592a0102fbae087.tar.gz",
"url": "https://github.com/nixos/nix/archive/dc09e6193bffcab37d3d43107eae9464395ab51d.tar.gz",
"url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
},
"nixpkgs": {

View file

@ -5,6 +5,6 @@
</a>
</div>
<iframe src="https://ghbtns.com/github-btn.html?user=nix-dot-dev&repo=nix.dev&type=star&count=true&size=large" frameborder="0" scrolling="0" width="170" height="30" title="GitHub"></iframe>
<iframe src="https://ghbtns.com/github-btn.html?user=nixos&repo=nix.dev&type=star&count=true&size=large" frameborder="0" scrolling="0" width="170" height="30" title="GitHub"></iframe>
<p>Official documentation for getting things done with Nix.</p>

View file

@ -1,17 +0,0 @@
<style type="text/css">
#mce-EMAIL { float: left; width: 55%; padding: 0.25em; }
#mc-embedded-subscribe { float: right; width: 40%; padding: 0.25em; }
#mc-embedded-subscribe-form { overflow: auto; margin-bottom: 20px; }
</style>
<h3>Email notifications</h3>
<form action="https://cachix.us1.list-manage.com/subscribe/post?u=16430f4cf6653e9f26595c40a&amp;id=f726238699"
method="post" id="mc-embedded-subscribe-form" name="mc-embedded-subscribe-form" class="validate" target="_blank" novalidate>
<p>An occasional email about new tutorials, no spam.</p>
<input type="email" value="" name="EMAIL" class="email" id="mce-EMAIL" placeholder="Email" required>
<div id="mce-responses" class="clear">
<div class="response" id="mce-error-response" style="display:none"></div>
<div class="response" id="mce-success-response" style="display:none"></div>
</div> <!-- real people should not fill this in and expect good things - do not remove this or risk form bot signups-->
<input type="submit" value="Subscribe" name="subscribe" id="mc-embedded-subscribe" class="button">
</form>

View file

@ -200,7 +200,6 @@ html_sidebars = {
"about.html",
"search-field.html",
"sbt-sidebar-nav.html",
"subscribe.html",
],
}
@ -419,7 +418,7 @@ linkcheck_ignore = [
# returns 403 on CI
r"https://www.lesswrong.com",
# Linkcheck fails on anchors in GH READMEs, see https://github.com/sphinx-doc/sphinx/issues/9016
r"https://github.com/cachix/install-nix-action#how-do-i-run-nixos-tests",
r"https://github.com/.+/.+#.+$",
# Linkcheck fails on anchors in GH browser/file viewer, see https://github.com/sphinx-doc/sphinx/issues/11484
r"https://github\.com/.+/.+/blob/.*#.*$",
r"https://github\.com/.+/.+/tree/.*#.*$",

View file

@ -196,6 +196,11 @@ Notify the authors *before* using their work.
Using free licenses other than CC-BY-SA 4.0 is possible for individual documents, and by contributing changes to those documents you agree to license your work accordingly.
:::{note}
If you have written a tutorial or guide related to Nix, please consider licensing it under CC-BY-SA 4.0!
This will allow us to feature your work as official documentation if it complements or improves upon existing materials.
:::
```{toctree}
:hidden:

View file

@ -1,3 +1,4 @@
(how-to-contribute)=
# How to contribute
The Nix ecosystem is developed by many volunteers and a few paid developers, maintaining one of the largest open source software distributions in the world.

View file

@ -45,7 +45,11 @@ See <https://github.com/nix-community/home-manager>
### What's the recommended process for building custom packages?
> E.g. if I git clone nixpkgs how do I use the cloned repo to define new / updated packages?
Please read [](packaging-existing-software).
### How to use a clone of the Nixpkgs repository to update or write new packages?
Please read [](packaging-existing-software) and the [Nixpkgs contributing guide](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
## NixOS

View file

@ -1,4 +1,4 @@
(direnv)=
(automatic-direnv)=
# Automatic environment activation with `direnv`
Instead of manually activating the environment for each project, you can reload a [declarative shell](declarative-reproducible-envs) every time you enter the project's directory or change the `shell.nix` inside it.

View file

@ -4,8 +4,8 @@
```{toctree}
:maxdepth: 1
sharing-dependencies.md
Automatic environments <direnv>
sharing-dependencies.md
dependency-management.md
Python development environment <./python-environment.md>
```

View file

@ -1,7 +1,7 @@
(python-dev-environment)=
# Setting up a Python development environment
In this example you will build a Python web application using the Flask web framework as an exercise.
In this example you will build a Python web application using the [Flask](https://palletsprojects.com/p/flask/) web framework as an exercise.
To make best use of it you should be familiar with [defining declarative shell environments](declarative-reproducible-envs).
Create a new file called `myapp.py` and add the following code:
@ -26,40 +26,33 @@ if __name__ == "__main__":
run()
```
This is a simple Flask application which serves a JSON document with the message
"Hello, Nix!".
This is a simple Flask application which serves a JSON document with the message `"Hello, Nix!"`.
To declare the development environment, create a new file `shell.nix`:
Create a new file `shell.nix` to declare the development environment:
```{code-block} nix shell.nix
{ pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-22.11") {} }:
pkgs.mkShell {
packages = [
(pkgs.python3.withPackages (ps: [
ps.flask
]))
pkgs.curl
pkgs.jq
packages = with pkgs; [
(python3.withPackages (ps: [ ps.flask ]))
curl
jq
];
}
```
This creates a shell environment with an instance of `python3` that includes the `flask` package.
This describes a shell environment with an instance of `python3` that includes the `flask` package using [`python3.withPackages`](https://nixos.org/manual/nixpkgs/stable/#python.withpackages-function).
It also contains [`curl`], a utility to perform web
requests, and [`jq`], a tool to parse and format JSON documents.
[`curl`]: https://curl.se
[`jq`]: https://stedolan.github.io/jq/
[virtualenv]: https://virtualenv.pypa.io/en/latest/
[`curl`]: https://search.nixos.org/packages?show=curl
[`jq`]: https://search.nixos.org/packages?show=jq
Both of them are not Python packages.
If you went with Python's [virtualenv], it would not be possible to add these utilities
to the development environment without additional manual steps.
If you went with Python's [virtualenv](https://virtualenv.pypa.io/en/latest/), it would not be possible to add these utilities to the development environment without additional manual steps.
Use `nix-shell` to launch the shell environment you just declared:
Run `nix-shell` to enter the environment you just declared:
```shell-session
$ nix-shell
@ -71,13 +64,13 @@ these 93 paths will be fetched (109.50 MiB download, 468.52 MiB unpacked):
/nix/store/138azk9hs5a2yp3zzx6iy1vdwi9q26wv-hook
...
[nix-shell:~/dev-environment]$
[nix-shell:~]$
```
Start the web application within this shell environment:
```shell-session
[nix-shell:~/dev-environment]$ python ./myapp.py
[nix-shell:~]$ python ./myapp.py
* Serving Flask app 'myapp'
* Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
@ -88,25 +81,31 @@ Press CTRL+C to quit
```
You now have a running Python web application.
Try it out!
Open a new terminal to start another session of the shell environment and follow the commands below:
```shell-session
$ nix-shell
[nix-shell:~/dev-environment]$ curl 127.0.0.1:5000
[nix-shell:~]$ curl 127.0.0.1:5000
{"message":"Hello, Nix!"}
[nix-shell:~/dev-environment]$ curl 127.0.0.1:5000 | jq '.message'
[nix-shell:~]$ curl 127.0.0.1:5000 | jq '.message'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 26 100 26 0 0 13785 0 --:--:-- --:--:-- --:--:-- 26000
"Hello, Nix!"
```
As demonstrated, we can use both `curl` and `jq` to test the running web application without any manual installation.
Nix does all of that for us.
As demonstrated, you can use both `curl` and `jq` to test the running web application without any manual installation.
We can commit the files we created to version control and share them with other people.
Others can now use the same shell environment as long as they have Nix installed.
You can commit the files we created to version control and share them with other people.
Others can now use the same shell environment as long as they have [Nix installed](install-nix).
## Next steps
- [](packaging-existing-software)
- [](file-sets)
- [](automatic-direnv)
- [](./dependency-management.md)

View file

@ -36,13 +36,12 @@ This is a [known issue](https://github.com/NixOS/nix/issues/1251).
It means that using a new version of Nix upgraded the SQLite schema of the [database](https://nix.dev/manual/nix/2.18/glossary#gloss-nix-database), and then you tried to use an older version Nix.
The solution is to dump the database, use the old Nix version to initialize it, and then re-import the data:
The solution is to dump the database, and use the old Nix version to re-import the data:
```shell-session
$ /path/to/nix/unstable/bin/nix-store --dump-db > /tmp/db.dump
$ mv /nix/var/nix/db /nix/var/nix/db.toonew
$ mkdir /nix/var/nix/db
$ nix-store --init # this is the old nix-store
$ nix-store --load-db < /tmp/db.dump
```

View file

@ -68,6 +68,27 @@ The following illustrate of what can be achieved with the Nix ecosystem:
- Atomic upgrades and rollbacks.
## Who is Nix for?
Nix is a tool for people who both need computers to do exactly as intended, repeatably, far into the future, and who are familiar with command line interfaces and plain text editors.
You don't have to be a professional software developer and you don't need formal education in informatics to greatly benefit from Nix.
However, experience with complex software projects and knowing some informatics helps with appreciating why it's useful and how it works.
And it helps with learning how to use it effectively and [how to make improvements](how-to-contribute).
You probably won't want to go back to a world without Nix if you're a:
- Full-stack or back-end developer
- Test engineer
- Embedded systems developer
- DevOps engineer
- System administrator
- Data scientist
- Natural scientist
- Student of a technical field
- Open source software enthusiast
```{toctree}
:hidden:

View file

@ -0,0 +1,118 @@
# Upload build results to S3
This guide shows how to use the Nix [`post-build-hook`](https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-post-build-hook) configuration option to automatically upload build results to an S3-compatible binary cache.
## Implementation Caveats
This is a simple and working example, but it is not suitable for all use cases.
The post-build hook program runs after each executed build, and blocks the build loop.
The build loop exits if the hook program fails.
Concretely, this implementation will make Nix slow or unusable when the network connection is slow or unreliable.
A more advanced implementation might pass the store paths to a user-supplied daemon or queue for processing the store paths outside of the build loop.
# Prerequisites
<!-- TODO: this information will move: https://github.com/NixOS/nix/issues/7769 -->
This tutorial assumes you have [configured an S3-compatible binary cache](https://nixos.org/manual/nix/stable/package-management/s3-substituter.html), and that the `root` user's default AWS profile can upload to the bucket.
# Set up a Signing Key
Use [`nix-store --generate-binary-cache-key`](https://nixos.org/manual/nix/stable/command-ref/nix-store/generate-binary-cache-key.html) to create a pair of cryptographic keys.
You will sign paths with the private key, and distribute the public key for verifying the authenticity of the paths.
```console
$ nix-store --generate-binary-cache-key example-nix-cache-1 /etc/nix/key.private /etc/nix/key.public
$ cat /etc/nix/key.public
example-nix-cache-1:1/cKDz3QCCOmwcztD2eV6Coggp6rqc9DGjWv7C0G+rM=
```
Then update [`nix.conf`](../command-ref/conf-file.md) on any machine that will access the cache.
Add the cache URL to [`substituters`](../command-ref/conf-file.md#conf-substituters) and the public key to [`trusted-public-keys`](../command-ref/conf-file.md#conf-trusted-public-keys):
substituters = https://cache.nixos.org/ s3://example-nix-cache
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= example-nix-cache-1:1/cKDz3QCCOmwcztD2eV6Coggp6rqc9DGjWv7C0G+rM=
Machines that build for the cache must sign derivations using the private key.
On those machines, add the path to the key file to the [`secret-key-files`](../command-ref/conf-file.md#conf-secret-key-files) field in their [`nix.conf`](../command-ref/conf-file.md):
secret-key-files = /etc/nix/key.private
We will restart the Nix daemon in a later step.
# Implementing the build hook
Write the following script to `/etc/nix/upload-to-cache.sh`:
```bash
#!/bin/sh
set -eu
set -f # disable globbing
export IFS=' '
echo "Uploading paths" $OUT_PATHS
exec nix copy --to "s3://example-nix-cache" $OUT_PATHS
```
> **Note**
>
> The `$OUT_PATHS` variable is a space-separated list of Nix store
> paths. In this case, we expect and want the shell to perform word
> splitting to make each output path its own argument to `nix
> store sign`. Nix guarantees the paths will not contain any spaces,
> however a store path might contain glob characters. The `set -f`
> disables globbing in the shell.
Then make sure the hook program is executable by the `root` user:
```console
# chmod +x /etc/nix/upload-to-cache.sh
```
# Updating Nix Configuration
Edit `/etc/nix/nix.conf` to run our hook, by adding the following
configuration snippet at the end:
post-build-hook = /etc/nix/upload-to-cache.sh
Then, restart the `nix-daemon`.
# Testing
Build any derivation, for example:
```console
$ nix-build -E '(import <nixpkgs> {}).writeText "example" (builtins.toString builtins.currentTime)'
this derivation will be built:
/nix/store/s4pnfbkalzy5qz57qs6yybna8wylkig6-example.drv
building '/nix/store/s4pnfbkalzy5qz57qs6yybna8wylkig6-example.drv'...
running post-build-hook '/home/grahamc/projects/github.com/NixOS/nix/post-hook.sh'...
post-build-hook: Signing paths /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
post-build-hook: Uploading paths /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
/nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
```
Then delete the path from the store, and try substituting it from the
binary cache:
```console
$ rm ./result
$ nix-store --delete /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
```
Now, copy the path back from the cache:
```console
$ nix-store --realise /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
copying path '/nix/store/m8bmqwrch6l3h8s0k3d673xpmipcdpsa-example from 's3://example-nix-cache'...
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/m8bmqwrch6l3h8s0k3d673xpmipcdpsa-example
```
# Conclusion
We now have a Nix installation configured to automatically sign and
upload every local build to a remote binary cache.
Before deploying this to production, be sure to consider the
[implementation caveats](#implementation-caveats).

View file

@ -1,4 +1,4 @@
# Recommended reading
# Further reading
## Nix language tutorials
@ -55,3 +55,4 @@
- [Overlays](https://www.youtube.com/watch?v=dGAL3gMXvug) (Jon Ringer, 2020)
- [Adding a module to NixOS](https://www.youtube.com/watch?v=bkDYmvKINm8) (Jon Ringer, 2022)
- [Upgrade kernel](https://www.youtube.com/watch?v=Zi_vbddNXtg) (Jon Ringer, 2020)
- [NixOS Foundation on YouTube](https://www.youtube.com/@NixOS-Foundation/playlists)

View file

@ -7,11 +7,12 @@ These sections contains collections of detailed technical descriptions.
:glob:
:maxdepth: 2
glossary.md
./nix-manual.md
Nixpkgs manual <https://nixos.org/manual/nixpkgs/stable/>
NixOS manual <https://nixos.org/manual/nixos/stable/>
Recommended support tools <https://github.com/nix-community/awesome-nix>
Community projects <https://github.com/nix-community/>
Support tools <https://github.com/nix-community/awesome-nix>
../recommended-reading.md
pinning-nixpkgs.md
glossary.md
```

View file

@ -49,8 +49,7 @@ The build platform is determined automatically by Nix during the configure phase
The host platform is best determined by running this command on the host platform:
```shell-session
$ gnu-config=$(nix-build '<nixpkgs>' -I nixpkgs=channel:nixos-22.11 -A gnu-config)
$ "$gnu-config"/config.guess
$ $(nix-build '<nixpkgs>' -I nixpkgs=channel:nixos-22.11 -A gnu-config)/config.guess
aarch64-unknown-linux-gnu
```

View file

@ -8,7 +8,8 @@ Using these features directly can be tricky however:
- Coercion of paths to strings, such as the wide-spread pattern of `src = ./.`,
makes the derivation dependent on the name of the current directory.
It always adds the entire directory, including unneeded files that cause unnecessary new builds when they change.
Furthermore, it always adds the entire directory to the store, including unneeded files,
which causes unnecessary new builds when they change.
- The [`builtins.path`](https://nixos.org/manual/nix/stable/language/builtins.html#builtins-path) function
(and equivalently [`lib.sources.cleanSourceWith`](https://nixos.org/manual/nixpkgs/stable/#function-library-lib.sources.cleanSourceWith))
@ -554,14 +555,8 @@ This includes too much though, as not all of these files are needed to build the
:::{note}
When using the [`flakes` and `nix-command` experimental features](https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake),
it's [not really possible](https://github.com/NixOS/nix/issues/9292) to use this function, even with
```shell-session
$ nix build path:.
```
However it's also not needed, because by default, `nix build` only allows access to files tracked by Git.
this function isn't needed, because `nix build` by default only allows access to files tracked by Git.
However, in order to provide the same developer experience for stable Nix, use of this function is nevertheless recommended.
:::
## Intersection

View file

@ -54,12 +54,28 @@ $ echo all gone | lolcat
The program lolcat is currently not installed.
```
## Running programs once
You can go even faster, by running any program directly:
```console
$ nix-shell -p cowsay --run "cowsay Nix"
```
If the command consists only of the program name, no quotes are needed:
```console
$ nix-shell -p hello --run hello
```
## Search for packages
What can you put in a shell environment?
If you can think of it, there's probably a Nix package of it.
:::{tip}
Enter the program name you want to run in [search.nixos.org](https://search.nixos.org/packages) to find packages that provide it.
:::
For the following example, find the package names for these programs:

View file

@ -17,18 +17,20 @@ Declarative shell environments allow you to:
### What will you learn?
In the {ref}`ad-hoc-envs` tutorial, we looked at imperatively creating shell environments using `nix-shell -p`, when we need a quick way to access tools without having to install them globally.
We also saw how to execute that command with a specific Nixpkgs revision using a Git commit as an argument, to recreate the same environment used previously.
In the {ref}`ad-hoc-envs` tutorial, you learned how to imperatively create shell environments using `nix-shell -p`.
This is great when you want to quickly access tools without installing them permanently.
You also learned how to execute that command with a specific Nixpkgs revision using a Git commit as an argument, to recreate the same environment used previously.
In this tutorial we'll take a look at how to create reproducible shell environments given a declarative configuration in a {term}`Nix file`.
In this tutorial we'll take a look at how to create reproducible shell environments with a declarative configuration in a {term}`Nix file`.
This file can be shared with anyone to recreate the same environment on a different machine.
### How long will it take?
30 minutes
### What will you need?
### What do you need?
- A basic understanding of the [Nix language](reading-nix-language)
- A rudimentary understanding of the [Nix language](reading-nix-language)
## Entering a temporary shell
@ -50,7 +52,7 @@ Create a file called `shell.nix` with these contents:
```nix
let
nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-22.11";
nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-23.11";
pkgs = import nixpkgs { config = {}; overlays = []; };
in
@ -102,7 +104,7 @@ Set your `GIT_EDITOR` to use the `nvim` from the shell environment:
```diff
let
nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-22.11";
nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-23.11";
pkgs = import nixpkgs { config = {}; overlays = []; };
in
@ -142,7 +144,7 @@ Set `shellHook` to output the current repository status:
```diff
let
nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-22.11";
nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-23.11";
pkgs = import nixpkgs { config = {}; overlays = []; };
in
@ -161,9 +163,16 @@ Set `shellHook` to output the current repository status:
}
```
Exit the shell by typing `exit` or pressing `Ctrl`+`D`, then start it again with `nix-shell` to observe the effect.
## References
- [`mkShell` documentation](https://nixos.org/manual/nixpkgs/stable/#sec-pkgs-mkShell)
- Nixpkgs [shell functions and utilities](https://nixos.org/manual/nixpkgs/stable/#ssec-stdenv-functions) documentation
- [`nix-shell` documentation](https://nix.dev/manual/nix/2.18/command-ref/nix-shell)
## Next steps
- [](automatic-direnv)
- [](../../guides/recipes/sharing-dependencies.md)
- [](../../guides/recipes/dependency-management.md)

View file

@ -5,12 +5,12 @@
This tutorial series is where you should start learning Nix.
In these lessons, you will use basic Nix commands to obtain almost any piece of software, create development environments on the fly, and learn how to make reproducible scripts.
You will also learn the Nix language, and use it to build portable, reproducible development environments.
You will also learn reading the Nix language, and later use it to build portable, reproducible development environments.
```{toctree}
:maxdepth: 1
ad-hoc-shell-environments.md
reproducible-scripts.md
towards-reproducibility-pinning-nixpkgs.md
declarative-shell.md
towards-reproducibility-pinning-nixpkgs.md
```

View file

@ -71,7 +71,12 @@ We specify `bash` as the interpreter for the rest of the file with the `-i` opti
We enable the `--pure` option to prevent the script from implicitly using programs that may already exist on the system that will run the script.
With the `-p` option we specify the packages required for the script to run.
The command `xml2json` is provided by the package `python3Packages.xmljson`, while `bash`, `jq`, and `curl` are provided by packages of the same name. `cacert` must be present for SSL authentication to work. Use [search.nixos.org](https://search.nixos.org/packages) to find packages providing the program you need.
The command `xml2json` is provided by the package `python3Packages.xmljson`, while `bash`, `jq`, and `curl` are provided by packages of the same name.
`cacert` must be present for SSL authentication to work.
:::{tip}
Use [search.nixos.org](https://search.nixos.org/packages) to find packages providing the program you need.
:::
The parameter of `-I` refers to a specific Git commit of the Nixpkgs repository.
This ensures that the script will always run with the exact same packages versions, everywhere.

View file

@ -1,3 +1,4 @@
(module-system-deep-dive)=
# Module system deep dive
Or: *Wrapping the world in modules*

View file

@ -1028,12 +1028,12 @@ Functions are everywhere in the Nix language and deserve particular attention.
A function always takes exactly one argument.
Argument and function body are separated by a colon (`:`).
Wherever you see a colon (`:`) in Nix language code:
Wherever you find a colon (`:`) in Nix language code:
- On its left is the function argument
- On its right is the function body.
Function arguments are the third way, apart from [attribute sets](attrset) and [`let` expressions](let), to assign names to values.
Notably, values are not known in advance: the names are used as placeholders that are filled when [calling a function](calling-functions).
Notably, values are not known in advance: the names are placeholders that are filled when [calling a function](calling-functions).
Function declarations in the Nix language can appear in different forms.
Each of them is explained in the following, and here is an overview:
@ -1560,7 +1560,7 @@ After reading `file.nix` the Nix expression is equivalent to the file contents:
Since a Nix file can contain any Nix expression, `import`ed functions can be applied to arguments immediately.
That is, whenever you see additional tokens after a call to `import`, the value it returns should be a function, and anything that follows are arguments to that function.
That is, whenever you find additional tokens after a call to `import`, the value it returns should be a function, and anything that follows are arguments to that function.
Example:
@ -1658,7 +1658,7 @@ A fully reproducible example would therefore look like this:
```{code-block} nix
:class: expression
let
nixpkgs = fetchTarball https://github.com/NixOS/nixpkgs/archive/06278c77b5d162e62df170fec307e83f1812d94b.tar.gz;
nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/archive/06278c77b5d162e62df170fec307e83f1812d94b.tar.gz";
pkgs = import nixpkgs {};
in
pkgs.lib.strings.toUpper "always pin your sources"
@ -1870,7 +1870,7 @@ It is usually wrapped by the Nixpkgs build mechanism `stdenv.mkDerivation`, whic
You will probably never encounter `derivation` in practice.
:::
Whenever you see `mkDerivation`, it denotes something that Nix will eventually *build*.
Whenever you encounter `mkDerivation`, it denotes something that Nix will eventually *build*.
Example: [a package using `mkDerivation`](mkDerivation-example)

View file

@ -5,6 +5,7 @@ myst:
"keywords": "Docker, containers, Nix, reproducible, build, tutorial"
---
(nixos-docker-images)=
# Building and running Docker images
[Docker](https://www.docker.com/) is a set of tools and services used to build, manage and deploy containers.

View file

@ -1,3 +1,4 @@
(bootable-iso-image)=
# Building a bootable ISO image
:::{note}

View file

@ -1,14 +1,14 @@
# NixOS
Learn how to install, configure, and deploy NixOS.
Learn how to configure, install, and deploy NixOS.
```{toctree}
:maxdepth: 1
building-and-running-docker-images.md
building-bootable-iso-image.md
deploying-nixos-using-terraform.md
installing-nixos-on-a-raspberry-pi.md
nixos-configuration-on-vm.md
integration-testing-using-virtual-machines.md
building-bootable-iso-image.md
installing-nixos-on-a-raspberry-pi.md
continuous-integration-github-actions.md
building-and-running-docker-images.md
deploying-nixos-using-terraform.md
```

View file

@ -44,7 +44,7 @@ pkgs.nixosTest {
machine2 = { config, pkgs, ... }: {
# ...
};
}
};
testScript = { nodes, ... }: ''
# ...
'';
@ -321,3 +321,9 @@ $ nix-build server-client-test.nix
A good inspiration is [Matrix bridging with an IRC](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/matrix/appservice-irc.nix).
<!-- TODO: move examples from https://nixos.wiki/wiki/NixOS_Testing_library to the NixOS manual and troubleshooting tips to nix.dev -->
## Next steps
- [](module-system-deep-dive)
- [](bootable-iso-image)
- [](nixos-docker-images)

View file

@ -6,20 +6,21 @@ One of the most important features of NixOS is the ability to configure the enti
NixOS configurations can be used to test and use NixOS using a virtual machine, independent of an installation on a "bare metal" computer.
:::{important}
A NixOS configuration is a Nix language function following the [NixOS module](https://nixos.org/manual/nixos/stable/index.html#sec-writing-modules) convention.
:::
## What will you learn?
This tutorial serves as an introduction creating NixOS virtual machines.
Virtual machines are a practical tool for debugging NixOS configurations.
Virtual machines are a practical tool for experimenting with or debugging NixOS configurations.
## What do you need?
- A working [Nix installation](https://nix.dev/manual/nix/2.18/installation/installation.html) on Linux, or [NixOS](https://nixos.org/manual/nixos/stable/index.html#sec-installation), with a graphical environment
- Basic knowledge of the [Nix language](reading-nix-language)
:::{important}
A NixOS configuration is a Nix language function following the [NixOS module](https://nixos.org/manual/nixos/stable/index.html#sec-writing-modules) convention.
For a thorough treatment of the module system, check the [](module-system-deep-dive) tutorial.
:::
## Starting from a default NixOS configuration
In this tutorial you will use a default configuration that is shipped with NixOS.
@ -95,7 +96,7 @@ The default NixOS configuration without comments is:
}
```
To be able to log in add the following lines to the returned attribute set:
To be able to log in, add the following lines to the returned attribute set:
```nix
users.users.alice = {
@ -109,7 +110,7 @@ To be able to log in add the following lines to the returned attribute set:
```
:::{admonition} NixOS
On NixOS your configuration generated with `nix-generate-config` contains this user configuration commented out.
On NixOS your configuration generated with `nixos-generate-config` contains this user configuration commented out.
:::
Additionally, you need to specify a password for this user.
@ -242,3 +243,9 @@ rm nixos.qcow2
- [Nix manual: `nix-build`](https://nix.dev/manual/nix/2.18/command-ref/nix-build.html).
- [Nix manual: common command-line options](https://nix.dev/manual/nix/2.18/command-ref/opt-common.html).
- [Nix manual: `NIX_PATH` environment variable](https://nix.dev/manual/nix/2.18/command-ref/env-common.html#env-NIX_PATH).
## Next steps
- [](module-system-deep-dive)
- [](integration-testing-vms)
- [](bootable-iso-image)

View file

@ -8,26 +8,45 @@ myst:
(packaging-existing-software)=
# Packaging existing software with Nix
One of Nix's primary use-cases is in addressing common difficulties encountered while packaging software, like managing dependencies.
One of Nix's primary use-cases is in addressing common difficulties encountered with packaging software, such as specifying and obtaining dependencies.
In the long term, Nix helps tremendously in alleviating that stress, but when *first* packaging existing software with Nix, it's common to encounter missing dependencies preventing builds from succeeding.
In the long term, Nix helps tremendously with alleviating such problems.
But when *first* packaging existing software with Nix, it's common to encounter errors that seem inscrutable.
In this tutorial, you'll create your first Nix derivations to package C/C++ software, taking advantage of the [Nixpkgs Standard Environment](https://nixos.org/manual/nixpkgs/stable/#part-stdenv) (`stdenv`) which automates much of the work of building self-contained C/C++ packages.
## Introduction
The tutorial begins by considering `hello`, an implementation of "hello world" which only requires dependencies provided by `stdenv`.
In this tutorial, you'll create your first [Nix derivations](https://nix.dev/manual/nix/2.18/language/derivations) to package C/C++ software, taking advantage of the [Nixpkgs Standard Environment](https://nixos.org/manual/nixpkgs/stable/#part-stdenv) (`stdenv`), which automates much of the work involved.
### What will you learn?
The tutorial begins with `hello`, an implementation of "hello world" which only requires dependencies already provided by `stdenv`.
Next, you will build more complex packages with their own dependencies, leading you to use additional derivation features.
You'll encounter and address Nix error messages, build failures, and a host of other issues, developing your iterative debugging techniques along the way.
:::{note}
A _package_ is an informally defined Nixpkgs concept referring to a Nix derivation representing an installation of some project.
Packages have mostly standardised attributes and output layouts, allowing them to be discovered in searches and installed into environments alongside other packages.
### What do you need?
For the purposes of this tutorial, "package" means something like "result of a derivation"; this is the artifact you or others will use, as a consequence of having "packaged existing software with Nix".
- Familiarity with the Unix shell and plain text editors
- You should be confident with [reading the Nix language](reading-nix-language). Feel free to go back and work through the tutorial first.
### How long does it take?
Going through all the steps carefully will take around 60 minutes.
## Your first package
:::{note}
<!--
TODO: link to the Nix manual glossary entry once it's in a released build:
https://hydra.nixos.org/job/nix/master/build.x86_64-linux/latest/download/manual/glossary.html#package
-->
A _package_ is a loosely defined concept that refers to either a collection of files and other data, or a {term}`Nix expression` representing such a collection before it comes into being.
Packages in Nixpkgs have a conventional structure, allowing them to be discovered in searches and composed in environments alongside other packages.
For the purposes of this tutorial, a "package" is a Nix language function that will evaluate to a derivation.
It will enable you or others to produce an artifact for practical use, as a consequence of having "packaged existing software with Nix".
:::
## A simple project
To start, consider this skeleton derivation:
```nix
@ -38,37 +57,42 @@ stdenv.mkDerivation { };
This is a function which takes an attribute set containing `stdenv`, and produces a derivation (which currently does nothing).
As you progress through this tutorial, you will update this several times, adding more details while following the general pattern.
### A package function
### Hello, World!
GNU Hello is an implementation of the "hello world" program, with source code accessible [from the GNU Project's FTP server](https://ftp.gnu.org/gnu/hello/).
To begin, you should add a `name` attribute to the set passed to `mkDerivation`; every derivation needs a name, and Nix will throw `error: derivation name missing` without one.
To begin, add a `name` attribute to the set passed to `mkDerivation`.
Every package needs a name and a version, and Nix will throw `error: derivation name missing` without.
```diff
...
stdenv.mkDerivation {
+ name = "hello";
...
+ pname = "hello";
+ version = "2.12.1";
```
Next, you will download the [latest version](https://ftp.gnu.org/gnu/hello/hello-2.12.1.tar.gz) of `hello` using `fetchzip`, which takes the URI path to the download file and a SHA256 hash of its contents.
Next, you will declare a dependency on the latest version of `hello`, and instruct Nix to use `fetchzip` to download the [source code archive](https://ftp.gnu.org/gnu/hello/hello-2.12.1.tar.gz).
:::{note}
`fetchzip` can fetch [more archives](https://nixos.org/manual/nixpkgs/stable/#fetchurl) than just zip files!
`fetchzip` can fetch [more archives](https://nixos.org/manual/nixpkgs/stable/#fetchurl) than just] zip files!
:::
The hash cannot be known until after the tarball has been downloaded and unpacked, but Nix will complain if the hash supplied to `fetchzip` was incorrect, so it is common practice to supply a fake one with `lib.fakeSha256` and change the derivation definition after Nix reports the correct hash:
The hash cannot be known until after the archive has been downloaded and unpacked.
Nix will complain if the hash supplied to `fetchzip` is incorrect.
It is common practice to supply a fake one with `lib.fakeSha256` and change the derivation definition after Nix reports the correct hash:
```nix
# hello.nix
{ lib
, stdenv
, fetchzip
{
lib,
stdenv,
fetchzip,
}:
stdenv.mkDerivation {
name = "hello";
pname = "hello";
version = "2.12.1";
src = fetchzip {
url = "https://ftp.gnu.org/gnu/hello/hello-2.12.1.tar.gz";
@ -77,7 +101,7 @@ stdenv.mkDerivation {
}
```
Save this file to `hello.nix` and try to build it with `nix-build`, observing your first build failure:
Save this file to `hello.nix` and run `nix-build` to observe your first build failure:
```console
$ nix-build hello.nix
@ -97,27 +121,29 @@ error: cannot evaluate a function that has an argument without a value ('lib')
Problem: the expression in `hello.nix` is a *function*, which only produces its intended output if it is passed the correct *arguments*.
### A new command
`lib` is available from `nixpkgs`, which must be imported with another Nix expression in order to pass it as an argument to this derivation.
### Building with `nix-build`
The recommended way to do this is to create a `default.nix` in the same directory as `hello.nix`, with the following contents:
`lib` is available from [`nixpkgs`](https://github.com/NixOS/nixpkgs/), which must be imported with another Nix expression in order to pass it as an argument to this derivation.
The recommended way to do this is to create a `default.nix` file in the same directory as `hello.nix`, with the following contents:
```nix
# default.nix
let
pkgs = import <nixpkgs> { };
nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-22.11";
pkgs = import nixpkgs { config = {}; overlays = []; };
in
{
hello = pkgs.callPackage ./hello.nix { };
}
```
This allows you to use `nix-build -A hello` to realize the derivation in `hello.nix`, similar to the current convention used in `nixpkgs`.
This allows you to run `nix-build -A hello` to realize the derivation in `hello.nix`, similar to the current convention used in Nixpkgs.
:::{note}
[`callPackage`] automatically passes attributes from `pkgs` to the given function, if they match attributes required by that function's argument attrset.
In this case, `callPackage` will supply `lib`, and `stdenv` to the function defined in `hello.nix`.
In this case, `callPackage` will supply `lib`, `stdenv`, and `fetchzip` to the function defined in `hello.nix`.
:::
Now run the `nix-build` command with the new argument:
@ -141,17 +167,20 @@ error:
```
### Finding the file hash
As expected, the incorrect file hash caused an error, and Nix helpfully provided the correct one, which you can now substitute into `hello.nix` to replace `lib.fakeSha256`:
As expected, the incorrect file hash caused an error, and Nix helpfully provided the correct one.
In `hello.nix`, replace `lib.fakeSha256` with the correct hash:
```nix
# hello.nix
{ lib
, stdenv
, fetchzip
{
lib,
stdenv,
fetchzip,
}:
stdenv.mkDerivation {
name = "hello";
pname = "hello";
version = "2.12.1";
src = fetchzip {
url = "https://ftp.gnu.org/gnu/hello/hello-2.12.1.tar.gz";
@ -179,7 +208,7 @@ building
Great news: the derivation built successfully!
The console output shows that `configure` was called, which produced a `Makefile` that was then used to build the project.
It wasn't necessary to write any build instructions in this case because the `stdenv` build system is based on `autoconf`, which automatically detected the structure of the project directory.
It wasn't necessary to write any build instructions in this case because the `stdenv` build system is based on [GNU Autoconf](https://www.gnu.org/software/autoconf/), which automatically detected the structure of the project directory.
### Build result
Check your working directory for the result:
@ -189,7 +218,7 @@ $ ls
default.nix hello.nix result
```
This `result` is a symbolic link to a Nix store location containing the built binary; you can call `./result/bin/hello` to execute this program:
This `result` is a [symbolic link](https://en.wikipedia.org/wiki/Symbolic_link) to a Nix store location containing the built binary; you can call `./result/bin/hello` to execute this program:
```console
$ ./result/bin/hello
@ -200,15 +229,17 @@ Congratulations, you have successfully packaged your first program with Nix!
Next, you'll package another piece of software with external-to-`stdenv` dependencies that present new challenges, requiring you to make use of more `mkDerivation` features.
## Something bigger
## A package with dependencies
Now you will package a somewhat more complicated program, [`icat`](https://github.com/atextor/icat), which allows you to render images in your terminal.
To start, modify the `default.nix` from the previous section by adding a new attribute for `icat`:
Change the `default.nix` from the previous section by adding a new attribute for `icat`:
```nix
# default.nix
let
pkgs = import <nixpkgs> { };
nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-22.11";
pkgs = import nixpkgs { config = {}; overlays = []; };
in
{
hello = pkgs.callPackage ./hello.nix { };
@ -216,39 +247,44 @@ in
}
```
Now copy `hello.nix` to a new file, `icat.nix`, and update the `name` attribute in that file:
Copy `hello.nix` to a new file `icat.nix`, and update the `pname` and `version` attributes in that file:
```nix
# icat.nix
{ lib
, stdenv
, fetchzip
{
lib,
stdenv,
fetchzip,
}:
stdenv.mkDerivation {
name = "icat";
pname = "icat";
version = "v0.5";
src = fetchzip {
...
# ...
};
}
```
Now to download the source code.
`icat`'s upstream repository is hosted on [GitHub](https://github.com/atextor/icat), so you should modify the previous [source fetcher](https://nixos.org/manual/nixpkgs/stable/#chap-pkgs-fetchers), this time using `fetchFromGitHub` instead of `fetchzip`, updating the argument attribute set to the function accordingly:
`icat`'s upstream repository is hosted on [GitHub](https://github.com/atextor/icat), so you should replace the previous [source fetcher](https://nixos.org/manual/nixpkgs/stable/#chap-pkgs-fetchers).
This time you will use [`fetchFromGitHub`](https://nixos.org/manual/nixpkgs/stable/#fetchfromgithub) instead of `fetchzip`, by updating the argument attribute set to the function accordingly:
```nix
# icat.nix
{ lib
, stdenv
, fetchFromGitHub
{
lib,
stdenv,
fetchFromGitHub,
}:
stdenv.mkDerivation {
name = "icat";
pname = "icat";
version = "v0.5";
src = fetchFromGitHub {
...
# ...
};
}
```
@ -256,19 +292,27 @@ stdenv.mkDerivation {
### Fetching source from GitHub
While `fetchzip` required `url` and `sha256` arguments, more are needed for [`fetchFromGitHub`](https://nixos.org/manual/nixpkgs/stable/#fetchfromgithub).
The source is hosted on GitHub at `https://github.com/atextor/icat`, which already gives the first two arguments:
- `owner`: the name of the account controlling the repository; `owner = "atextor";`
- `repo`: the name of the repository to fetch; `repo = "icat";`
The source is URL is `https://github.com/atextor/icat`, which already gives the first two arguments:
- `owner`: the name of the account controlling the repository
You can navigate to the project's [Releases page](https://github.com/atextor/icat/releases) to find a suitable `rev`, such as the git commit hash or tag (e.g. `v1.0`) corresponding to the release you want to fetch.
```
owner = "atextor";
```
- `repo`: the name of the repository to fetch
```
repo = "icat";
``````
Navigate to the project's [Tags page](https://github.com/atextor/icat/tags) to find a suitable [Git revision](https://git-scm.com/docs/revisions) (`rev`), such as the Git commit hash or tag (e.g. `v1.0`) corresponding to the release you want to fetch.
In this case, the latest release tag is `v0.5`.
As in the `hello` example, a hash must also be supplied.
This time, instead of using `lib.fakeSha256` and letting `nix-build` report the correct one in an error, you can fetch the correct hash in the first place with the `nix-prefetch-url` command.
You need the SHA256 hash of the *contents* of the tarball (as opposed to the hash of the tarball file itself), so you will need to pass the `--unpack` and `--type sha256` arguments too:
You need the SHA256 hash of the *contents* of the tarball (as opposed to the hash of the tarball file itself).
Therefore pass the `--unpack` and `--type sha256` arguments:
```console
$ nix-prefetch-url --unpack https://github.com/atextor/icat/archive/refs/tags/v0.5.tar.gz --type sha256
@ -276,17 +320,19 @@ path is '/nix/store/p8jl1jlqxcsc7ryiazbpm7c1mqb6848b-v0.5.tar.gz'
0wyy2ksxp95vnh71ybj1bbmqd5ggp13x3mk37pzr99ljs9awy8ka
```
Now you can supply the correct hash to `fetchFromGitHub`:
Set the correct hash for `fetchFromGitHub`:
```nix
# icat.nix
{ lib
, stdenv
, fetchFromGitHub
{
lib,
stdenv,
fetchFromGitHub,
}:
stdenv.mkDerivation {
name = "icat";
pname = "icat";
version = "v0.5";
src = fetchFromGitHub {
owner = "atextor";
@ -298,7 +344,8 @@ stdenv.mkDerivation {
```
### Missing dependencies
Running `nix-build` with the new `icat` attribute, an entirely new issue is reported:
Running `nix-build` with the new `icat` attribute, an entirely new issue is reported:
```console
$ nix-build -A icat
@ -324,20 +371,23 @@ error: builder for '/nix/store/l5wz9inkvkf0qhl8kpl39vpg2xfm2qpy-icat.drv' failed
A compiler error!
The `icat` source was pulled from GitHub, and Nix tried to build what it found, but compilation failed due to a missing dependency: the `imlib2` header.
If you [search for `imlib2` on search.nixos.org](https://search.nixos.org/packages?channel=23.05&from=0&size=50&sort=relevance&type=packages&query=imlib2), you'll find that `imlib2` is already in `nixpkgs`.
If you [search for `imlib2` on search.nixos.org](https://search.nixos.org/packages?query=imlib2), you'll find that `imlib2` is already in Nixpkgs.
You can add this package to your build environment by adding `imlib2` to the set of inputs to the expression in `icat.nix`, and then adding `imlib2` to the list of `buildInputs` in `stdenv.mkDerivation`:
Add this package to your build environment by adding `imlib2` to the arguments of the function in `icat.nix`.
Then add the argument's value `imlib2` to the list of `buildInputs` in `stdenv.mkDerivation`:
```nix
# icat.nix
{ lib
, stdenv
, fetchFromGitHub
, imlib2
{
lib,
stdenv,
fetchFromGitHub,
imlib2,
}:
stdenv.mkDerivation {
name = "icat";
pname = "icat";
version = "v0.5";
src = fetchFromGitHub {
owner = "atextor";
@ -372,30 +422,99 @@ error: builder for '/nix/store/bw2d4rp2k1l5rg49hds199ma2mz36x47-icat.drv' failed
For full logs, run 'nix log /nix/store/bw2d4rp2k1l5rg49hds199ma2mz36x47-icat.drv'.
```
You can see a few warnings which should be corrected in the upstream code, but the important bit for this tutorial is `fatal error: X11/Xlib.h: No such file or directory`: another dependency is missing.
You can see a few warnings which should be corrected in the upstream code.
But the important bit for this tutorial is `fatal error: X11/Xlib.h: No such file or directory`: another dependency is missing.
## Finding packages
:::{note}
Determining from where to source a dependency is currently a somewhat-involved process: it helps to become familiar with searching the `nixpkgs` source for keywords.
Consider using `nix-locate` from the [`nix-index`](https://github.com/nix-community/nix-index) tool to find derivations that provide what you need.
:::
Determining from where to source a dependency is currently a somewhat involved, because package names don't always correspond to library or program names.
You will need the `Xlib.h` headers from the `X11` C package, the Nixpkgs derivation for which is `libX11`, available in the `xorg` package set.
There are multiple ways to figure this out:
### `search.nixos.org`
:::{tip}
The easiest way to find what you need is on search.nixos.org/packages.
:::
Unfortunately in this case, [searching for `x11`](https://search.nixos.org/packages?query=x11) produces too many irrelevant results because X11 is ubiquitous.
On the left side bar there is a list package sets, and [selecting `xorg`](https://search.nixos.org/packages?channel=23.11&buckets={%22package_attr_set%22%3A[%22xorg%22]%2C%22package_license_set%22%3A[]%2C%22package_maintainers_set%22%3A[]%2C%22package_platforms%22%3A[]}&query=x11) shows something promising.
In case all else fails, it helps to become familiar with searching the [Nixpkgs source code](https://github.com/nixos/nixpkgs) for keywords.
### Git and `rg`
To find name assignments in the source, search for `"<keyword> ="`.
For example, these are the search results for [`"x11 = "`](https://github.com/search?q=repo%3ANixOS%2Fnixpkgs+%22x11+%3D%22&type=code) or [`"libx11 ="`](https://github.com/search?q=repo%3ANixOS%2Fnixpkgs+%22libx11+%3D%22&type=code) on Github .
Or fetch a local clone of the repository and use `rg`.
Nixpkgs is huge.
Only clone the latest revision if you don't want to wait a long time:
```console
$ nix-shell -p git ripgrep
[nix-shell:~]$ git glone https://github.com/NixOS/nixpkgs --depth 1
```
To narrow down results, specify which subdirectory you want to search:
```console
[nix-shell:~]$ rg "x11 =" pkgs
pkgs/tools/X11/primus/default.nix
21: primus = if useNvidia then primusLib_ else primusLib_.override { nvidia_x11 = null; };
22: primus_i686 = if useNvidia then primusLib_i686_ else primusLib_i686_.override { nvidia_x11 = null; };
pkgs/applications/graphics/imv/default.nix
38: x11 = [ libGLU xorg.libxcb xorg.libX11 ];
pkgs/tools/X11/primus/lib.nix
14: if nvidia_x11 == null then libGL
pkgs/top-level/linux-kernels.nix
573: ati_drivers_x11 = throw "ati drivers are no longer supported by any kernel >=4.1"; # added 2021-05-18;
... <a lot more results>
```
Since `rg` is case sensitive by default,
Add `-i` to make sure you don't miss anything:
```
[nix-shell:~]$ rg -i "libx11 =" pkgs
pkgs/applications/version-management/monotone-viz/graphviz-2.0.nix
55: ++ lib.optional (libX11 == null) "--without-x";
pkgs/top-level/all-packages.nix
14191: libX11 = xorg.libX11;
pkgs/servers/x11/xorg/default.nix
1119: libX11 = callPackage ({ stdenv, pkg-config, fetchurl, xorgproto, libpthreadstubs, libxcb, xtrans, testers }: stdenv.mkDerivation (finalAttrs: {
pkgs/servers/x11/xorg/overrides.nix
147: libX11 = super.libX11.overrideAttrs (attrs: {
```
### `nix-locate`
Consider using `nix-locate` from the [`nix-index`](https://github.com/nix-community/nix-index) tool to find derivations that provide what you need.
### Adding package sets as dependencies
Add this to your derivation's input attribute set and to `buildInputs`:
```nix
# icat.nix
{ lib
, stdenv
, fetchFromGitHub
, imlib2
, xorg
{
lib,
stdenv,
fetchFromGitHub,
imlib2,
xorg,
}:
stdenv.mkDerivation {
name = "icat";
pname = "icat";
version = "v0.5";
src = fetchFromGitHub {
owner = "atextor";
@ -414,7 +533,10 @@ Only add the top-level `xorg` derivation to the input attrset, rather than the f
Because Nix is lazily-evaluated, using `xorg.libX11` means that we only include the `libX11` attribute and the derivation doesn't actually include all of `xorg` into the build context.
:::
## Fixing build failures
Run the last command again:
```console
$ nix-build -A icat
this derivation will be built:
@ -438,28 +560,36 @@ error: builder for '/nix/store/x1d79ld8jxqdla5zw2b47d2sl87mf56k-icat.drv' failed
The missing dependency error is solved, but there is now another problem: `make: *** No rule to make target 'install'. Stop.`
### `installPhase`
The `stdenv` is automatically working with the `Makefile` that comes with `icat`: you can see in the console output that `configure` and `make` are executed without issue, so the `icat` binary is compiling successfully.
`stdenv` is automatically working with the `Makefile` that comes with `icat`.
The console output showas that `configure` and `make` are executed without issue, so the `icat` binary is compiling successfully.
The failure occurs when the `stdenv` attempts to run `make install`: the `Makefile` included in the project happens to lack an `install` target, and the `README` in the `icat` repository only mentions using `make` to build the tool, leaving the installation step up to users.
The failure occurs when the `stdenv` attempts to run `make install`.
The `Makefile` included in the project happens to lack an `install` target.
The `README` in the `icat` repository only mentions using `make` to build the tool, leaving the installation step up to users.
To add this step to your derivation, use the [`installPhase` attribute](https://nixos.org/manual/nixpkgs/stable/#ssec-install-phase), which contains a list of command strings to execute to perform the installation.
To add this step to your derivation, use the [`installPhase` attribute](https://nixos.org/manual/nixpkgs/stable/#ssec-install-phase).
It contains a list of command strings that are executed to perform the installation.
Because the `make` step completes successfully, the `icat` executable is available in the build directory, and you only need to copy it from there to the output directory.
Because `make` finishes successfully, the `icat` executable is available in the build directory.
You only need to copy it from there to the output directory.
In Nix, the output directory is stored in the `$out` variable, accessible in the derivation's component scripts.
In Nix, the output directory is stored in the `$out` variable.
That variable is accessible in the derivation's [`builder` execution environment](https://nix.dev/manual/nix/2.19/language/derivations#builder-execution).
Create a `bin` directory within the `$out` directory and copy the `icat` binary there:
```nix
# icat.nix
{ lib
, stdenv
, fetchFromGitHub
, imlib2
, xorg
{
lib,
stdenv,
fetchFromGitHub,
imlib2,
xorg,
}:
stdenv.mkDerivation {
name = "icat";
pname = "icat";
version = "v0.5";
src = fetchFromGitHub {
owner = "atextor";
@ -478,35 +608,45 @@ stdenv.mkDerivation {
```
### Phases and hooks
Nixpkgs `stdenv.mkDerivation` derivations are separated into [phases](https://nixos.org/manual/nixpkgs/stable/#sec-stdenv-phases), each of which is intended to control some aspect of the build process.
You saw earlier how `stdenv.mkDerivation` expected the project's `Makefile` to have an `install` target, and failed when it didn't.
Nixpkgs `stdenv.mkDerivation` derivations are separated into [phases](https://nixos.org/manual/nixpkgs/stable/#sec-stdenv-phases).
Each is intended to control some aspect of the build process.
Earlier you observed how `stdenv.mkDerivation` expected the project's `Makefile` to have an `install` target, and failed when it didn't.
To fix this, you defined a custom `installPhase` containing instructions for copying the `icat` binary to the correct output location, in effect installing it.
Up to that point, the `stdenv.mkDerivation` automatically determined the `buildPhase` information for the `icat` package.
During derivation realisation, there are a number of shell functions ("hooks", in `nixpkgs`) which may execute in each derivation phase, which do things like set variables, source files, create directories, and so on.
During derivation realisation, there are a number of shell functions ("hooks", in Nixpkgs) which may execute in each derivation phase.
Hooks do things like set variables, source files, create directories, and so on.
These are specific to each phase, and run both before and after that phase's execution, controlling the build environment and helping to prevent environment-modifying behavior defined within packages from creating sources of nondeterminism within and between Nix derivations.
These are specific to each phase, and run both before and after that phase's execution.
They modify the build environment for common operations during the build.
It's good practice when packaging software with Nix to include calls to these hooks in the derivation phases you define, even when you don't make direct use of them; this facilitates easy [overriding](https://nixos.org/manual/nixpkgs/stable/#chap-overrides) of specific parts of the derivation later, in addition to the previously-mentioned reproducibility benefits.
It's good practice when packaging software with Nix to include calls to these hooks in the derivation phases you define, even when you don't make direct use of them.
This facilitates easy [overriding](https://nixos.org/manual/nixpkgs/stable/#chap-overrides) of specific parts of the derivation later.
And it keeps the code tidy and makes it easier to read.
You should now adjust your `installPhase` to call the appropriate hooks:
Adjust your `installPhase` to call the appropriate hooks:
```nix
# icat.nix
...
# ...
installPhase = ''
runHook preInstall
mkdir -p $out/bin
cp icat $out/bin
runHook postInstall
'';
...
# ...
```
### A successful build
Running the `nix-build` command once more will finally do what you want, and more safely than before; you can `ls` in the local directory to find a `result` symlink to a location in the Nix store:
## A successful build
Running the `nix-build` command once more will finally do what you want, repeatably.
Call `ls` in the local directory to find a `result` symlink to a location in the Nix store:
```console
$ ls
@ -524,6 +664,9 @@ default.nix hello.nix icat.nix result
## Next steps
- [Add your own new packages to Nixpkgs](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md)
- [](../contributing/how-to-contribute.md)
- [](../contributing/how-to-get-help.md)
- [](sharing-dependencies)
- [](direnv)
- [](automatic-direnv)
- [](python-dev-environment)