NixOS/nix.dev
NixOS/nix.dev
1
0
Fork 0
mirror of https://github.com/NixOS/nix.dev.git synced 2024-10-18 14:32:43 -04:00
Code Activity

Convert the remaining tutorials from .rst to .md

Browse source
This commit is contained in:
Nejc Zupan 2021-12-10 14:48:34 +00:00
parent 205e31841c
commit e5e8253278
14 changed files with 965 additions and 1035 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

143
source/tutorials/declarative-and-reproducible-developer-environments.md Normal file
View file

@ -0,0 +1,143 @@
(declarative-reproducible-envs)=
# Declarative and reproducible developer environments
In the {ref}`ad-hoc-envs` tutorial we looked at providing shell
environments for when we need a quick'n'dirty way of getting hold
of some tools.
In this tutorial we'll take a look how to create {term}`reproducible`
shell environments given a declarative configuration file called a Nix expression.
## When are declarative shell environments useful?
This is the quickest approach to getting started with Nix:
- use single command to invoke it via `nix-shell`
- it works across different operating systems (Linux / MacOS)
- you share the exact same environment with all developers
Developer environments allow you to:
- provide CLI tools, such as `psql`, `jq`, `tmux`, etc
- provide developer libraries, such as `zlib`, `openssl`, etc
- set shell environment variables
- execute bash during environment activation
## Getting started
At the top-level of your project create `shell.nix` with the following contents:
```nix
{ pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/3590f02e7d5760e52072c1a729ee2250b5560746.tar.gz") {} }:
pkgs.mkShell {
buildInputs = [
pkgs.which
pkgs.htop
pkgs.zlib
];
}
```
:::{note}
To understand the first line, read through {ref}`pinning nixpkgs tutorial <ref-pinning-nixpkgs>`.
:::
We import `nixpkgs` and make a shell with `which` and `htop` available in `$PATH`.
`zlib` provides libraries and headers in case we're compiling something against it.
To enter the environment:
```shell-session
$ nix-shell
these paths will be fetched (0.07 MiB download, 0.20 MiB unpacked):
/nix/store/072a6x7rwv5f8wr6f5s1rq8nnm767cfp-htop-2.2.0
copying path '/nix/store/072a6x7rwv5f8wr6f5s1rq8nnm767cfp-htop-2.2.0' from 'https://cache.nixos.org'...
[nix-shell:~]$
```
The command will start downloading the missing packages from the <https://cache.nixos.org> binary cache.
Once it's done, you are dropped into a new
shell. This shell provides the packages specified in `shell.nix`.
Run `htop` to confirm that it is present. Quit the program by hitting
`q`.
Now, try `which htop` to check where the `htop` command is on disk.
You should see something similar to this:
```shell-session
[nix-shell:~]$ which htop
/nix/store/y3w2i8kfdbfj9rx287ad52rahjpgv423-htop-2.2.0/bin/htop
```
## Customizing your developer environment
Given the following `shell.nix`:
```nix
{ pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/3590f02e7d5760e52072c1a729ee2250b5560746.tar.gz") {} }:
pkgs.mkShell {
buildInputs = [
pkgs.which
pkgs.htop
pkgs.zlib
];
shellHook = ''
echo hello
'';
MY_ENVIRONMENT_VARIABLE = "world";
}
```
Running `nix-shell` we observe:
```shell-session
$ nix-shell
hello
[nix-shell:~]$ echo $MY_ENVIRONMENT_VARIABLE
world
```
- The `shellHook` section allows you to execute bash while entering the shell environment.
- Any attributes passed to `mkShell` function are available once the shell environment is active.
## `direnv`: Automatically activating the environment on directory change
Besides activating the environment for each project, every time you change
`shell.nix` you need to re-enter the shell.
You can use `direnv` to automate this process for you, with the downside that each developer needs
to install it globally.
### Setting up `direnv`
1. [Install direnv with your OS package manager](https://direnv.net/docs/installation.html#from-system-packages)
2. [Hook it into your shell](https://direnv.net/docs/hook.html)
At the top-level of your project run:
```
echo "use nix" > .envrc && direnv allow
```
The next time your launch your terminal and enter the top-level of your project direnv will check for changes.
```shell-session
$ cd myproject
direnv: loading myproject/.envrc
direnv: using nix
hello
```
## Next steps
- {ref}`pinning-nixpkgs` to see different ways to import nixpkgs
- To quickly set up a Nix project read through
[Getting started Nix template](https://github.com/nix-dot-dev/getting-started-nix-template).

156
source/tutorials/declarative-and-reproducible-developer-environments.rst
View file

@ -1,156 +0,0 @@
.. _declarative-reproducible-envs:
Declarative and reproducible developer environments
===================================================
In the :ref:`ad-hoc-envs` tutorial we looked at providing shell
environments for when we need a quick'n'dirty way of getting hold
of some tools.
In this tutorial we'll take a look how to create :term:`reproducible`
shell environments given a declarative configuration file called a Nix expression.
When are declarative shell environments useful?
-----------------------------------------------
This is the quickest approach to getting started with Nix:
- use single command to invoke it via ``nix-shell``
- it works across different operating systems (Linux / MacOS)
- you share the exact same environment with all developers
Developer environments allow you to:
- provide CLI tools, such as ``psql``, ``jq``, ``tmux``, etc
- provide developer libraries, such as ``zlib``, ``openssl``, etc
- set shell environment variables
- execute bash during environment activation
Getting started
---------------
At the top-level of your project create ``shell.nix`` with the following contents:
.. code:: nix
{ pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/3590f02e7d5760e52072c1a729ee2250b5560746.tar.gz") {} }:
pkgs.mkShell {
buildInputs = [
pkgs.which
pkgs.htop
pkgs.zlib
];
}
.. note:: To understand the first line, read through :ref:`pinning nixpkgs tutorial <ref-pinning-nixpkgs>`.
We import ``nixpkgs`` and make a shell with ``which`` and ``htop`` available in ``$PATH``.
``zlib`` provides libraries and headers in case we're compiling something against it.
To enter the environment:
.. code:: shell-session
$ nix-shell
these paths will be fetched (0.07 MiB download, 0.20 MiB unpacked):
/nix/store/072a6x7rwv5f8wr6f5s1rq8nnm767cfp-htop-2.2.0
copying path '/nix/store/072a6x7rwv5f8wr6f5s1rq8nnm767cfp-htop-2.2.0' from 'https://cache.nixos.org'...
[nix-shell:~]$
The command will start downloading the missing packages from the https://cache.nixos.org binary cache.
Once it's done, you are dropped into a new
shell. This shell provides the packages specified in ``shell.nix``.
Run ``htop`` to confirm that it is present. Quit the program by hitting
``q``.
Now, try ``which htop`` to check where the ``htop`` command is on disk.
You should see something similar to this:
.. code:: shell-session
[nix-shell:~]$ which htop
/nix/store/y3w2i8kfdbfj9rx287ad52rahjpgv423-htop-2.2.0/bin/htop
Customizing your developer environment
--------------------------------------
Given the following ``shell.nix``:
.. code:: nix
{ pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/3590f02e7d5760e52072c1a729ee2250b5560746.tar.gz") {} }:
pkgs.mkShell {
buildInputs = [
pkgs.which
pkgs.htop
pkgs.zlib
];
shellHook = ''
echo hello
'';
MY_ENVIRONMENT_VARIABLE = "world";
}
Running ``nix-shell`` we observe:
.. code:: shell-session
$ nix-shell
hello
[nix-shell:~]$ echo $MY_ENVIRONMENT_VARIABLE
world
- The ``shellHook`` section allows you to execute bash while entering the shell environment.
- Any attributes passed to ``mkShell`` function are available once the shell environment is active.
``direnv``: Automatically activating the environment on directory change
------------------------------------------------------------------------
Besides activating the environment for each project, every time you change
``shell.nix`` you need to re-enter the shell.
You can use ``direnv`` to automate this process for you, with the downside that each developer needs
to install it globally.
Setting up ``direnv``
*********************
1. `Install direnv with your OS package manager <https://direnv.net/docs/installation.html#from-system-packages>`_
2. `Hook it into your shell <https://direnv.net/docs/hook.html>`_
At the top-level of your project run::
echo "use nix" > .envrc && direnv allow
The next time your launch your terminal and enter the top-level of your project direnv will check for changes.
.. code:: shell-session
$ cd myproject
direnv: loading myproject/.envrc
direnv: using nix
hello
Next steps
----------
- :ref:`pinning-nixpkgs` to see different ways to import nixpkgs
- To quickly set up a Nix project read through
`Getting started Nix template <https://github.com/nix-dot-dev/getting-started-nix-template>`_.

169
source/tutorials/deploying-nixos-using-terraform.md Normal file
View file

@ -0,0 +1,169 @@
---
html_meta:
"description lang=en": "Continuous Integration with GitHub Actions and Cachix"
"keywords": "NixOS, deployment, Terraform, AWS"
---
(deploying-nixos-using-terraform)=
# Deploying NixOS using Terraform
Assuming you're [familiar with the basics of Terraform](https://www.terraform.io/intro/index.html),
by the end of tutorial you will have provisioned an Amazon AWS instance with Terraform
and will be able to use Nix to deploy incremental changes to NixOS, running on the instance.
We'll look at how to boot a NixOS machine and how to deploy the incremental changes:
## Booting NixOS image
1. Start by providing the terraform executable:
```shell
nix-shell -p terraform
```
2. We are using [Terraform Cloud](https://app.terraform.io) as a [state/locking backend](https://www.terraform.io/docs/state/purpose.html):
```shell
terraform login
```
3. Make sure to [create an organization](https://app.terraform.io/app/organizations/new) like `myorganization` in your Terraform Cloud account.
4. Inside `myorganization` [create a workspace](https://app.terraform.io/app/cachix/workspaces/new) by choosing **CLI-driven workflow** and pick a name like `myapp`.
5. Inside your workspace, under `Settings` / `General` change Execution Mode to `Local`.
6. Inside a new directory create a `main.tf` file with the following contents. This will start an AWS instance with the NixOS image using one SSH keypair and an SSH security group:
```
terraform {
backend "remote" {
organization = "myorganization"
workspaces {
name = "myapp"
}
}
}
provider "aws" {
region = "eu-central-1"
}
module "nixos_image" {
source = "git::https://github.com/tweag/terraform-nixos.git//aws_image_nixos?ref=5f5a0408b299874d6a29d1271e9bffeee4c9ca71"
release = "20.09"
}
resource "aws_security_group" "ssh_and_egress" {
ingress {
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = [ "0.0.0.0/0" ]
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "tls_private_key" "state_ssh_key" {
algorithm = "RSA"
}
resource "local_file" "machine_ssh_key" {
sensitive_content = tls_private_key.state_ssh_key.private_key_pem
filename = "${path.module}/id_rsa.pem"
file_permission = "0600"
}
resource "aws_key_pair" "generated_key" {
key_name = "generated-key-${sha256(tls_private_key.state_ssh_key.public_key_openssh)}"
public_key = tls_private_key.state_ssh_key.public_key_openssh
}
resource "aws_instance" "machine" {
ami = module.nixos_image.ami
instance_type = "t3.micro"
security_groups = [ aws_security_group.ssh_and_egress.name ]
key_name = aws_key_pair.generated_key.key_name
root_block_device {
volume_size = 50 # GiB
}
}
output "public_dns" {
value = aws_instance.machine.public_dns
}
```
The only NixOS specific snippet is:
```
module "nixos_image" {
source = "git::https://github.com/tweag/terraform-nixos.git/aws_image_nixos?ref=5f5a0408b299874d6a29d1271e9bffeee4c9ca71"
release = "20.09"
}
```
:::{note}
The `aws_image_nixos` module will return an NixOS AMI given a [NixOS release number](https://status.nixos.org)
so that `aws_instance` resource can reference the AMI in [instance_type](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance#instance_type) argument.
:::
5. Make sure to [configure AWS credentials](https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication).
6. Applying the Terraform configuration should get you a running NixOS:
```shell
terraform init
terraform apply
```
## Deploying NixOS changes
Once the AWS instance is running an NixOS image via Terraform, we can teach Terraform to always build
the latest NixOS configuration and apply those changes to your instance.
1. Create `configuration.nix` with the following contents:
```nix
{ config, lib, pkgs, ... }: {
imports = [ <nixpkgs/nixos/modules/virtualisation/amazon-image.nix> ];
# Open https://search.nixos.org/options for all options
}
```
2. Append the following snippet to your `main.tf`:
```
module "deploy_nixos" {
source = "git::https://github.com/tweag/terraform-nixos.git//deploy_nixos?ref=5f5a0408b299874d6a29d1271e9bffeee4c9ca71"
nixos_config = "${path.module}/configuration.nix"
target_host = aws_instance.machine.public_ip
ssh_private_key_file = local_file.machine_ssh_key.filename
ssh_agent = false
}
```
3. Deploy:
```shell
terraform init
terraform apply
```
## Caveats
- The `deploy_nixos` module requires NixOS to be installed on the target machine and Nix on the host machine.
- The `deploy_nixos` module doesn't work when the client and target architectures are different (unless you use [distributed builds](https://nixos.org/manual/nix/unstable/advanced-topics/distributed-builds.html)).
- If you need to inject a value into Nix, there is no elegant solution.
- Each machine is evaluated separately, so note that your memory requirements will grow linearly with the number of machines.
## Next steps
- It's possible to [switch to use Google Compute Engine provider](https://github.com/tweag/terraform-nixos/tree/master/google_image_nixos#readme).
- [deploy_nixos module](https://github.com/tweag/terraform-nixos/tree/master/deploy_nixos#readme) supports a number arguments, for example to upload keys, etc.

186
source/tutorials/deploying-nixos-using-terraform.rst
View file

@ -1,186 +0,0 @@
.. _deploying-nixos-using-terraform:
.. meta::
:description: Continuous Integration with GitHub Actions and Cachix
:keywords: NixOS, deployment, Terraform, AWS
Deploying NixOS using Terraform
===============================
Assuming you're `familiar with the basics of Terraform <https://www.terraform.io/intro/index.html>`_,
by the end of tutorial you will have provisioned an Amazon AWS instance with Terraform
and will be able to use Nix to deploy incremental changes to NixOS, running on the instance.
We'll look at how to boot a NixOS machine and how to deploy the incremental changes:
Booting NixOS image
-------------------
1. Start by providing the terraform executable:
.. code:: shell
nix-shell -p terraform
2. We are using `Terraform Cloud <https://app.terraform.io>`_ as a `state/locking backend <https://www.terraform.io/docs/state/purpose.html>`_:
.. code:: shell
terraform login
3. Make sure to `create an organization <https://app.terraform.io/app/organizations/new>`_ like ``myorganization`` in your Terraform Cloud account.
4. Inside ``myorganization`` `create a workspace <https://app.terraform.io/app/cachix/workspaces/new>`_ by choosing **CLI-driven workflow** and pick a name like ``myapp``.
5. Inside your workspace, under ``Settings`` / ``General`` change Execution Mode to ``Local``.
6. Inside a new directory create a ``main.tf`` file with the following contents. This will start an AWS instance with the NixOS image using one SSH keypair and an SSH security group:
.. code::
terraform {
backend "remote" {
organization = "myorganization"
workspaces {
name = "myapp"
}
}
}
provider "aws" {
region = "eu-central-1"
}
module "nixos_image" {
source = "git::https://github.com/tweag/terraform-nixos.git//aws_image_nixos?ref=5f5a0408b299874d6a29d1271e9bffeee4c9ca71"
release = "20.09"
}
resource "aws_security_group" "ssh_and_egress" {
ingress {
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = [ "0.0.0.0/0" ]
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "tls_private_key" "state_ssh_key" {
algorithm = "RSA"
}
resource "local_file" "machine_ssh_key" {
sensitive_content = tls_private_key.state_ssh_key.private_key_pem
filename = "${path.module}/id_rsa.pem"
file_permission = "0600"
}
resource "aws_key_pair" "generated_key" {
key_name = "generated-key-${sha256(tls_private_key.state_ssh_key.public_key_openssh)}"
public_key = tls_private_key.state_ssh_key.public_key_openssh
}
resource "aws_instance" "machine" {
ami = module.nixos_image.ami
instance_type = "t3.micro"
security_groups = [ aws_security_group.ssh_and_egress.name ]
key_name = aws_key_pair.generated_key.key_name
root_block_device {
volume_size = 50 # GiB
}
}
output "public_dns" {
value = aws_instance.machine.public_dns
}
The only NixOS specific snippet is:
.. code::
module "nixos_image" {
source = "git::https://github.com/tweag/terraform-nixos.git/aws_image_nixos?ref=5f5a0408b299874d6a29d1271e9bffeee4c9ca71"
release = "20.09"
}
.. note::
The ``aws_image_nixos`` module will return an NixOS AMI given a `NixOS release number <https://status.nixos.org>`_
so that ``aws_instance`` resource can reference the AMI in `instance_type <https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance#instance_type>`_ argument.
5. Make sure to `configure AWS credentials <https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication>`_.
6. Applying the Terraform configuration should get you a running NixOS:
.. code:: shell
terraform init
terraform apply
Deploying NixOS changes
-----------------------
Once the AWS instance is running an NixOS image via Terraform, we can teach Terraform to always build
the latest NixOS configuration and apply those changes to your instance.
1. Create ``configuration.nix`` with the following contents:
.. code:: nix
{ config, lib, pkgs, ... }: {
imports = [ <nixpkgs/nixos/modules/virtualisation/amazon-image.nix> ];
# Open https://search.nixos.org/options for all options
}
2. Append the following snippet to your ``main.tf``:
.. code::
module "deploy_nixos" {
source = "git::https://github.com/tweag/terraform-nixos.git//deploy_nixos?ref=5f5a0408b299874d6a29d1271e9bffeee4c9ca71"
nixos_config = "${path.module}/configuration.nix"
target_host = aws_instance.machine.public_ip
ssh_private_key_file = local_file.machine_ssh_key.filename
ssh_agent = false
}
3. Deploy:
.. code:: shell
terraform init
terraform apply
Caveats
-------
- The ``deploy_nixos`` module requires NixOS to be installed on the target machine and Nix on the host machine.
- The ``deploy_nixos`` module doesn't work when the client and target architectures are different (unless you use `distributed builds <https://nixos.org/manual/nix/unstable/advanced-topics/distributed-builds.html>`_).
- If you need to inject a value into Nix, there is no elegant solution.
- Each machine is evaluated separately, so note that your memory requirements will grow linearly with the number of machines.
Next steps
----------
- It's possible to `switch to use Google Compute Engine provider <https://github.com/tweag/terraform-nixos/tree/master/google_image_nixos#readme>`_.
- `deploy_nixos module <https://github.com/tweag/terraform-nixos/tree/master/deploy_nixos#readme>`_ supports a number arguments, for example to upload keys, etc.

82
source/tutorials/dev-environment.md Normal file
View file

@ -0,0 +1,82 @@
# Setup a development environment
As an exercise, let us build a Python web application using the Flask
web framework.
Create a new file `default.nix`. This file is conventionally used for
specifying packages:
```nix
{ pkgs ? import <nixpkgs> {} }:
pkgs.python3Packages.buildPythonApplication {
pname = "myapp";
src = ./.;
version = "0.1";
propagatedBuildInputs = [ pkgs.python3Packages.flask ];
}
```
You will also need a simple Flask app as `myapp.py`:
```python
#! /usr/bin/env python
from flask import Flask
app = Flask(__name__)
@app.route("/")
def hello():
return "Hello, Nix!"
def run():
app.run(host="0.0.0.0")
if __name__ == "__main__":
run()
```
and a `setup.py` script:
```python
from setuptools import setup
setup(
name='myapp',
version='0.1',
py_modules=['myapp'],
entry_points={
'console_scripts': ['myapp = myapp:run']
},
)
```
Now build the package with:
```bash
nix-build
```
This will create a symbolic link `result` to our package's path in the
Nix store, which looks like
`/nix/store/6i4l781jwk5vbia8as32637207kgkllj-myapp-0.1`. Look around
to see what is inside.
You may notice we can run the application from the package like
`./result/bin/myapp.py`. We can still use the `default.nix` as a
shell environment to get the same result:
```bash
nix-shell default.nix
python3 myapp.py
```
In this context, Nix takes on the role that you would otherwise use pip
or virtualenv for. Nix installs required dependencies and separates the
environment from others on your system.
You can check this Nix configuration into version control and share it
with others to make sure you are all running the same software.
Especially with many dependencies this is a great way to prevent
configuration drift between different team members & contributors.

83
source/tutorials/dev-environment.rst
View file

@ -1,83 +0,0 @@
Setup a development environment
===============================
As an exercise, let us build a Python web application using the Flask
web framework.
Create a new file ``default.nix``. This file is conventionally used for
specifying packages:
.. code:: nix
{ pkgs ? import <nixpkgs> {} }:
pkgs.python3Packages.buildPythonApplication {
pname = "myapp";
src = ./.;
version = "0.1";
propagatedBuildInputs = [ pkgs.python3Packages.flask ];
}
You will also need a simple Flask app as ``myapp.py``:
.. code:: python
#! /usr/bin/env python
from flask import Flask
app = Flask(__name__)
@app.route("/")
def hello():
return "Hello, Nix!"
def run():
app.run(host="0.0.0.0")
if __name__ == "__main__":
run()
and a ``setup.py`` script:
.. code:: python
from setuptools import setup
setup(
name='myapp',
version='0.1',
py_modules=['myapp'],
entry_points={
'console_scripts': ['myapp = myapp:run']
},
)
Now build the package with:
.. code:: bash
nix-build
This will create a symbolic link ``result`` to our package's path in the
Nix store, which looks like
``/nix/store/6i4l781jwk5vbia8as32637207kgkllj-myapp-0.1``. Look around
to see what is inside.
You may notice we can run the application from the package like
``./result/bin/myapp.py``. We can still use the ``default.nix`` as a
shell environment to get the same result:
.. code:: bash
nix-shell default.nix
python3 myapp.py
In this context, Nix takes on the role that you would otherwise use pip
or virtualenv for. Nix installs required dependencies and separates the
environment from others on your system.
You can check this Nix configuration into version control and share it
with others to make sure you are all running the same software.
Especially with many dependencies this is a great way to prevent
configuration drift between different team members & contributors.

74
source/tutorials/install-nix.md Normal file
View file

@ -0,0 +1,74 @@
(install-nix)=
# Install Nix
## Linux
Install Nix on via the recommended [multi-user installation](https://nixos.org/manual/nix/stable/installation/multi-user.html):
```bash
sh <(curl -L https://nixos.org/nix/install) --daemon
```
:::{note}
For security you may want to [verify the installation script] using GPG signatures.
:::
## macOS
Install Nix via the recommended [multi-user installation](https://nixos.org/manual/nix/stable/installation/multi-user.html):
```bash
sh <(curl -L https://nixos.org/nix/install) --darwin-use-unencrypted-nix-store-volume --daemon
```
:::{note}
For security you may want to [verify the installation script] using GPG signatures.
:::
## Windows (WSL2)
Install Nix via the recommended [single-user installation](https://nixos.org/manual/nix/stable/installation/single-user.html):
```bash
sh <(curl -L https://nixos.org/nix/install) --no-daemon
```
:::{note}
For security you may want to [verify the installation script] using GPG signatures.
:::
## Docker
Start a Docker shell with Nix:
```bash
$ docker run -it nixos/nix
```
Or start a Docker shell with Nix exposing a `workdir` directory:
```bash
$ mkdir workdir
$ docker run -it -v $(pwd)/workdir:/workdir nixos/nix
```
The `workdir` example from above can be also used to start hacking on nixpkgs:
```bash
$ git clone git@github.com:NixOS/nixpkgs
$ docker run -it -v $(pwd)/nixpkgs:/nixpkgs nixos/nix
docker> nix-build -I nixpkgs=/nixpkgs -A hello
docker> find ./result # this symlink points to the build package
```
## Verify installation
Check that the installation by opening **a new terminal** and typing:
```bash
$ nix-env --version
nix-env (Nix) 2.3.15
```
[verify the installation script]: https://nixos.org/download.html#nix-verify-installation

85
source/tutorials/install-nix.rst
View file

@ -1,85 +0,0 @@
.. _install-nix:
Install Nix
===========
Linux
-----
Install Nix on via the recommended `multi-user installation <https://nixos.org/manual/nix/stable/installation/multi-user.html>`_:
.. code:: bash
sh <(curl -L https://nixos.org/nix/install) --daemon
.. note::
For security you may want to `verify the installation script`_ using GPG signatures.
macOS
-----
Install Nix via the recommended `multi-user installation <https://nixos.org/manual/nix/stable/installation/multi-user.html>`_:
.. code:: bash
sh <(curl -L https://nixos.org/nix/install) --darwin-use-unencrypted-nix-store-volume --daemon
.. note::
For security you may want to `verify the installation script`_ using GPG signatures.
Windows (WSL2)
--------------
Install Nix via the recommended `single-user installation <https://nixos.org/manual/nix/stable/installation/single-user.html>`_:
.. code:: bash
sh <(curl -L https://nixos.org/nix/install) --no-daemon
.. note::
For security you may want to `verify the installation script`_ using GPG signatures.
Docker
------
Start a Docker shell with Nix:
.. code:: bash
$ docker run -it nixos/nix
Or start a Docker shell with Nix exposing a ``workdir`` directory:
.. code:: bash
$ mkdir workdir
$ docker run -it -v $(pwd)/workdir:/workdir nixos/nix
The ``workdir`` example from above can be also used to start hacking on nixpkgs:
.. code:: bash
$ git clone git@github.com:NixOS/nixpkgs
$ docker run -it -v $(pwd)/nixpkgs:/nixpkgs nixos/nix
docker> nix-build -I nixpkgs=/nixpkgs -A hello
docker> find ./result # this symlink points to the build package
Verify installation
-------------------
Check that the installation by opening **a new terminal** and typing:
.. code:: bash
$ nix-env --version
nix-env (Nix) 2.3.15
.. _verify the installation script: https://nixos.org/download.html#nix-verify-installation

189
source/tutorials/installing-nixos-on-a-raspberry-pi.md Normal file
View file

@ -0,0 +1,189 @@
---
html_meta:
"description lang=en": "Installing NixOS on a Raspberry Pi"
"keywords": "Raspberry Pi, rpi, NixOS, installation, image, tutorial"
---
# Installing NixOS on a Raspberry Pi
This tutorial assumes [Raspberry P 4 Model B with 4GB RAM](https://www.raspberrypi.org/products/raspberry-pi-4-model-b/).
Before starting this tutorial, make sure you have
[all necessary hardware](https://projects.raspberrypi.org/en/projects/raspberry-pi-setting-up/1):
- HDMI cable/adapter.
- 8GB+ SD card.
- SD card reader in case your machine doesn't have an SD slot.
- Power cable for your Raspberry Pi.
- USB keyboard.
:::{note}
This tutorial was written for the Raspberry Pi 4B. Using a previous supported revision, like the 3B or 3B+, is possible with tweaks.
:::
## Booting NixOS live image
:::{note}
Booting from USB may require an EEPROM firmware upgrade. This tutorial boots from an SD card to avoid such hiccups.
:::
Prepare the AArch64 image on your laptop:
```shell-session
$ nix-shell -p wget zstd
$ wget https://hydra.nixos.org/build/160738647/download/1/nixos-sd-image-22.05pre335501.c71f061c68b-aarch64-linux.img.zst
$ unzstd -d nixos-sd-image-22.05pre335501.c71f061c68b-aarch64-linux.img.zst
$ dmesg --follow
```
:::{note}
You can pick a newer image by going to [Hydra job](https://hydra.nixos.org/job/nixos/trunk-combined/nixos.sd_image.aarch64-linux),
clicking on a build and copying the link to the build product image.
:::
Your terminal should be printing kernel messages as they come in.
Plug in your SD card and your terminal should print what device it got assigned, for example `/dev/sdX`.
Press `ctrl-c` to stop `dmesg --follow`.
Copy NixOS to your SD card by replacing `sdX` with the name of your device:
```shell-session
sudo dd if=nixos-sd-image-22.05pre335501.c71f061c68b-aarch64-linux.img of=/dev/sdX bs=4096 conv=fsync status=progress
```
Once that command exits, **move the SD card into your Raspberry Pi and power it on**.
You should be greeted with a fresh shell!
In case the image doesn't boot, it's worth [updating the firmware](https://www.raspberrypi.org/documentation/computers/raspberry-pi.html#updating-the-bootloader)
and retry booting the image again.
## Getting internet connection
Run `sudo -i` to get a root shell for the rest of the tutorial.
At this point we'll need internet connection. If you can use an ethernet cable, plug it in.
In case you're connecting to a wifi run `iwconfig` to see what is the name of your wireless
network interface. In case it's `wlan0` replace `SSID` and `passphrase` with your data and run:
```shell-session
# wpa_supplicant -B -i wlan0 -c <(wpa_passphrase 'SSID' 'passphrase') &
```
Once you see in your terminal that connection is established, run `host nixos.org` to
check that DNS resolves correctly.
In case you've made a typo, run `pkill wpa_supplicant` and start over.
## Updating firmware
To benefit from updates and bug fixes from the vendor, we'll start by updating Raspberry Pi firmware:
```shell-session
# nix-shell -p raspberrypi-eeprom
# mount /dev/disk/by-label/FIRMWARE /mnt
# BOOTFS=/mnt FIRMWARE_RELEASE_STATUS=stable rpi-eeprom-update -d -a
```
## Installing NixOS
For initial installation we'll install [XFCE](https://www.xfce.org/) desktop environment
with user `guest` and SSH daemon.
```nix
{ config, pkgs, lib, ... }:
let
user = "guest";
password = "guest";
SSID = "mywifi";
SSIDpassword = "mypassword";
interface = "wlan0";
hostname = "myhostname";
in {
imports = ["${fetchTarball "https://github.com/NixOS/nixos-hardware/archive/936e4649098d6a5e0762058cb7687be1b2d90550.tar.gz" }/raspberry-pi/4"];
fileSystems = {
"/" = {
device = "/dev/disk/by-label/NIXOS_SD";
fsType = "ext4";
options = [ "noatime" ];
};
};
networking = {
hostName = hostname;
wireless = {
enable = true;
networks."${SSID}".psk = SSIDpassword;
interfaces = [ interface ];
};
};
environment.systemPackages = with pkgs; [ vim ];
services.openssh.enable = true;
users = {
mutableUsers = false;
users."${user}" = {
isNormalUser = true;
password = password;
extraGroups = [ "wheel" ];
};
};
# Enable GPU acceleration
hardware.raspberry-pi."4".fkms-3d.enable = true;
services.xserver = {
enable = true;
displayManager.lightdm.enable = true;
desktopManager.xfce.enable = true;
};
hardware.pulseaudio.enable = true;
}
```
To save time on typing the whole configuration, download it:
```shell-session
# curl -L https://tinyurl.com/nixos-rpi4-tutorial > /etc/nixos/configuration.nix
```
At the top of `/etc/nixos/configuration.nix` there are a few variables that you want to configure,
most important being your wifi connection details, this time specified in declarative way.
Once you're ready to install NixOS:
```shell-session
# nixos-install --root /
# reboot
```
In case your system doesn't boot, select the oldest configuration in the bootloader menu to get back to live image and start over.
## Making changes
It booted, congratulations!
To make further changes to the configuration, [search through NixOS options](https://search.nixos.org/options),
edit `/etc/nixos/configuration.nix` and update your system:
```shell-session
$ sudo -i
# nixos-rebuild switch
```
## Next steps
- Once you have successfully running OS, try upgrading it with `nixos-rebuild switch --upgrade`
and reboot to the old configuration if something broke.
- To tweak bootloader options affecting hardware, [see config.txt options](https://www.raspberrypi.org/documentation/configuration/config-txt/)
and change the options by running `mount /dev/disk/by-label/FIRMWARE /mnt` and opening `/mnt/config.txt`.
- To see the power of declarative configuration, try replacing `xfce` with `kodi` in `/etc/nixos/configuration.nix` and `reboot`.

199
source/tutorials/installing-nixos-on-a-raspberry-pi.rst
View file

@ -1,199 +0,0 @@
Installing NixOS on a Raspberry Pi
==================================
.. meta::
:description: Installing NixOS on a Raspberry Pi
:keywords: Raspberry Pi, rpi, NixOS, installation, image, tutorial
This tutorial assumes `Raspberry P 4 Model B with 4GB RAM <https://www.raspberrypi.org/products/raspberry-pi-4-model-b/>`_.
Before starting this tutorial, make sure you have
`all necessary hardware <https://projects.raspberrypi.org/en/projects/raspberry-pi-setting-up/1>`_:
- HDMI cable/adapter.
- 8GB+ SD card.
- SD card reader in case your machine doesn't have an SD slot.
- Power cable for your Raspberry Pi.
- USB keyboard.
.. note::
This tutorial was written for the Raspberry Pi 4B. Using a previous supported revision, like the 3B or 3B+, is possible with tweaks.
Booting NixOS live image
------------------------
.. note:: Booting from USB may require an EEPROM firmware upgrade. This tutorial boots from an SD card to avoid such hiccups.
Prepare the AArch64 image on your laptop:
.. code:: shell-session
$ nix-shell -p wget zstd
$ wget https://hydra.nixos.org/build/160738647/download/1/nixos-sd-image-22.05pre335501.c71f061c68b-aarch64-linux.img.zst
$ unzstd -d nixos-sd-image-22.05pre335501.c71f061c68b-aarch64-linux.img.zst
$ dmesg --follow
.. note::
You can pick a newer image by going to `Hydra job <https://hydra.nixos.org/job/nixos/trunk-combined/nixos.sd_image.aarch64-linux>`_,
clicking on a build and copying the link to the build product image.
Your terminal should be printing kernel messages as they come in.
Plug in your SD card and your terminal should print what device it got assigned, for example ``/dev/sdX``.
Press ``ctrl-c`` to stop ``dmesg --follow``.
Copy NixOS to your SD card by replacing ``sdX`` with the name of your device:
.. code:: shell-session
sudo dd if=nixos-sd-image-22.05pre335501.c71f061c68b-aarch64-linux.img of=/dev/sdX bs=4096 conv=fsync status=progress
Once that command exits, **move the SD card into your Raspberry Pi and power it on**.
You should be greeted with a fresh shell!
In case the image doesn't boot, it's worth `updating the firmware <https://www.raspberrypi.org/documentation/computers/raspberry-pi.html#updating-the-bootloader>`_
and retry booting the image again.
Getting internet connection
---------------------------
Run ``sudo -i`` to get a root shell for the rest of the tutorial.
At this point we'll need internet connection. If you can use an ethernet cable, plug it in.
In case you're connecting to a wifi run ``iwconfig`` to see what is the name of your wireless
network interface. In case it's ``wlan0`` replace ``SSID`` and ``passphrase`` with your data and run:
.. code:: shell-session
# wpa_supplicant -B -i wlan0 -c <(wpa_passphrase 'SSID' 'passphrase') &
Once you see in your terminal that connection is established, run ``host nixos.org`` to
check that DNS resolves correctly.
In case you've made a typo, run ``pkill wpa_supplicant`` and start over.
Updating firmware
-----------------
To benefit from updates and bug fixes from the vendor, we'll start by updating Raspberry Pi firmware:
.. code:: shell-session
# nix-shell -p raspberrypi-eeprom
# mount /dev/disk/by-label/FIRMWARE /mnt
# BOOTFS=/mnt FIRMWARE_RELEASE_STATUS=stable rpi-eeprom-update -d -a
Installing NixOS
----------------
For initial installation we'll install `XFCE <https://www.xfce.org/>`_ desktop environment
with user ``guest`` and SSH daemon.
.. code:: nix
{ config, pkgs, lib, ... }:
let
user = "guest";
password = "guest";
SSID = "mywifi";
SSIDpassword = "mypassword";
interface = "wlan0";
hostname = "myhostname";
in {
imports = ["${fetchTarball "https://github.com/NixOS/nixos-hardware/archive/936e4649098d6a5e0762058cb7687be1b2d90550.tar.gz" }/raspberry-pi/4"];
fileSystems = {
"/" = {
device = "/dev/disk/by-label/NIXOS_SD";
fsType = "ext4";
options = [ "noatime" ];
};
};
networking = {
hostName = hostname;
wireless = {
enable = true;
networks."${SSID}".psk = SSIDpassword;
interfaces = [ interface ];
};
};
environment.systemPackages = with pkgs; [ vim ];
services.openssh.enable = true;
users = {
mutableUsers = false;
users."${user}" = {
isNormalUser = true;
password = password;
extraGroups = [ "wheel" ];
};
};
# Enable GPU acceleration
hardware.raspberry-pi."4".fkms-3d.enable = true;
services.xserver = {
enable = true;
displayManager.lightdm.enable = true;
desktopManager.xfce.enable = true;
};
hardware.pulseaudio.enable = true;
}
To save time on typing the whole configuration, download it:
.. code:: shell-session
# curl -L https://tinyurl.com/nixos-rpi4-tutorial > /etc/nixos/configuration.nix
At the top of `/etc/nixos/configuration.nix` there are a few variables that you want to configure,
most important being your wifi connection details, this time specified in declarative way.
Once you're ready to install NixOS:
.. code:: shell-session
# nixos-install --root /
# reboot
In case your system doesn't boot, select the oldest configuration in the bootloader menu to get back to live image and start over.
Making changes
--------------
It booted, congratulations!
To make further changes to the configuration, `search through NixOS options <https://search.nixos.org/options>`_,
edit ``/etc/nixos/configuration.nix`` and update your system:
.. code:: shell-session
$ sudo -i
# nixos-rebuild switch
Next steps
----------
- Once you have successfully running OS, try upgrading it with `nixos-rebuild switch --upgrade`
and reboot to the old configuration if something broke.
- To tweak bootloader options affecting hardware, `see config.txt options <https://www.raspberrypi.org/documentation/configuration/config-txt/>`_
and change the options by running ``mount /dev/disk/by-label/FIRMWARE /mnt`` and opening ``/mnt/config.txt``.
- To see the power of declarative configuration, try replacing ``xfce`` with ``kodi`` in ``/etc/nixos/configuration.nix`` and ``reboot``.

218
source/tutorials/integration-testing-using-virtual-machines.md Normal file
View file

@ -0,0 +1,218 @@
(integration-testing-vms)=
# Integration testing using virtual machines (VMs)
One of the most powerful features in the Nix ecosystem is **the ability
to provide a set of declarative NixOS configurations and use a simple
Python interface** to interact with them using [QEMU](https://www.qemu.org/)
as the backend.
Those tests are widely used to ensure that NixOS works as intended, so in general they are called **NixOS tests**.
They can be written and launched outside of NixOS, on any Linux machine (with
[MacOS support coming soon](https://github.com/NixOS/nixpkgs/issues/108984)).
Integration tests are reproducible due to the design properties of Nix,
making them a valuable part of a Continuous Integration (CI) pipeline.
## Testing a typical web application backed by PostgreSQL
This tutorial follows [PostgREST tutorial](https://postgrest.org/en/stable/tutorials/tut0.html),
a generic [RESTful API](https://restfulapi.net/) for PostgreSQL.
If you skim over the official tutorial, you'll notice there's quite a bit of setup
in order to test if all the steps work.
We are going to set up:
- A VM named `server` running postgreSQL and postgREST.
- A VM named `client` running HTTP client queries using `curl`.
- A `testScript` orchestrating testing logic between `client` and `server`.
## Writing the test
Create `postgrest.nix`:
% TODO: highlight nix https://github.com/pygments/pygments/issues/1793
```{code-block}
:linenos: true
let
# Pin nixpkgs, see pinning tutorial for more details
nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/archive/0f8f64b54ed07966b83db2f20c888d5e035012ef.tar.gz";
pkgs = import nixpkgs {};
# Single source of truth for all tutorial constants
database = "postgres";
schema = "api";
table = "todos";
username = "authenticator";
password = "mysecretpassword";
webRole = "web_anon";
postgrestPort = 3000;
# NixOS module shared between server and client
sharedModule = {
# Since it's common for CI not to have $DISPLAY available, we have to explicitly tell the tests "please don't expect any screen available"
virtualisation.graphics = false;
};
in pkgs.nixosTest ({
# NixOS tests are run inside a virtual machine, and here we specify system of the machine.
system = "x86_64-linux";
nodes = {
server = { config, pkgs, ... }: {
imports = [ sharedModule ];
networking.firewall.allowedTCPPorts = [ postgrestPort ];
services.postgresql = {
enable = true;
initialScript = pkgs.writeText "initialScript.sql" ''
create schema ${schema};
create table ${schema}.${table} (
id serial primary key,
done boolean not null default false,
task text not null,
due timestamptz
);
insert into ${schema}.${table} (task) values ('finish tutorial 0'), ('pat self on back');
create role ${webRole} nologin;
grant usage on schema ${schema} to ${webRole};
grant select on ${schema}.${table} to ${webRole};
create role ${username} inherit login password '${password}';
grant ${webRole} to ${username};
'';
};
users = {
mutableUsers = false;
users = {
# For ease of debugging the VM as the `root` user
root.password = "";
# Create a system user that matches the database user so that we
# can use peer authentication. The tutorial defines a password,
# but it's not necessary.
"${username}".isSystemUser = true;
};
};
systemd.services.postgrest = {
wantedBy = [ "multi-user.target" ];
after = [ "postgresql.service" ];
script =
let
configuration = pkgs.writeText "tutorial.conf" ''
db-uri = "postgres://${username}:${password}@localhost:${toString config.services.postgresql.port}/${database}"
db-schema = "${schema}"
db-anon-role = "${username}"
'';
in "${pkgs.haskellPackages.postgrest}/bin/postgrest ${configuration}";
serviceConfig.User = username;
};
};
client = {
imports = [ sharedModule ];
};
};
# Disable linting for simpler debugging of the testScript
skipLint = true;
testScript = ''
import json
import sys
start_all()
server.wait_for_open_port(${toString postgrestPort})
expected = [
{"id": 1, "done": False, "task": "finish tutorial 0", "due": None},
{"id": 2, "done": False, "task": "pat self on back", "due": None},
]
actual = json.loads(
client.succeed(
"${pkgs.curl}/bin/curl http://server:${toString postgrestPort}/${table}"
)
)
assert expected == actual, "table query returns expected content"
'';
})
```
A few notes:
- Between the machines defined inside the `nodes` attribute, hostnames
are resolved based on their attribute names. In this case we have `client` and `server`.
- The testing framework exposes a wide set of operations used inside the `testScript`.
A full set of testing operations is part of
[VM testing operations API Reference](https://nixos.org/manual/nixos/stable/index.html#sec-nixos-tests).
## Running tests
To set up all machines and execute the test script:
```shell-session
$ nix-build postgrest.nix
```
You'll notice an error message if something goes wrong.
In case the tests succeed, you should see at the end:
```shell-session
...
test script finished in 10.96s
cleaning up
killing client (pid 10)
killing server (pid 22)
(0.00 seconds)
/nix/store/bx7z3imvxxpwkkza10vb23czhw7873w2-vm-test-run-unnamed
```
## Developing and debugging tests
When developing tests or when something breaks, it's useful to interactively fiddle
with the script or access a terminal for a machine.
To interactively start a Python session with a testing framework:
```shell-session
$ $(nix-build -A driver postgrest.nix)/bin/nixos-test-driver
...
starting VDE switch for network 1
>>>
```
You can run [any of the testing operations](https://nixos.org/manual/nixos/stable/index.html#sec-nixos-tests).
The `testScript` attribute from our `postgrest.nix` definition can be executed with `test_script()` function.
To start all machines and enter a telnet terminal to a specific machine:
```shell-session
>>> start_all()
...
>>> server.shell_interact()
server: Terminal is ready (there is no prompt):
uname -a
Linux server 5.10.37 #1-NixOS SMP Fri May 14 07:50:46 UTC 2021 x86_64 GNU/Linux
```
## Next steps
- Running integration tests on CI requires hardware acceleration, which many CIs do not support.
To run integration tests on {ref}`GitHub Actions <github-actions>` see
[how to disable hardware acceleration](https://github.com/cachix/install-nix-action#how-can-i-run-nixos-tests).
- NixOS comes with a large set of tests that serve also as educational examples. A good inspiration is [Matrix bridging with an IRC](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/matrix-appservice-irc.nix).

233
source/tutorials/integration-testing-using-virtual-machines.rst
View file

@ -1,233 +0,0 @@
.. _integration-testing-vms:
Integration testing using virtual machines (VMs)
================================================
One of the most powerful features in the Nix ecosystem is **the ability
to provide a set of declarative NixOS configurations and use a simple
Python interface** to interact with them using `QEMU <https://www.qemu.org/>`_
as the backend.
Those tests are widely used to ensure that NixOS works as intended, so in general they are called **NixOS tests**.
They can be written and launched outside of NixOS, on any Linux machine (with
`MacOS support coming soon <https://github.com/NixOS/nixpkgs/issues/108984>`_).
Integration tests are reproducible due to the design properties of Nix,
making them a valuable part of a Continuous Integration (CI) pipeline.
Testing a typical web application backed by PostgreSQL
------------------------------------------------------
This tutorial follows `PostgREST tutorial <https://postgrest.org/en/stable/tutorials/tut0.html>`_,
a generic `RESTful API <https://restfulapi.net/>`_ for PostgreSQL.
If you skim over the official tutorial, you'll notice there's quite a bit of setup
in order to test if all the steps work.
We are going to set up:
- A VM named ``server`` running postgreSQL and postgREST.
- A VM named ``client`` running HTTP client queries using ``curl``.
- A ``testScript`` orchestrating testing logic between ``client`` and ``server``.
Writing the test
----------------
Create ``postgrest.nix``:
.. TODO: highlight nix https://github.com/pygments/pygments/issues/1793
.. code-block::
:linenos:
let
# Pin nixpkgs, see pinning tutorial for more details
nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/archive/0f8f64b54ed07966b83db2f20c888d5e035012ef.tar.gz";
pkgs = import nixpkgs {};
# Single source of truth for all tutorial constants
database = "postgres";
schema = "api";
table = "todos";
username = "authenticator";
password = "mysecretpassword";
webRole = "web_anon";
postgrestPort = 3000;
# NixOS module shared between server and client
sharedModule = {
# Since it's common for CI not to have $DISPLAY available, we have to explicitly tell the tests "please don't expect any screen available"
virtualisation.graphics = false;
};
in pkgs.nixosTest ({
# NixOS tests are run inside a virtual machine, and here we specify system of the machine.
system = "x86_64-linux";
nodes = {
server = { config, pkgs, ... }: {
imports = [ sharedModule ];
networking.firewall.allowedTCPPorts = [ postgrestPort ];
services.postgresql = {
enable = true;
initialScript = pkgs.writeText "initialScript.sql" ''
create schema ${schema};
create table ${schema}.${table} (
id serial primary key,
done boolean not null default false,
task text not null,
due timestamptz
);
insert into ${schema}.${table} (task) values ('finish tutorial 0'), ('pat self on back');
create role ${webRole} nologin;
grant usage on schema ${schema} to ${webRole};
grant select on ${schema}.${table} to ${webRole};
create role ${username} inherit login password '${password}';
grant ${webRole} to ${username};
'';
};
users = {
mutableUsers = false;
users = {
# For ease of debugging the VM as the `root` user
root.password = "";
# Create a system user that matches the database user so that we
# can use peer authentication. The tutorial defines a password,
# but it's not necessary.
"${username}".isSystemUser = true;
};
};
systemd.services.postgrest = {
wantedBy = [ "multi-user.target" ];
after = [ "postgresql.service" ];
script =
let
configuration = pkgs.writeText "tutorial.conf" ''
db-uri = "postgres://${username}:${password}@localhost:${toString config.services.postgresql.port}/${database}"
db-schema = "${schema}"
db-anon-role = "${username}"
'';
in "${pkgs.haskellPackages.postgrest}/bin/postgrest ${configuration}";
serviceConfig.User = username;
};
};
client = {
imports = [ sharedModule ];
};
};
# Disable linting for simpler debugging of the testScript
skipLint = true;
testScript = ''
import json
import sys
start_all()
server.wait_for_open_port(${toString postgrestPort})
expected = [
{"id": 1, "done": False, "task": "finish tutorial 0", "due": None},
{"id": 2, "done": False, "task": "pat self on back", "due": None},
]
actual = json.loads(
client.succeed(
"${pkgs.curl}/bin/curl http://server:${toString postgrestPort}/${table}"
)
)
assert expected == actual, "table query returns expected content"
'';
})
A few notes:
- Between the machines defined inside the ``nodes`` attribute, hostnames
are resolved based on their attribute names. In this case we have ``client`` and ``server``.
- The testing framework exposes a wide set of operations used inside the ``testScript``.
A full set of testing operations is part of
`VM testing operations API Reference <https://nixos.org/manual/nixos/stable/index.html#sec-nixos-tests>`_.
Running tests
-------------
To set up all machines and execute the test script:
.. code:: shell-session
$ nix-build postgrest.nix
You'll notice an error message if something goes wrong.
In case the tests succeed, you should see at the end:
.. code:: shell-session
...
test script finished in 10.96s
cleaning up
killing client (pid 10)
killing server (pid 22)
(0.00 seconds)
/nix/store/bx7z3imvxxpwkkza10vb23czhw7873w2-vm-test-run-unnamed
Developing and debugging tests
------------------------------
When developing tests or when something breaks, it's useful to interactively fiddle
with the script or access a terminal for a machine.
To interactively start a Python session with a testing framework:
.. code:: shell-session
$ $(nix-build -A driver postgrest.nix)/bin/nixos-test-driver
...
starting VDE switch for network 1
>>>
You can run `any of the testing operations <https://nixos.org/manual/nixos/stable/index.html#sec-nixos-tests>`_.
The ``testScript`` attribute from our ``postgrest.nix`` definition can be executed with ``test_script()`` function.
To start all machines and enter a telnet terminal to a specific machine:
.. code:: shell-session
>>> start_all()
...
>>> server.shell_interact()
server: Terminal is ready (there is no prompt):
uname -a
Linux server 5.10.37 #1-NixOS SMP Fri May 14 07:50:46 UTC 2021 x86_64 GNU/Linux
Next steps
----------
- Running integration tests on CI requires hardware acceleration, which many CIs do not support.
To run integration tests on :ref:`GitHub Actions <github-actions>` see
`how to disable hardware acceleration <https://github.com/cachix/install-nix-action#how-can-i-run-nixos-tests>`_.
- NixOS comes with a large set of tests that serve also as educational examples. A good inspiration is `Matrix bridging with an IRC <https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/matrix-appservice-irc.nix>`_.

90
source/tutorials/towards-reproducibility-pinning-nixpkgs.md Normal file
View file

@ -0,0 +1,90 @@
(pinning-nixpkgs)=
# Towards reproducibility: Pinning nixpkgs
In various Nix examples, you'll often see references to [\<nixpkgs>](https://github.com/NixOS/nixpkgs), as follows.
```nix
{ pkgs ? import <nixpkgs> {}
}:
...
```
This is **convenient** to quickly demonstrate a Nix expression and get it working by importing Nix packages.
However, the resulting Nix expression **is not fully reproducible**. The `<nixpkgs>` reference
is set from the **local** `$NIX_PATH` environment variable. In most cases, this is set at the time Nix is installed
to the `nixpkgs-unstable` channel, and therefore it is likely to differ from machine to machine.
:::{note}
[Channels](https://nixos.wiki/wiki/Nix_channels) are a way of distributing Nix software, but they are being phased out.
Even though they are still used by default, it is recommended to avoid channels
and `<nixpkgs>` by always setting `NIX_PATH=` to be empty.
:::
## Pinning packages with URLs inside a Nix expression
To create **fully reproducible** Nix expressions, we can pin an exact versions of nixpkgs.
The simplest way to do this is to fetch the required nixpkgs version as a tarball specified via the relevant git commit hash:
```nix
{ pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/3590f02e7d5760e52072c1a729ee2250b5560746.tar.gz") {}
}:
...
```
Picking the commit can be done via [status.nixos.org](https://status.nixos.org/),
which lists all the releases and the latest commit that has passed all tests.
When choosing a commit, it is recommended to follow either
- the **latest stable NixOS** release by using a specific version, such as `nixos-21.05`, **or**
- the latest **unstable release** via `nixos-unstable`.
## Dependency management with niv
If you'd like a bit more automation around bumping dependencies, including nixpkgs,
[niv](https://github.com/nmattia/niv/) is made for exactly that. Niv itself is available
in `nixpkgs` so using it is simple:
```
$ nix-shell -p niv --run "niv init"
```
This command will generate `nix/sources.json` with information about how and where
dependencies are fetched. It will also create `nix/sources.nix` which glues the sources together in Nix.
By default `niv` will use the **latest stable** NixOS release. However, you should check to see which version is currently specified in [the niv repository](https://github.com/nmattia/niv) if you require a specific release, as it might lag behind.
You can see which version `niv` is tracking as follow:
> \$ niv show
And you can change the tracking branch to the one you want like this:
> \$ niv modify nixpkgs --branch nixos-21.05
You can use the generated `nix/sources.nix` with a top-level `default.nix`:
```nix
{ sources ? import ./nix/sources.nix
, pkgs ? import sources.nixpkgs {}
}:
...
```
And you can update all the dependencies by running:
```
$ nix-shell -p niv --run "niv update"
```
## Next steps
- For more examples and details of the different ways to pin `nixpkgs`, see {ref}`ref-pinning-nixpkgs`.
- To quickly setup a Nix project read through
[Getting started Nix template](https://github.com/nix-dot-dev/getting-started-nix-template).

93
source/tutorials/towards-reproducibility-pinning-nixpkgs.rst
View file

@ -1,93 +0,0 @@
.. _pinning-nixpkgs:
Towards reproducibility: Pinning nixpkgs
========================================
In various Nix examples, you'll often see references to `<nixpkgs> <https://github.com/NixOS/nixpkgs>`_, as follows.
.. code:: nix
{ pkgs ? import <nixpkgs> {}
}:
...
This is **convenient** to quickly demonstrate a Nix expression and get it working by importing Nix packages.
However, the resulting Nix expression **is not fully reproducible**. The ``<nixpkgs>`` reference
is set from the **local** ``$NIX_PATH`` environment variable. In most cases, this is set at the time Nix is installed
to the ``nixpkgs-unstable`` channel, and therefore it is likely to differ from machine to machine.
.. note::
`Channels <https://nixos.wiki/wiki/Nix_channels>`_ are a way of distributing Nix software, but they are being phased out.
Even though they are still used by default, it is recommended to avoid channels
and ``<nixpkgs>`` by always setting ``NIX_PATH=`` to be empty.
Pinning packages with URLs inside a Nix expression
--------------------------------------------------
To create **fully reproducible** Nix expressions, we can pin an exact versions of nixpkgs.
The simplest way to do this is to fetch the required nixpkgs version as a tarball specified via the relevant git commit hash:
.. code:: nix
{ pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/3590f02e7d5760e52072c1a729ee2250b5560746.tar.gz") {}
}:
...
Picking the commit can be done via `status.nixos.org <https://status.nixos.org/>`_,
which lists all the releases and the latest commit that has passed all tests.
When choosing a commit, it is recommended to follow either
* the **latest stable NixOS** release by using a specific version, such as ``nixos-21.05``, **or**
* the latest **unstable release** via ``nixos-unstable``.
Dependency management with niv
------------------------------
If you'd like a bit more automation around bumping dependencies, including nixpkgs,
`niv <https://github.com/nmattia/niv/>`_ is made for exactly that. Niv itself is available
in ``nixpkgs`` so using it is simple::
$ nix-shell -p niv --run "niv init"
This command will generate ``nix/sources.json`` with information about how and where
dependencies are fetched. It will also create ``nix/sources.nix`` which glues the sources together in Nix.
By default ``niv`` will use the **latest stable** NixOS release. However, you should check to see which version is currently specified in `the niv repository <https://github.com/nmattia/niv>`_ if you require a specific release, as it might lag behind.
You can see which version ``niv`` is tracking as follow:
$ niv show
And you can change the tracking branch to the one you want like this:
$ niv modify nixpkgs --branch nixos-21.05
You can use the generated ``nix/sources.nix`` with a top-level ``default.nix``:
.. code:: nix
{ sources ? import ./nix/sources.nix
, pkgs ? import sources.nixpkgs {}
}:
...
And you can update all the dependencies by running::
$ nix-shell -p niv --run "niv update"
Next steps
----------
- For more examples and details of the different ways to pin ``nixpkgs``, see :ref:`ref-pinning-nixpkgs`.
- To quickly setup a Nix project read through
`Getting started Nix template <https://github.com/nix-dot-dev/getting-started-nix-template>`_.