nix/mk/common-test.sh

# shellcheck shell=bash

# Remove overall test dir (at most one of the two should match) and
# remove file extension.

test_name=$(echo -n "${test?must be defined by caller (test runner)}" | sed \
    -e "s|^src/[^/]*-test/data/||" \
    -e "s|^tests/functional/||" \
    -e "s|\.sh$||" \
    )

# shellcheck disable=SC2016
TESTS_ENVIRONMENT=(
    "TEST_NAME=$test_name"
    'NIX_REMOTE='
    'PS4=+(${BASH_SOURCE[0]-$0}:$LINENO) '
)

read -r -a bash <<< "${BASH:-/usr/bin/env bash}"

run () {
   cd "$(dirname "$1")" && env "${TESTS_ENVIRONMENT[@]}" "${bash[@]}" -x -e -u -o pipefail "$(basename "$1")"
}
Start getting all shell scripts passing shellcheck Like with the formatter, we are blacklisting most files by default. Do a few files to get us started, and get a sense of what this looks like. 2024-05-27 09:40:49 -04:00			`# shellcheck shell=bash`

Refactor bash test build system a bit The basic idea here is to separate a few intertwined notions: 1. Not all "run bash tests" are "install tests" 2. Not all "run bash tests" use `tests/functional/init.sh`, or any pre-test initialization at all. This will used in the next commit when we have a test that check unit test golden master data. Also, move our custom `PS4` from the test to the test runner, as it is part of how we want to display the tests, not the test themselves. Co-authored-by: Robert Hensing <roberth@users.noreply.github.com> 2023-11-08 00:30:55 -05:00			`# Remove overall test dir (at most one of the two should match) and`
			`# remove file extension.`
Start getting all shell scripts passing shellcheck Like with the formatter, we are blacklisting most files by default. Do a few files to get us started, and get a sense of what this looks like. 2024-05-27 09:40:49 -04:00
Shellcheck some test scripts Progress on #10795 2024-05-28 10:10:02 -04:00			`test_name=$(echo -n "${test?must be defined by caller (test runner)}" \| sed \`
Unit tests and external libraries 2024-06-27 11:28:08 -04:00			`-e "s\|^src/[^/]*-test/data/\|\|" \`
Refactor bash test build system a bit The basic idea here is to separate a few intertwined notions: 1. Not all "run bash tests" are "install tests" 2. Not all "run bash tests" use `tests/functional/init.sh`, or any pre-test initialization at all. This will used in the next commit when we have a test that check unit test golden master data. Also, move our custom `PS4` from the test to the test runner, as it is part of how we want to display the tests, not the test themselves. Co-authored-by: Robert Hensing <roberth@users.noreply.github.com> 2023-11-08 00:30:55 -05:00			`-e "s\|^tests/functional/\|\|" \`
			`-e "s\|\.sh$\|\|" \`
			`)`
Put functional tests in `tests/functional` I think it is bad for these reasons when `tests/` contains a mix of functional and integration tests - Concepts is harder to understand, the documentation makes a good unit vs functional vs integration distinction, but when the integration tests are just two subdirs within `tests/` this is not clear. - Source filtering in the `flake.nix` is more complex. We need to filter out some of the dirs from `tests/`, rather than simply pick the dirs we want and take all of them. This is a good sign the structure of what we are trying to do is not matching the structure of the files. With this change we have a clean: ```shell-session $ git show 'HEAD:tests' tree HEAD:tests functional/ installer/ nixos/ ``` 2023-10-05 12:12:18 -04:00
Start getting all shell scripts passing shellcheck Like with the formatter, we are blacklisting most files by default. Do a few files to get us started, and get a sense of what this looks like. 2024-05-27 09:40:49 -04:00			`# shellcheck disable=SC2016`
Refactor bash test build system a bit The basic idea here is to separate a few intertwined notions: 1. Not all "run bash tests" are "install tests" 2. Not all "run bash tests" use `tests/functional/init.sh`, or any pre-test initialization at all. This will used in the next commit when we have a test that check unit test golden master data. Also, move our custom `PS4` from the test to the test runner, as it is part of how we want to display the tests, not the test themselves. Co-authored-by: Robert Hensing <roberth@users.noreply.github.com> 2023-11-08 00:30:55 -05:00			`TESTS_ENVIRONMENT=(`
			`"TEST_NAME=$test_name"`
			`'NIX_REMOTE='`
			`'PS4=+(${BASH_SOURCE[0]-$0}:$LINENO) '`
			`)`
Make `./mk/run-test.sh` work by itself; add `mk/debug-test.sh` First, logic is consolidated in the shell script instead of being spread between them and makefiles. That makes understanding what is going on a little easier. This would not be super interesting by itself, but it gives us a way to debug tests more easily. That in turn I hope is much more compelling. See the updated manual for details. Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io> Co-authored-by: Eelco Dolstra <edolstra@gmail.com> Co-authored-by: Robert Hensing <roberth@users.noreply.github.com> 2022-12-15 20:17:08 -05:00
Start getting all shell scripts passing shellcheck Like with the formatter, we are blacklisting most files by default. Do a few files to get us started, and get a sense of what this looks like. 2024-05-27 09:40:49 -04:00			`read -r -a bash <<< "${BASH:-/usr/bin/env bash}"`
Make `./mk/run-test.sh` work by itself; add `mk/debug-test.sh` First, logic is consolidated in the shell script instead of being spread between them and makefiles. That makes understanding what is going on a little easier. This would not be super interesting by itself, but it gives us a way to debug tests more easily. That in turn I hope is much more compelling. See the updated manual for details. Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io> Co-authored-by: Eelco Dolstra <edolstra@gmail.com> Co-authored-by: Robert Hensing <roberth@users.noreply.github.com> 2022-12-15 20:17:08 -05:00
Refactor bash test build system a bit The basic idea here is to separate a few intertwined notions: 1. Not all "run bash tests" are "install tests" 2. Not all "run bash tests" use `tests/functional/init.sh`, or any pre-test initialization at all. This will used in the next commit when we have a test that check unit test golden master data. Also, move our custom `PS4` from the test to the test runner, as it is part of how we want to display the tests, not the test themselves. Co-authored-by: Robert Hensing <roberth@users.noreply.github.com> 2023-11-08 00:30:55 -05:00			`run () {`
Start getting all shell scripts passing shellcheck Like with the formatter, we are blacklisting most files by default. Do a few files to get us started, and get a sense of what this looks like. 2024-05-27 09:40:49 -04:00			`cd "$(dirname "$1")" && env "${TESTS_ENVIRONMENT[@]}" "${bash[@]}" -x -e -u -o pipefail "$(basename "$1")"`
Refactor bash test build system a bit The basic idea here is to separate a few intertwined notions: 1. Not all "run bash tests" are "install tests" 2. Not all "run bash tests" use `tests/functional/init.sh`, or any pre-test initialization at all. This will used in the next commit when we have a test that check unit test golden master data. Also, move our custom `PS4` from the test to the test runner, as it is part of how we want to display the tests, not the test themselves. Co-authored-by: Robert Hensing <roberth@users.noreply.github.com> 2023-11-08 00:30:55 -05:00			`}`