1
0
Fork 0
mirror of https://github.com/NixOS/nix synced 2024-09-20 11:11:03 -04:00
Commit graph

2366 commits

Author SHA1 Message Date
pennae 2be6b14328 match line endings used by parser and error reports
the parser treats a plain \r as a newline, error reports do not. this
can lead to interesting divergences if anything makes use of this
feature, with error reports pointing to wrong locations in the input (or
even outside the input altogether).
2024-03-06 23:11:12 +01:00
pennae 1edd6fada5 report inherit attr errors at the duplicate name
previously we reported the error at the beginning of the binding
block (for plain inherits) or the beginning of the attr list (for
inherit-from), effectively hiding where exactly the error happened.

this also carries over to runtime positions of attributes in sets as
reported by unsafeGetAttrPos. we're not worried about this changing
observable eval behavior because it *is* marked unsafe, and the new
behavior is much more useful.
2024-03-06 23:11:12 +01:00
pennae 4147ecfb1c normalize formal order on ExprLambda::show
we already normalize attr order to lexicographic, doing the same for
formals makes sense. doubly so because the order of formals would
otherwise depend on the context of the expression, which is not quite as
useful as one might expect.
2024-03-06 23:11:12 +01:00
pennae d384ecd553 keep copies of parser inputs that are in-memory only
the parser modifies its inputs, which means that sharing them between
the error context reporting system and the parser itself can confuse the
reporting system. usually this led to early truncation of error context
reports which, while not dangerous, can be quite confusing.
2024-03-06 23:11:12 +01:00
Eelco Dolstra 6b976a1898
Merge pull request #10143 from fricklerhandwerk/test-help
add tests for showing help
2024-03-04 20:29:03 +01:00
Valentin Gagarin 8d23847571
fix indentation
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-04 19:25:44 +01:00
Valentin Gagarin 8a6ef3bae5
less scary jq
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-04 19:25:28 +01:00
Valentin Gagarin 4b15ca2ffb add tests for showing help 2024-03-04 16:14:00 +01:00
Eelco Dolstra 438855952b
Merge pull request #10122 from edolstra/arg-from-file
Add --arg-from-file and --arg-from-stdin
2024-03-04 13:29:58 +01:00
Théophane Hufschmitt 8a9eec375c
Merge pull request #10073 from olmokramer/nix-flake-update-multiple-inputs
Accept multiple inputs in `nix flake update`
2024-03-04 09:13:26 +01:00
Olmo Kramer b1ad729add
Add test for nix flake update with multiple inputs 2024-03-03 13:59:39 +01:00
Jade Lovelace 2625e9fb0a Ban building Nix with NDEBUG
When reviewing old PRs, I found that #9997 adds some code to ensure one
particular assert is always present. But, removing asserts isn't
something we do in our own release builds either in the flake here or in
nixpkgs, and is plainly a bad idea that increases support burden,
especially if other distros make bad choices of build flags in their Nix
packaging.

For context, the assert macro in the C standard is defined to do nothing
if NDEBUG is set.

There is no way in our build system to set -DNDEBUG without manually
adding it to CFLAGS, so this is simply a configuration we do not use.
Let's ban it at compile time.

I put this preprocessor directive in src/libutil.cc because it is not
obvious where else to put it, and it seems like the most logical file
since you are not getting a usable nix without it.
2024-03-02 11:22:46 -08:00
Théophane Hufschmitt 11a1dcc43b Properly fail on flakerefs that don't point to a directory
Directly fail if a flakeref points to something that isn't a directory
instead of falling back to the logic of trying to look up the hierarchy
to find a valid flake root.

Fix https://github.com/NixOS/nix/issues/9868
2024-03-02 10:24:20 +01:00
Eelco Dolstra ba9b6b29b7
Merge pull request #10111 from obsidiansystems/git-objects
Support symlinks properly with `git-hashing` experimental feature
2024-03-01 15:00:43 +01:00
Eelco Dolstra 8ce1f6800b Add --arg-from-stdin to read an argument from stdin 2024-03-01 14:39:42 +01:00
Eelco Dolstra 291b10c607 Add --arg-from-file for reading a string from a file 2024-03-01 14:35:27 +01:00
Théophane Hufschmitt 65b79c52c6 Fix a typo in a test comment
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-03-01 09:31:28 +01:00
Théophane Hufschmitt a55c6a0f47 Add a NixOS test for the sandbox escape
Test that we can't leverage abstract unix domain sockets to leak file
descriptors out of the sandbox and modify the path after it has been
registered.
2024-03-01 09:31:28 +01:00
John Ericson 5a2985431c Revert "Revert "Merge pull request #9546 from NixOS/nixos-23.11""
This reverts commit d6d7d2cb46.
2024-02-29 14:52:31 -05:00
John Ericson d6d7d2cb46 Revert "Merge pull request #9546 from NixOS/nixos-23.11"
This reverts commit 587c7dcb2b, reversing
changes made to 864fc85fc8.
2024-02-29 14:39:29 -05:00
John Ericson 9b506ff0c1 Activate hermetic.nix variation only for new layered store tests 2024-02-29 10:06:53 -05:00
John Ericson cb4f85f11c Merge branch 'master' into overlayfs-store 2024-02-29 09:54:21 -05:00
Eelco Dolstra b1586a6799
Merge pull request #10090 from bobvanderlinden/profile-error-install-twice
profile install: warn on installing package twice
2024-02-29 10:04:08 +01:00
Bob van der Linden 14adff1711
profile install: skip and warn on installing package twice 2024-02-29 08:29:38 +01:00
John Ericson bcb5f235f9 Support symlinks properly with git-hashing experimental feature
Before, they would not be written to a file `FileSystemObjectSink`
correctly.
2024-02-28 12:24:45 -05:00
DavHau 358c26fd13 fetchTree: shallow git fetching by default
Motivation:
make git fetching more efficient for most repos by default
2024-02-28 13:27:22 +07:00
Théophane Hufschmitt 6147d27afb Bump the required daemon version for the git hashing tests
The required version check was a bit too lenient, and
`nixpkgs#nixUnstable` was considered valid while it didn't have the fix.
2024-02-28 07:11:22 +01:00
Théophane Hufschmitt bbef03872b Bump the required daemon version for the impure-env test
The required version check was a bit too lenient, and
`nixpkgs#nixUnstable` was considered valid while it didn't have the fix.
2024-02-28 07:08:21 +01:00
John Ericson 201551c937 Add Git object hashing to the store layer
Part of RFC 133

Extracted from our old IPFS branches.

Co-Authored-By: Matthew Bauer <mjbauer95@gmail.com>
Co-Authored-By: Carlo Nucera <carlo.nucera@protonmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Florian Klink <flokli@flokli.de>
2024-02-27 11:27:34 -05:00
Eelco Dolstra c3e9e3d0c3
Merge pull request #9767 from obsidiansystems/canon-path-split
Support Windows paths in `canonPath` and `absPath`
2024-02-27 17:12:29 +01:00
John Ericson 354ba27296
Merge pull request #10093 from NixOS/revert-10084-remove-dead-git-code
Revert "Remove dead Git code"
2024-02-27 09:54:25 -05:00
Eelco Dolstra 5b0d78ec73
Merge pull request #10091 from bobvanderlinden/default-package-name
Fix extraction of name for defaultPackage URLs
2024-02-27 15:14:20 +01:00
Bob van der Linden e5d9130a5b
Fix extraction of name for defaultPackage URLs 2024-02-27 07:53:05 +01:00
Théophane Hufschmitt be0052b45f
Revert "Remove dead Git code" 2024-02-27 06:39:30 +01:00
Robert Hensing 4c7f0ef6ca
Merge pull request #9847 from pennae/inherit-from-dedup
deduplicate inherit-from source expr work
2024-02-26 20:25:58 +01:00
pennae cefd0302b5 evaluate inherit (from) exprs only once per directive
desugaring inherit-from to syntactic duplication of the source expr also
duplicates side effects of the source expr (such as trace calls) and
expensive computations (such as derivationStrict).
2024-02-26 19:07:08 +01:00
Théophane Hufschmitt 219705ff64 Remove dead code
Most of the code in `git.{cc,hh}` is dead, so get rid of it.
2024-02-26 11:07:47 +01:00
Rebecca Turner fe6408b5df
Update snapshots 2024-02-22 17:58:55 -08:00
Théophane Hufschmitt 6a5210f48e
Merge pull request #9815 from NixOS/nix-hash-path
`nix hash path`, text hashing for `nix store add`, and preparatory refactors
2024-02-22 17:15:34 +01:00
Eelco Dolstra f183eef4a0
Merge pull request #10060 from bobvanderlinden/pr-simple-profile-remove-suggestion
profile: suggest removal using profile entry name
2024-02-22 11:02:12 +01:00
Bob van der Linden 4ae5091716
nix profile: suggest removal using entry name
When a file conflict arises during a package install a suggestion is
made to remove the old entry. This was previously done using the
installable URLs of the old entry. These URLs are quite verbose and
often do not equal the URL of the existing entry.

This change uses the recently introduced profile entry name for the
suggestion, resulting in a simpler output.

The improvement is easily seen in the change to the functional test.
2024-02-21 21:58:36 +01:00
John Ericson 2080d89b87
Merge pull request #10038 from edolstra/tarball-git-cache
Use the Git cache for tarball flakes
2024-02-21 15:47:02 -05:00
John Ericson efd36b49e8 nix hash path, and preperatory refactors
- `nix store add` supports text hashing

  With functional test ensuring it matches `builtins.toFile`.

- Factored-out flags for both commands

- Move all common reusable flags to `libcmd`

  - They are not part of the *definition* of the CLI infra, just a usag
    of it.

  - The `libstore` flag couldn't go in `args.hh` in libutil anyways,
    would be awkward for it to live alone

- Shuffle around `Cmd*` hierarchy so flags for deprecated commands don't
  end up on the new ones
2024-02-21 12:11:25 -05:00
Eelco Dolstra 071dd2b3a4 Input: Replace 'locked' bool by isLocked() method
It's better to just check whether the input has all the attributes
needed to consider itself locked (e.g. whether a Git input has an
'rev' attribute).

Also, the 'locked' field was actually incorrect for Git inputs: it
would be set to true even for dirty worktrees. As a result, we got
away with using fetchTree() internally even though fetchTree()
requires a locked input in pure mode. In particular, this allowed
'--override-input' to work by accident.

The fix is to pass a set of "overrides" to call-flake.nix for all the
unlocked inputs (i.e. the top-level flake and any --override-inputs).
2024-02-20 16:59:15 +01:00
Eelco Dolstra 0acd783190 Don't send settings that depend on disabled experimental features to the daemon
This fixes warnings like

   warning: Ignoring setting 'auto-allocate-uids' because experimental feature 'auto-allocate-uids' is not enabled
   warning: Ignoring setting 'impure-env' because experimental feature 'configurable-impure-env' is not enabled

when using the daemon and the user didn't actually set those settings.

Note: this also hides those settings from `nix config show`, but that
seems a good thing.
2024-02-20 15:28:42 +01:00
Théophane Hufschmitt d2c6a93bd5
Merge pull request #10044 from edolstra/empty-git-repos
Handle empty Git repositories / workdirs
2024-02-20 14:01:23 +01:00
Eelco Dolstra 930b9c8269 PosixSourceAccessor: Support roots that are not directories
We have to support this for `fetchTree { type = "file" }` (and
probably other types of trees that can have a non-directory at the
root, like NARs).
2024-02-20 12:51:15 +01:00
Eelco Dolstra b00f412f81 Remove bad.tar.xz check, since libarchive doesn't care 2024-02-20 12:51:15 +01:00
Eelco Dolstra db012d1e63 tests/functional/tarball.sh: Fix invalid file:// URLs 2024-02-20 12:51:15 +01:00
Théophane Hufschmitt 6f4bb1b584
Merge pull request #10043 from edolstra/fix-readonly-fetchToStore
fetchToStore(): Don't always respect settings.readOnlyMode
2024-02-20 12:07:48 +01:00
Eelco Dolstra 7cb4d0c5b7 fetchToStore(): Don't always respect settings.readOnlyMode
It's now up to the caller whether readOnlyMode should be applied. In
some contexts (like InputScheme::fetch()), we always need to fetch.
2024-02-20 11:46:49 +01:00
Eelco Dolstra 9e762454cf Support empty Git repositories / workdirs
Fixes #10039.
2024-02-20 11:40:02 +01:00
John Ericson 319ec6f84a Support Windows paths in canonPath and absPath
`canonPath` and `absPath` work on native paths, and so should switch
between supporting Unix paths and Windows paths accordingly.

The templating is because `CanonPath`, which shares the implementation,
should always be Unix style. It is the pure "nix-native" path type for
virtual file operations --- it is part of Nix's "business logic", and
should not vary with the host OS accordingly.
2024-02-16 10:31:36 -05:00
John Ericson d17e1d9737 Purify CanonPath
The core `CanonPath` constructors were using `absPath`, but `absPath` in
some situations does IO which is not appropriate. It turns out that
these constructors avoided those situations, and thus were pure, but it
was far from obvious this was the case.

To remedy the situation, abstract the core algorithm from `canonPath` to
use separately in `CanonPath` without any IO. No we know by-construction
that those constructors are pure.

That leaves `CanonPath::fromCWD` as the only operation which uses IO /
is impure. Add docs on it, and `CanonPath` as a whole, explaining the
situation.

This is also necessary to support Windows paths on windows without
messing up `CanonPath`. But, I think it is good even without that.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-02-16 09:47:25 -05:00
John Ericson d53c8901ef
Merge pull request #9985 from alois31/symlink-resolution
Restore `builtins.pathExists` behavior on broken symlinks
2024-02-16 09:24:03 -05:00
Théophane Hufschmitt d857914e1a
Merge pull request #9931 from 9999years/pretty-printer
Pretty-print values in the REPL
2024-02-14 13:32:58 +01:00
Eelco Dolstra 2b946aad8e
Merge pull request #10006 from edolstra/fix-impure-fetchurl
<nix/fetchurl.nix>: Restore support for "impure = true"
2024-02-13 22:50:55 +01:00
Alois Wohlschlager 89e21ab4bd
Restore builtins.pathExists behavior on broken symlinks
Commit 83c067c0fa changed `builtins.pathExists`
to resolve symlinks before checking for existence. Consequently, if the path
refers to a symlink itself, existence of the target of the symlink (instead of
the symlink itself) was checked. Restore the previous behavior by skipping
symlink resolution in the last component.
2024-02-13 18:09:55 +01:00
John Ericson 41dd9857c7 Proper parse and render functions for FileIngestionMethod and ContentAddressMethod
No outward facing behavior is changed.

Older methods with same names that operate on on method + algo pair (for
old-style `<method>:algo`) are renamed to `*WithAlgo`.)

The functions are unit-tested in the same way the names for the hash
algorithms are tested.
2024-02-13 10:30:16 -05:00
John Ericson 8bebf9607c Split hash.sh test in two
Converting hashes and hashing files are pretty separate tasks, and more
test parallelism is better.
2024-02-13 09:44:14 -05:00
Eelco Dolstra bb63bd50e6 <nix/fetchurl.nix>: Restore support for "impure = true" 2024-02-13 14:14:20 +01:00
pennae ecf8b12d60 group inherit by source during Expr::show
for plain inherits this is really just a stylistic choice, but for
inherit-from it actually fixes an exponential size increase problem
during expr printing (as may happen during assertion failure reporting,
on during duplicate attr detection in the parser)
2024-02-12 13:58:29 +01:00
pennae 6c08fba533 use the same bindings print for ExprAttrs and ExprLet
this also has the effect of sorting let bindings lexicographically
rather than by symbol creation order as was previously done, giving a
better canonicalization in the process.
2024-02-12 13:35:00 +01:00
pennae 73065a400d add test for inherit expr printing 2024-02-12 13:32:33 +01:00
pennae 8669c02468 add test for inherit-from semantics 2024-02-12 13:32:33 +01:00
Rebecca Turner 953eb0cba2
Fix tests 2024-02-08 15:55:20 -08:00
Rebecca Turner c0e7f50c1a
Rename hintfmt to HintFmt 2024-02-08 11:58:25 -08:00
Rebecca Turner 149bd63afb
Cleanup fmt.hh
When I started contributing to Nix, I found the mix of definitions and
names in `fmt.hh` to be rather confusing, especially the small
difference between `hintfmt` and `hintformat`. I've renamed many classes
and added documentation to most definitions.

- `formatHelper` is no longer exported.
- `fmt`'s documentation is now with `fmt` rather than (misleadingly)
  above `formatHelper`.
- `yellowtxt` is renamed to `Magenta`.

  `yellowtxt` wraps its value with `ANSI_WARNING`, but `ANSI_WARNING`
  has been equal to `ANSI_MAGENTA` for a long time. Now the name is
  updated.
- `normaltxt` is renamed to `Uncolored`.
- `hintfmt` has been merged into `hintformat` as extra constructor
  functions.
- `hintformat` has been renamed to `hintfmt`.
- The single-argument `hintformat(std::string)` constructor has been
  renamed to a static member `hintformat::interpolate` to avoid pitfalls
  with using user-generated strings as format strings.
2024-02-08 11:51:03 -08:00
Théophane Hufschmitt 1ba9780cf5
Merge pull request #9834 from 9999years/structured-errors
Towards structured error classes
2024-02-08 20:00:25 +01:00
Théophane Hufschmitt a8050d9b83
Merge pull request #9928 from 9999years/error-messages-in-nix-repl
Improve error printing in `nix repl`
2024-02-08 16:21:13 +01:00
Théophane Hufschmitt 46a0625a40
Merge pull request #9929 from 9999years/dont-print-values-in-magenta
Don't print the first bracket in values in magenta in error messages
2024-02-08 10:56:18 +01:00
Théophane Hufschmitt 9b8b486091
Merge pull request #9933 from pennae/debugger-fix
fix debugger crashing while printing envs
2024-02-08 10:48:02 +01:00
Rebecca Turner c0a15fb7d0
Pretty-print values in the REPL
Pretty-print values in the REPL by printing each item in a list or
attrset on a separate line. When possible, single-item lists and
attrsets are printed on one line, as long as they don't contain a nested
list, attrset, or thunk.

Before:
```
{ attrs = { a = { b = { c = { }; }; }; }; list = [ 1 ]; list' = [ 1 2 3 ]; }
```

After:
```
{
  attrs = {
    a = {
      b = {
        c = { };
      };
    };
  };
  list = [ 1 ];
  list' = [
    1
    2
    3
  ];
}
```
2024-02-05 13:23:38 -08:00
Eelco Dolstra a6737b7e17 CanonPath, SourcePath: Change operator + to /
This is less confusing and makes it more similar to std::filesystem::path.
2024-02-05 15:17:39 +01:00
Eelco Dolstra e190c20c33
Merge pull request #9926 from 9999years/fix-cycle-detection-in-nix-repl
Fix cycle detection in `nix repl`
2024-02-05 14:24:28 +01:00
pennae 5ccb06ee1b fix debugger crashing while printing envs
fixes #9932
2024-02-04 17:12:04 +01:00
Rodney Lorrimar e1131b5927
print-dev-env: Avoid using unbound shellHook variable
Some tools which consume the "nix print-dev-env" rc script (such as
"nix-direnv") are sensitive to the use of unbound variables. They use
"set -u".

The "nix print-dev-env" rc script initially unsets "shellHook", then
loads variables from the derivation, and then evaluates "shellHook".
However, most derivations don't have a "shellHook" attribute.

So users get the error "shellHook: unbound variable". This can be
demonstrated with the command:

    nix print-dev-env nixpkgs#hello | bash -u

This commit changes the rc script to provide an empty fallback value
for the "shellHook" variable.

Closes: #7951 #8253
2024-02-04 13:57:13 +08:00
Rebecca Turner 770d2bc779
Key repeated values on attribute binding pointers, not value pointers
Closes #8672
2024-02-03 21:23:06 -08:00
Rebecca Turner 9646d62b0c
Don't print values in magenta
This fixes the opening bracket of lists/attrsets being printed in
magenta, unlike the closing bracket.

https://github.com/NixOS/nix/pull/9753#issuecomment-1904616088
2024-02-03 21:17:22 -08:00
Rebecca Turner c5d525cd84
Print error messages but not traces
This makes output of values that include errors much cleaner.

Before:
```
nix-repl> { err = builtins.throw "uh oh!"; }
{ err = «error:
       … while calling the 'throw' builtin
         at «string»:1:9:
            1| { err = builtins.throw "uh oh!"; }
             |         ^

       error: uh oh!»; }
```

After:
```
nix-repl> { err = builtins.throw "uh oh!"; }
{ err = «error: uh oh!»; }
```

But if the whole expression throws an error, source locations and (if
applicable) a stack trace are printed, like you'd expect:

```
nix-repl> builtins.throw "uh oh!"
error:
       … while calling the 'throw' builtin
         at «string»:1:1:
            1| builtins.throw "uh oh!"
             | ^

       error: uh oh!
```
2024-02-03 20:50:16 -08:00
Eelco Dolstra e67458e5b8 Better test fix 2024-02-02 13:34:56 +01:00
Eelco Dolstra 05535be03a Fix test 2024-02-02 13:07:08 +01:00
Rebecca Turner c6a89c1a16
libexpr: Support structured error classes
While preparing PRs like #9753, I've had to change error messages in
dozens of code paths. It would be nice if instead of

    EvalError("expected 'boolean' but found '%1%'", showType(v))

we could write

    TypeError(v, "boolean")

or similar. Then, changing the error message could be a mechanical
refactor with the compiler pointing out places the constructor needs to
be changed, rather than the error-prone process of grepping through the
codebase. Structured errors would also help prevent the "same" error
from having multiple slightly different messages, and could be a first
step towards error codes / an error index.

This PR reworks the exception infrastructure in `libexpr` to
support exception types with different constructor signatures than
`BaseError`. Actually refactoring the exceptions to use structured data
will come in a future PR (this one is big enough already, as it has to
touch every exception in `libexpr`).

The core design is in `eval-error.hh`. Generally, errors like this:

    state.error("'%s' is not a string", getAttrPathStr())
      .debugThrow<TypeError>()

are transformed like this:

    state.error<TypeError>("'%s' is not a string", getAttrPathStr())
      .debugThrow()

The type annotation has moved from `ErrorBuilder::debugThrow` to
`EvalState::error`.
2024-02-01 16:39:38 -08:00
Eelco Dolstra 1ee42c5b88 builtin:fetchurl: Ensure a fixed-output derivation
Previously we didn't check that the derivation was fixed-output, so
you could use builtin:fetchurl to impurely fetch a file.
2024-02-01 21:46:01 +01:00
John Ericson 31881d651a Merge remote-tracking branch 'upstream/master' into overlayfs-store 2024-02-01 11:07:47 -05:00
Robert Hensing 4072a8fea0
Merge pull request #9867 from hercules-ci/issue-912
#912 allow leading period
2024-01-31 19:10:59 +01:00
Robert Hensing f1b4663805 Disallow store path names that are . or .. (plus opt. -)
As discussed in the maintainer meeting on 2024-01-29.

Mainly this is to avoid a situation where the name is parsed and
treated as a file name, mostly to protect users.
.-* and ..-* are also considered invalid because they might strip
on that separator to remove versions. Doesn't really work, but that's
what we decided, and I won't argue with it, because .-* probably
doesn't seem to have a real world application anyway.
We do still permit a 1-character name that's just "-", which still
poses a similar risk in such a situation. We can't start disallowing
trailing -, because a non-zero number of users will need it and we've
seen how annoying and painful such a change is.

What matters most is preventing a situation where . or .. can be
injected, and to just get this done.
2024-01-31 18:35:19 +01:00
Robert Hensing 8406da2877 test: Generate distinct hashes
Gen::just is the constant generator. Don't just return that!
2024-01-31 18:35:19 +01:00
Robert Hensing 69bbd5852a test: Generate distinct path names
Gen::just is the constant generator. Don't just return that!
2024-01-31 18:35:19 +01:00
Robert Hensing b13e6a76b4 parseStorePath: Support leading period 2024-01-31 18:35:19 +01:00
Eelco Dolstra b36ff47e7c Resolve symlinks in a few more places
Fixes #9882.
2024-01-30 15:35:31 +01:00
Eelco Dolstra f6719032cf Shut up a gcc warning 2024-01-29 15:22:44 +01:00
Robert Hensing 9ddd0f2af8 Revert "StorePath: reject names starting with '.'"
This reverts commit 24bda0c7b3.
2024-01-27 11:18:03 +01:00
Rebecca Turner 772897a1cd
Color diff output in tests/functional/lang tests
Use `diff --color=always` to print colored output for language test
failures. I've also flipped the arguments so that expected lines missing
from the actual output will be marked with a red `-` and additional
lines found in the actual output will be marked with a green `+`.
Previously it was the other way around, which was very confusing.
2024-01-26 10:08:56 -08:00
Robert Hensing 212ba69e6f
Merge pull request #9843 from hercules-ci/test-pr-9626
tests/functional/fetchGit.sh: Test fetchGit/fetchTree error message
2024-01-24 13:55:41 +01:00
Robert Hensing 6532dd50fc tests/functional/fetchGit.sh: Test fetchGit/fetchTree error message
Follow-up for https://github.com/NixOS/nix/pull/9626
176dcd5c61
2024-01-24 13:19:02 +01:00
Robert Hensing 5b7bfd2d6b
Merge pull request #9754 from 9999years/print-value-when-coercion-fails
Print the value in `error: cannot coerce` messages
2024-01-24 12:48:39 +01:00
Théophane Hufschmitt 69d0ae27e3
Merge pull request #9841 from obsidiansystems/float-speed-factor
Convert `Machine::speedFactor` from a non-neg int to a non-neg float
2024-01-24 11:28:54 +01:00
John Ericson 1e24db6f9a Convert Machine::speedFactor from a non-neg int to a non-neg float
The short motivation is to match Hydra, so we can de-dup.

The long version is layed out in
https://github.com/NixOS/nix/issues/9840.
2024-01-24 01:03:07 -05:00
tomberek 1c260fa6d1
Merge pull request #9481 from iFreilicht/disallow-nix-search-without-search-terms
nix search: Disallow empty regex
2024-01-23 20:59:52 -05:00
Rebecca Turner 83bb494a30
Print the value in error: cannot coerce messages
This extends the `error: cannot coerce a TYPE to a string` message
to print the value that could not be coerced. This helps with debugging
by making it easier to track down where the value is being produced
from, especially in errors with deep or unhelpful stack traces.
2024-01-23 15:15:41 -08:00
Robert Hensing 08bf2846df
Merge pull request #9653 from obsidiansystems/improve-parse-sink
Improve the `ParseSink` interface
2024-01-23 01:04:57 +01:00
John Ericson 6365bbfa81 Improve the FileSystemObjectSink interface
More invariants are enforced in the type, and less state needs to be
stored in the main sink itself. The method here is roughly that known as
"session types".

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-01-22 18:01:21 -05:00
John Ericson 966d6fcd01 ParseSink -> FileSystemObjectSink
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-01-22 18:01:18 -05:00
Maximilian Bosch 81499a0b93
libexpr: print value of what is attempted to be called as function
Low-hanging fruit in the spirit of #9753 and #9754 (means 9999years did
all the hard work already).

This basically prints out what was attempted to be called as function,
i.e.

  map (import <nixpkgs> {}) [ 1 2 3 ]

now gives the following error message:

    error:
           … while calling the 'map' builtin
             at «string»:1:1:
                1| map (import <nixpkgs> {}) [ 1 2 3 ]
                 | ^

           … while evaluating the first argument passed to builtins.map

           error: expected a function but found a set: { _type = "pkgs"; AAAAAASomeThingsFailToEvaluate = «thunk»; AMB-plugins = «thunk»; ArchiSteamFarm = «thunk»; BeatSaberModManager = «thunk»; CHOWTapeModel = «thunk»; ChowCentaur = «thunk»; ChowKick = «thunk»; ChowPhaser = «thunk»; CoinMP = «thunk»;  «18783 attributes elided»}
2024-01-22 22:41:42 +01:00
Robert Hensing 5f72a97092
Merge pull request #9753 from 9999years/print-value-on-type-error
Print the value in `value is X while a Y is expected` error
2024-01-22 22:18:16 +01:00
Robert Hensing 74534829f2
Merge pull request #9830 from hercules-ci/test-cross-version-remote-build
tests/nixos: Test remote build against older versions
2024-01-22 19:42:57 +01:00
Robert Hensing c4d7c4a848 nixos/tests/remote-builds*: Format
nixpkgs-fmt
2024-01-22 18:47:59 +01:00
Robert Hensing 5167351efb tests/nixos/remote-builds*: Inline module + format 2024-01-22 18:44:16 +01:00
John Ericson 1fb2582969 Create unit tests for the serve proto handshake
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-01-22 12:43:11 -05:00
Robert Hensing e502d1cf94 tests/nixos: Test remote build against older versions 2024-01-22 18:39:15 +01:00
Rebecca Turner cb7fbd4d83
Print value on type error
Adds the failing value to `value is <TYPE> while a <TYPE> is expected`
error messages.
2024-01-22 08:56:02 -08:00
John Ericson 202c5e2afc Start standardizing hash algo flags
Do this if we want to do `--hash-algo` everywhere, and not `--algo` for
hash commands.

The new `nix hash convert` is updated. Deprecated new CLI commands are
left as-is (`nix hash path` needs to be redone and is also left as-is).
2024-01-20 17:29:35 -05:00
John Ericson 356352c370 Add missing --hash-algo flag to nix store add 2024-01-19 23:11:18 -05:00
Eelco Dolstra e6e160a075
Merge pull request #9632 from cole-h/nix-daemon-testing
Use `nix daemon` in the test suite
2024-01-19 16:21:51 +01:00
DavHau bc00fa4647 fetchTree/fetchGit: re-enable shallow fetching
Add several tests for git fetching:
- shallow-cache-separation: can fetch the same repo shallowly and non-shallowly
- shallow-ignore-ref: ensure that ref gets ignored when shallow=true is set
- ssh-shallow: can fetch a git repo via ssh using shallow=1
2024-01-19 20:30:47 +07:00
Robert Hensing 346d513d86 tests/nixos/fetch-git: Add http-auth test 2024-01-18 22:34:38 +01:00
Robert Hensing ed975e953c tests/nixos/fetch-git: Testsupport for private repos 2024-01-18 22:29:26 +01:00
Robert Hensing 0bd9e10aea
Merge pull request #9676 from DavHau/git-testsuite
initialize test suite for git fetchers
2024-01-18 21:51:48 +01:00
Robert Hensing 1254170405 tests/nixos/fetch-git: Make the store paths unique 2024-01-18 21:00:40 +01:00
Robert Hensing 94eba0ebbb tests/nixos/fetch-git: Memoize -> save
Memoization is for thunk-like behavior whereas this is executed eagerly.
2024-01-18 21:00:40 +01:00
Robert Hensing fd41979d78 tests/nixos/fetch-git: Factor out gitea repo module 2024-01-18 21:00:40 +01:00
Robert Hensing 78074bdea4 tests/nixos/fetch-git: Apply suggestions 2024-01-18 12:53:53 +01:00
Robert Hensing 2a3c5e6b8b
Merge pull request #9480 from NixOS/libfetchers-git-exportIgnore
libfetchers/git: Support export-ignore
2024-01-16 23:03:46 +01:00
Eelco Dolstra 302625e83b Merge remote-tracking branch 'origin/master' into handle-missing-gc-socket 2024-01-16 13:18:58 +01:00
Robert Hensing cbc319e9be tests/functional/lang: Test substring with negative length 2024-01-16 12:19:31 +01:00
Robert Hensing b2deff1947
Merge pull request #9747 from awakesecurity/mz/fix-quadratic-splitString
Fix performance of builtins.substring for empty substrings
2024-01-16 12:18:59 +01:00
John Ericson 03a6ca9b25 tests/functional/nix-profile.sh: Add missing --no-link
Otherwise we get a stray `tests/functional/result`, which can cause
spurious failures later.

(I got a failure because the test temp dir effecting the store dir
changed. This caused a test later because Nix didn't want to remove the
old `result` because it wasn't pointing inside the new Nix store.)
2024-01-14 15:27:44 -05:00
Cole Helbling bbcd9fcfc1 Arbitrarily bring back some nix-daemon calls
This means that both `nix daemon` and `nix-daemon` will be (somewhat)
tested.
2024-01-13 11:27:04 -08:00
John Ericson c58da62a06
Merge pull request #9737 from obsidiansystems/sys-error-split
Separate `SystemError` from `SysError`
2024-01-12 12:41:36 -05:00
Mel Zuser 1996105e91 added test for empty substring special case 2024-01-12 09:40:21 -08:00
John Ericson 6208ca7209 Separate SystemError from SysError
Most of this is a `catch SysError` -> `catch SystemError` sed. This
is a rather pure-churn change I would like to get out of the way. **The
intersting part is `src/libutil/error.hh`.**

On Unix, we will only throw the `SysError` concrete class, which has
the same constructors that `SystemError` used to have.

On Windows, we will throw `WinError` *and* `SysError`. `WinError`
(which will be created in a later PR), will use a `DWORD` instead of
`int` error value, and `GetLastError()`, which is the Windows equivalent
of the `errno` machinery. Windows will *also* use `SysError` because
Window's "libc" (MSVCRT) implements the POSIX interface, and we use it
too.

As the docs describe, while we *throw* one of the 3 choices above (2
concrete classes or the alias), we should always *catch* `SystemError`.
This ensures no matter how the implementation changes for Windows (e.g.
between `SysError` and `WinError`) the catching logic stays the same
and stays correct.

Co-Authored-By volth <volth@volth.com>
Co-Authored-By Eugene Butler <eugene@eugene4.com>
2024-01-12 12:00:33 -05:00
Eelco Dolstra 52f949bbf5
Merge pull request #9656 from edolstra/nix-profile-stable-names
Make profile element names stable
2024-01-12 16:59:36 +01:00
Eelco Dolstra 72560f7bbe Add profile migration test 2024-01-12 16:33:15 +01:00
Robert Hensing 692e9197bc fetchTree: Disallow combination of submodules and exportIgnore for now 2024-01-12 15:31:53 +01:00
Robert Hensing 99bd12f0b1 fetchGit/fetchTree: Improve exportIgnore, submodule interaction
Also fingerprint and some preparatory improvements.

Testing is still not up to scratch because lots of logic is duplicated
between the workdir and commit cases.
2024-01-12 15:31:53 +01:00
Robert Hensing ce6d58a97c git fetcher: Add exportIgnore parameter
Enabled for fetchGit, which historically had this behavior,
among other behaviors we do not want in fetchGit.

fetchTree disables this parameter by default. It can choose the
simpler behavior, as it is still experimental.

I am not confident that the filtering implementation is future
proof. It should reuse a source filtering wrapper, which I believe
Eelco has already written, but not merged yet.
2024-01-12 15:31:52 +01:00
DavHau 4d0ecda33e fetchTree/fetchGit: add test for .gitattributes
...with the intention to prevent future regressions in fetchGit
2024-01-12 15:31:52 +01:00
Eelco Dolstra 0d55d660d5
Merge pull request #8043 from bobvanderlinden/pr-shell-env
nix: develop: use SHELL from rc script
2024-01-12 13:42:42 +01:00
Eelco Dolstra e21b3cf9db Merge remote-tracking branch 'origin/master' into nix-profile-stable-names 2024-01-12 13:36:27 +01:00
Eelco Dolstra 66bd1b0298 Merge remote-tracking branch 'origin/master' into pr-shell-env 2024-01-12 12:56:26 +01:00
Eelco Dolstra 3e23759834 gc-non-blocking.sh: Add explanation
Also name the _NIX_TEST_GC_SYNC environment variables logically.
2024-01-12 12:38:55 +01:00
Eelco Dolstra 5703c31325 Merge remote-tracking branch 'origin/master' into handle-missing-gc-socket 2024-01-12 12:26:25 +01:00
Rebecca Turner df84dd4d8d
Restore ambiguous value printer for nix-instantiate
The Nix team has requested that this output format remain unchanged.
I've added a warning to the man page explaining that `nix-instantiate
--eval` output will not parse correctly in many situations.
2024-01-11 16:34:36 -08:00
Rebecca Turner 0fa08b4516
Unify and refactor value printing
Previously, there were two mostly-identical value printers -- one in
`libexpr/eval.cc` (which didn't force values) and one in
`libcmd/repl.cc` (which did force values and also printed ANSI color
codes).

This PR unifies both of these printers into `print.cc` and provides a
`PrintOptions` struct for controlling the output, which allows for
toggling whether values are forced, whether repeated values are tracked,
and whether ANSI color codes are displayed.

Additionally, `PrintOptions` allows tuning the maximum number of
attributes, list items, and bytes in a string that will be displayed;
this makes it ideal for contexts where printing too much output (e.g.
all of Nixpkgs) is distracting. (As requested by @roberth in
https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735)

Please read the tests for example output.

Future work:
- It would be nice to provide this function as a builtin, perhaps
  `builtins.toStringDebug` -- a printing function that never fails would
  be useful when debugging Nix code.
- It would be nice to support customizing `PrintOptions` members on the
  command line, e.g. `--option to-string-max-attrs 1000`.
2024-01-11 16:34:36 -08:00
DavHau 0f95330fde fetchGit: add simple test for ssh fetching
Also move tests to separate files which are auto-imported. This should allow people adding tests concurrently without introducing merge conflicts
2024-01-11 15:46:26 +07:00
DavHau 813c113b9e initialize test suite for git fetchers
solves #9388

This utilizes nixos vm tests to allow:
- writing tests for fetchTree and fetchGit involving actual networking.
- writing small independent test cases by automating local and remote repository setup per test case.

This adds:
  - a gitea module setting up a gitea server
  - a setup module that simplifies writing test cases by automating the repo setup.
  - a simple git http test case

Other improvements:
For all nixos tests, add capability of overriding the nix version to test against.
This should make it easier to prevent regressions. If a new test is added it can simply be ran against any older nix version without having to backport the test.
For example, for running the container tests against nix 2.12.0:
`nix build "$(nix eval --raw .#hydraJobs.tests.containers --impure --apply 't: (t.forNix "2.12.0").drvPath')^*" -L`
2024-01-11 14:44:00 +07:00
John Ericson 423484ad26 Only link with -pthread on Unix
We don't want this with MinGW.
2024-01-10 20:38:39 -05:00
John Ericson f9e5eb5f0a Make indentation in makesfiles consistent
Tab (as required) for rules, two spaces for `if`...`endif`.
2024-01-10 20:26:34 -05:00
Eelco Dolstra b91c935c2f
Merge pull request #9648 from cole-h/nix-shell-ordering
nix shell: reflect command line order in PATH order
2024-01-09 19:00:33 +01:00
John Ericson ff6de4a9ee
Merge pull request #9662 from shlevy/flat-fixed-references-assert
Improve error message for fixed-outputs with references.
2024-01-08 10:46:37 -05:00
John Ericson c4c636284e
Only test bug fix with new enough deamon 2024-01-08 10:17:28 -05:00
Shea Levy eeb2f083c5
Improve error message for fixed-outputs with references.
This codepath is possible, e.g. with a dockerTools.pullImage of an image with a Nix store.
2024-01-07 07:32:31 -05:00
John Ericson 37ea1612c7 flake: Go back to regular nixos-23.05-small
Finally get off the ad-hoc staging commit!

Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/36c4ac09e9bebcec1fa7b7539cddb0c9e837409c' (2023-11-30)
  → 'github:NixOS/nixpkgs/2c9c58e98243930f8cb70387934daa4bc8b00373' (2023-12-31)
2024-01-04 10:45:00 -05:00
Eelco Dolstra 3f834f5c64
Merge pull request #9657 from edolstra/fix-getNameFromURL
getNameFromURL(): Support uppercase characters in attribute names
2024-01-02 12:48:17 +01:00
w 4e3dc5f925 tests: test with conflicting profile links 2023-12-30 06:24:06 +00:00
Rebecca Turner 7434caca05
Fix segfault on infinite recursion in some cases
This fixes a segfault on infinite function call recursion (rather than
infinite thunk recursion) by tracking the function call depth in
`EvalState`.

Additionally, to avoid printing extremely long stack traces, stack
frames are now deduplicated, with a `(19997 duplicate traces omitted)`
message. This should only really be triggered in infinite recursion
scenarios.

Before:

    $ nix-instantiate --eval --expr '(x: x x) (x: x x)'
    Segmentation fault: 11

After:

    $ nix-instantiate --eval --expr '(x: x x) (x: x x)'
    error: stack overflow

           at «string»:1:14:
                1| (x: x x) (x: x x)
                 |              ^

    $ nix-instantiate --eval --expr '(x: x x) (x: x x)' --show-trace
    error:
           … from call site
             at «string»:1:1:
                1| (x: x x) (x: x x)
                 | ^

           … while calling anonymous lambda
             at «string»:1:2:
                1| (x: x x) (x: x x)
                 |  ^

           … from call site
             at «string»:1:5:
                1| (x: x x) (x: x x)
                 |     ^

           … while calling anonymous lambda
             at «string»:1:11:
                1| (x: x x) (x: x x)
                 |           ^

           … from call site
             at «string»:1:14:
                1| (x: x x) (x: x x)
                 |              ^

           (19997 duplicate traces omitted)

           error: stack overflow
           at «string»:1:14:
                1| (x: x x) (x: x x)
                 |              ^
2023-12-29 22:16:44 -08:00
Shea Levy c3942ef85f
Build IFD in the build store when using eval-store.
Previously, IFDs would be built within the eval store, even though one
is typically using `--eval-store` precisely to *avoid* local builds.

Because the resulting Nix expression must be copied back to the eval
store in order to be imported, this requires the eval store to trust
the build store's signatures.
2023-12-23 21:33:56 -05:00
Shea Levy 9cb287657b
remote-store test: Break out IFD expression into a separate file 2023-12-23 17:15:09 -05:00
Eelco Dolstra 936a364226 getNameFromURL(): Support uppercase characters in attribute names
In particular, this makes it handle 'legacyPackages' correctly.
2023-12-22 16:35:58 +01:00
Eelco Dolstra 6268a45b65 nix profile: Make profile element names stable
The profile manifest is now an object keyed on the name returned by
getNameFromURL() at installation time, instead of an array. This
ensures that the names of profile elements don't change when other
elements are added/removed.
2023-12-22 16:21:30 +01:00
Eelco Dolstra 3187bc9ac3 nix profile: Remove indices 2023-12-22 16:21:30 +01:00
Felix Uhl 4b41118663 Move flakeref tests to new flake/ subdirectory 2023-12-22 09:38:13 +01:00
Felix Uhl 26d7b0c793 Move url-name utility to libexpr/flake 2023-12-22 09:33:02 +01:00
Felix Uhl 397cf4e285 nix search: Disallow empty regex
Fixes #4739
Fixes #3553 in spirit IMO
2023-12-21 22:13:43 +01:00
Cole Helbling 1fb43d1eee tests: add a test for command line ordering 2023-12-21 10:23:07 -08:00
Eelco Dolstra 9d9d9ff0de Merge remote-tracking branch 'origin/master' into profile-names-instead-of-index 2023-12-21 16:21:26 +01:00
Eelco Dolstra 75e10e42f3
Merge pull request #9631 from cole-h/fixup-check-warnings
Fix warnings when running checks
2023-12-20 17:47:30 +01:00
Eelco Dolstra 7cfd6c0efe
Merge pull request #9325 from NixOS/accessor-add-to-store
Content addressing and adding to store cleanup
2023-12-19 15:10:31 +01:00
Rebecca Turner 0cee56db1a
Fix logging.sh test on macOS
On macOS in the `nix develop` shell, `make
tests/functional/logging.sh.test` errors:

    ++(logging.sh:18) mktemp
    +(logging.sh:18) builder=/var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.StuabKUhMh
    +(logging.sh:19) echo -e '#!/bin/sh\nmkdir $out'
    +++(logging.sh:22) mktemp -d
    ++(logging.sh:22) nix-build -E 'with import ./config.nix; mkDerivation { name = "fnord"; builder = /var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.StuabKUhMh; }' --out-link /var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.oaKcy0NXqC/result
    error:
           … while calling the 'derivationStrict' builtin
             at <nix/derivation-internal.nix>:9:12:
                8|
                9|   strict = derivationStrict drvAttrs;
                 |            ^
               10|

           … while evaluating derivation 'fnord'
             whose name attribute is located at «string»:1:42

           … while evaluating attribute 'args' of derivation 'fnord'
             at /Users/wiggles/nix/tests/functional/config.nix:23:7:
               22|       builder = shell;
               23|       args = ["-e" args.builder or (builtins.toFile "builder-${args.name}.sh" ''
                 |       ^
               24|         if [ -e "$NIX_ATTRS_SH_FILE" ]; then source $NIX_ATTRS_SH_FILE; fi;

           error: path '/var' is a symlink
    +(logging.sh:22) outp=
    ++(logging.sh:22) onError
    ++(/Users/wiggles/nix/tests/functional/common/vars-and-functions.sh:237) set +x
    logging.sh: test failed at:
      main in logging.sh:22

This is because `mktemp` returns a path like
`/var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.qDY24l6bIM`,
where `/var` is a symlink to `/private/var`.

Then, we attempt to use that path as a `builder`, which errors because
symlinks are impure or whatever.

Anyways, we can fix this by using `realpath "$(mktemp)"` instead of
`mktemp` directly.

NB: This error doesn't seem to happen when I run the tests through `nix
flake check`. I'm not sure if Nix does something to `TMP` in that case.
2023-12-18 14:04:25 -08:00
Cole Helbling 1f7b62f123 Use nix daemon in the test suite
As part of the CLI stabilization effort, the last remaining checkbox (at
the moment) for `nix daemon` is that it "needs testing". This implements
the proposal of using `nix daemon` in place of `nix-daemon` in the test
suite.
2023-12-18 11:29:55 -08:00
Cole Helbling 123ef6a996 Fix warnings when running checks
`nix flake check` had these warnings:

    trace: warning: Module argument `nodes.client.config` is deprecated. Use `nodes.client` instead.
    trace: warning: Module argument `nodes.client.config` is deprecated. Use `nodes.client` instead.
    trace: warning: The option `services.openssh.permitRootLogin' defined in `/nix/store/3m3hfpmbjdf4w39qfjami7ljhvhczay1-source/tests/nixos/nix-copy.nix' has been renamed to `services.openssh.settings.PermitRootLogin'.
    trace: warning: Module argument `nodes.http_dns.config` is deprecated. Use `nodes.http_dns` instead.
    trace: warning: Module argument `nodes.github.config` is deprecated. Use `nodes.github` instead.
    trace: warning: Module argument `nodes.sourcehut.config` is deprecated. Use `nodes.sourcehut` instead.
2023-12-18 10:20:52 -08:00
John Ericson dfc876531f Organize content addressing, use SourceAccessor with Store::addToStore
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:41:54 -05:00
Rebecca Turner 5cb98095ba
Remove some blank lines from stack traces
This keeps hint messages, source location information, and source code
snippets grouped together, while making stack traces shorter (so that
more stack frames can be viewed on the same terminal).

Before:

    error:
           … while evaluating the attribute 'body'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:

                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:

                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed

           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|

After:

    error:
           … while evaluating the attribute 'body'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:
                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:
                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed
           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|
2023-12-15 23:57:26 -08:00
John Ericson 70f50cbb2a Functional Test for builtins.storeDir and builtins.currentSystem
These were under-tested. This tests the status quo and especially
previous commit of this PR better.
2023-12-14 19:47:10 -05:00
Matthew Bauer bcbdb09ccf Add eval-system option
`eval-system` option overrides just the value of `builtins.currentSystem`.
This is more useful than overriding `system` since you can build these
derivations on remote builders which can work on the given system.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-14 19:04:00 -05:00
Ramses 1e3d811840
worker protocol: serialise cgroup stats in BuildResult (#9598)
By doing so, they get reported when building through the daemon via either `unix://` or `ssh-ng://`.
2023-12-13 16:37:17 -05:00
John Ericson e76df87814 Test nix copy --substitute-on-destination
It works with both `ssh://` and `ssh-ng://` now since #9600 (and
`ssh-ng:// didn't work before that).

Also, by making the two tests share code, we nudge ourselves towards
making sure there is feature parity.
2023-12-13 15:22:19 -05:00
Eelco Dolstra 1b7968ed86
Merge pull request #9547 from hercules-ci/allowed-scheme-without-slash
`allowed-uris`: match whole schemes without slashes
2023-12-13 20:23:33 +01:00
Eelco Dolstra 19ec1c9fd4 Improve the unsafeGetAttrPos test
We can use corepkgsFS->addFile() now to create a "real" position.
2023-12-13 15:15:30 +01:00
Eelco Dolstra faa4cae9ae LibExprTest: Ignore $NIX_PATH
Otherwise a broken $NIX_PATH can cause the test suite to fail.
2023-12-13 13:27:39 +01:00
Robert Hensing 0b87ba50c0 Revert "Add nix::isASCII*, locale-independent"
This reverts commit 79eb2920bb.

Not used at this time.
2023-12-12 17:46:34 +01:00
Robert Hensing 4eaeda6604 isValidSchemeName: Use regex
As requested by Eelco Dolstra. I think it used to be simpler.
2023-12-12 17:46:34 +01:00
Théophane Hufschmitt 0dfa66d120
Merge pull request #9593 from B4dM4n/fix-path-like-flake-query
Fix query parsing for path-like flakes
2023-12-12 17:15:30 +01:00
Théophane Hufschmitt 589fb105f3 Fix the VM tests
Work around https://github.com/NixOS/nixpkgs/issues/271146 until we can
depend on a Nixpkgs version containing https://github.com/NixOS/nixpkgs/pull/271423
2023-12-12 16:05:32 +01:00
John Ericson eae2717e00 tests: Use cp -ar instead of tar-untar pipe 2023-12-11 13:56:18 -05:00
John Ericson 8d0a03b5a2 Fix tests after last rename (path -> pathInLowerStore) 2023-12-11 13:48:42 -05:00
John Ericson c90e46d3f0
Update tests/functional/local-overlay-store/common.sh
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-12-11 13:45:46 -05:00
John Ericson c93f78f6fa Fix test a bit from previous commit 2023-12-11 13:36:53 -05:00
John Ericson 6bb13358e6
Update tests/functional/local-overlay-store/redundant-add-inner.sh
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-12-11 13:34:09 -05:00
John Ericson bf0bf3d1be local-overlay store tests: storeDirs -> setupStoreDirs
Co-Authored-By: Théophane Hufschmitt <theophane.hufschmitt@tweag.io>
2023-12-11 13:30:40 -05:00
John Ericson 245af3ea02 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-12-11 13:12:09 -05:00
John Ericson 9f39dda66c Fix building CA derivations with and eval store
I don't love the way this code looks. There are two larger problems:

- eval, build/scratch, destination stores (#5025) should have different
  types to reflect the fact that they are used for different purposes
  and those purposes correspond to different operations. It should be
  impossible to "use the wrong store" in my cases.

- Since drvs can end up in both the eval and build/scratch store, we
  should have some sort of union/layered store (not on the file sytem
  level, just conceptual level) that allows accessing both. This would
  get rid of the ugly "check both" boilerplate in this PR.

Still, it might be better to land this now / soon after minimal cleanup,
so we have a concrete idea of what problem better abstractions are
supposed to solve.
2023-12-11 12:17:36 -05:00
Fabian Möller 994f1b5c0d
Add test cases for flake urls with fragments 2023-12-11 16:05:34 +01:00
Robert Hensing a05bc9eb92 allowed-uris: Match whole schemes also when scheme is not followed by slashes 2023-12-11 12:18:04 +01:00
Robert Hensing d3a85b6834 isValidSchemeName: Add function 2023-12-11 12:12:43 +01:00
Robert Hensing 79eb2920bb Add nix::isASCII*, locale-independent 2023-12-11 12:12:43 +01:00
Robert Hensing 91ba7b2307 isAllowedURI: Extract function and test 2023-12-11 12:12:42 +01:00
John Ericson 5417990e31 Create ServeProto::BuildOptions and a serializer for it
More tests, and more serializers for Hydra reuse.
2023-12-09 11:35:13 -05:00
Robert Hensing c8458bd731
Merge pull request #9555 from 9999years/positions-in-errors
Pass positions when evaluating
2023-12-09 03:55:58 +01:00
Robert Hensing 6e8d598314 tests/lang/eval-fail-bad-string-interpolation-4: init 2023-12-09 02:52:49 +01:00
Robert Hensing 9b7b7a7561 Revert "Print the value in error: cannot coerce messages (#9553)"
This reverts commit f0ac2a35d5.

The request from the sibling PR, which also applies here, was not addressed.
https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735
2023-12-09 02:13:32 +01:00
Robert Hensing d4f6b1d38b
Merge pull request #9497 from edolstra/move-access-control
Move restricted/pure-eval access control out of the evaluator and into the accessor
2023-12-08 22:21:50 +01:00
Rebecca Turner f0ac2a35d5
Print the value in error: cannot coerce messages (#9553)
* Print the value in `error: cannot coerce` messages

This extends the `error: cannot coerce a TYPE to a string` message
to print the value that could not be coerced. This helps with debugging
by making it easier to track down where the value is being produced
from, especially in errors with deep or unhelpful stack traces.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-08 16:36:57 +00:00
Rebecca Turner 0b80935c22
Pass positions when evaluating
This includes position information in more places, making debugging
easier.

Before:

```
$ nix-instantiate --show-trace --eval tests/functional/lang/eval-fail-using-set-as-attr-name.nix
error:
       … while evaluating an attribute name

         at «none»:0: (source not available)

       error: value is a set while a string was expected
```

After:

```
error:
       … while evaluating an attribute name

         at /pwd/lang/eval-fail-using-set-as-attr-name.nix:5:10:

            4| in
            5|   attr.${key}
             |          ^
            6|

       error: value is a set while a string was expected
```
2023-12-07 10:27:21 -08:00
John Ericson a5521b7d94 Factor out ServeProto::Serialiser<UnkeyedValidPathInfo> and test
In the process, partially undo e89b5bd0bf
in that the ancient < 2.4 version is now supported again by the
serializer again. `LegacySSHStore`, instead of also asserting that the
version is at least 4, just checks that `narHash` is set.

This allows us to better test the serializer in isolation for both
versions (< 4 and >= 4).
2023-12-07 11:34:18 -05:00
Peter Kolloch e9a5365db6 hash.sh: Make failure tests more tolerant of additional output
"warning: you don'\''t have Internet access; disabling some network-dependent features" ...

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch bbba2055f0 Refactor concurrently added tests to use HashAlgorithm.
https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch fc6f29053a Renamed HashFormat::Base32 to HashFormat::Nix32
...and also adjusted parsing accordingly.

Also added CLI completion for HashFormats.

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch 5334c9c792 HashType: Rename to HashAlgorithm
To be consistent with CLI, nix API
and many other references.

As part of this, we also converted it to a scoped enum.

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch 0c2d5f7673 nix hash convert: s/--type/--algo/ + more functional tests
https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch 6bbd900d4f nix hash convert: added
This deviated from the proposal! See comments on the issue.

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:34 +01:00
Eelco Dolstra 53ab5d87c2 Use expectStderr 2023-12-06 14:05:32 +01:00
John Ericson b23273f6a2 Add missing -pthread for test support libraries
This is good in general (see how the other libraries also have long had
it, since 49fe9592a4) but in particular
needed to fix the NetBSD build.
2023-12-05 18:18:30 -05:00
Eelco Dolstra 83c067c0fa PosixSourceAccessor: Don't follow any symlinks
All path components must not be symlinks now (so the user needs to
call `resolveSymlinks()` when needed).
2023-12-05 23:02:59 +01:00
Eelco Dolstra 345f79d016 Check that we can't follow symlinks outside of the allowed paths 2023-12-05 16:33:02 +01:00
wh0 2e5abc0fd0
tests: avoid a chroot store without sandbox support 2023-12-03 17:18:58 -08:00
John Ericson 91b6833686 Move tests to separate directories, and document
Today, with the tests inside a `tests` intermingled with the
corresponding library's source code, we have a few problems:

- We have to be careful that wildcards don't end up with tests being
  built as part of Nix proper, or test headers being installed as part
  of Nix proper.

- Tests in libraries but not executables is not right:

  - It means each executable runs the previous unit tests again, because
    it needs the libraries.

  - It doesn't work right on Windows, which doesn't want you to load a
    DLL just for the side global variable . It could be made to work
    with the dlopen equivalent, but that's gross!

This reorg solves these problems.

There is a remaining problem which is that sibbling headers (like
`hash.hh` the test header vs `hash.hh` the main `libnixutil` header) end
up shadowing each other. This PR doesn't solve that. That is left as
future work for a future PR.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-01 10:48:58 -05:00
John Ericson 82359eba6b
Merge pull request #9233 from bouk/bouk/apply-config-inner
config: add included files into parsedContents before applying
2023-12-01 08:23:32 -05:00
Eelco Dolstra ea95327e72 Move restricted/pure-eval access control out of the evaluator and into the accessor 2023-11-30 16:16:17 +01:00
John Ericson 52e0911302 Use buildprefix in a few more places
`installcheck` doesn't yet work, but the rest of the build can now
happen mostly inside a separate build directory.

Progress on #9342

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-11-29 19:49:07 -05:00
Théophane Hufschmitt f300e11b05 Rename nix show-config to nix config show
Part of #7672
2023-11-28 14:28:19 +01:00
Bob van der Linden 06a745120b
nix: develop: remove test for interactive shell executable 2023-11-26 21:27:46 +01:00
Théophane Hufschmitt bf13943206
Merge pull request #9173 from vkryachko/transitive-input-overrides
Fix transitive input locking.
2023-11-23 08:54:45 +01:00
Matej Urbas 03c3af1bf9 mounted-ssh-ng store: integration tests 2023-11-21 13:35:12 -05:00
Eelco Dolstra c185e45ed7
Merge pull request #9410 from edolstra/shallow
Some shallow git fetching improvements
2023-11-21 12:58:01 +01:00
John Ericson 516e7ddc41
Merge pull request #9257 from Artturin/nixenvjsondrvpath
`nix-env --query`: fix `--json` ignoring `--drv-path`
2023-11-20 16:32:35 -05:00
Eelco Dolstra e4066c0444 Fetch specific Git revisions
This is more efficient, and necessary when using shallow=1 with a rev.
2023-11-20 17:25:40 +01:00
John Ericson 4ab27e5595 Merge remote-tracking branch 'upstream/master' into libgit2 2023-11-20 08:30:26 -05:00
John Ericson 87ac33f29a
Merge branch 'master' into nixenvjsondrvpath 2023-11-18 13:47:14 -05:00
Théophane Hufschmitt b1e7d7cad6
Merge pull request #9280 from R-VdP/rvdp/fix_remote_logging_phase_reporting
Include phase reporting in log file for ssh-ng builds
2023-11-17 14:37:09 +01:00
Eelco Dolstra 31ebc6028b Fix symlink handling
This restores the symlink handling behaviour prior to
94812cca98.

Fixes #9298.
2023-11-16 16:45:14 +01:00
Bob van der Linden eaa3629532
fixup! nix: develop: add tests for interactive shell 2023-11-16 15:12:31 +01:00
Bob van der Linden ceab20d056
nix: develop: add tests for interactive shell 2023-11-16 09:37:49 +01:00
Théophane Hufschmitt 84128461b6 Add a new nix store add command
Deprecate `nix store add-file` and `nix store add-path`, and replace
them with a single `nix store add` command.
2023-11-15 19:21:17 +01:00
John Ericson 9afa697ab6 Refactor bash test build system a bit
The basic idea here is to separate a few intertwined notions:

1. Not all "run bash tests" are "install tests"

2. Not all "run bash tests" use `tests/functional/init.sh`, or any
   pre-test initialization at all.

This will used in the next commit when we have a test that check unit
test golden master data.

Also, move our custom `PS4` from the test to the test runner, as it is
part of how we want to display the tests, not the test themselves.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-11-10 11:02:37 -05:00
Artturin a903f85f84 nix-env --query: fix --json ignoring --drv-path
```json
{
  "AMB-plugins": {
    "drvPath": "/nix/store/l99cb7h2hy8dg005arsjbd9kx0w05d3h-AMB-plugins-0.8.1.drv",
    "name": "AMB-plugins-0.8.1",
    "outputName": "out",
    "outputs": {
      "out": null
    },
    "pname": "AMB-plugins",
    "system": "x86_64-linux",
    "version": "0.8.1"
  },
  "ArchiSteamFarm": {
    "drvPath": "/nix/store/nhplgyjj34fz6hjmnyih25gxscfh8s7b-ArchiSteamFarm-5.4.12.5.drv",
    "name": "ArchiSteamFarm-5.4.12.5",
    "outputName": "out",
    "outputs": {
      "out": null
    },
    "pname": "ArchiSteamFarm",
    "system": "x86_64-linux",
    "version": "5.4.12.5"
  },
...
```
2023-11-09 22:31:32 +02:00
Eelco Dolstra 98a120b8b8 Merge remote-tracking branch 'origin/master' into libgit2 2023-11-09 16:48:41 +01:00
John Ericson f0adb72c23 Mark fetchTree as unstable again
As discussed in our last meeting, we need a bit more time, but we are
"time boxing" the work left to do to ensure there is not unbounded
delay.

Rather than putting it back underneath `flakes`, though, put it
underneath its own `fetch-tree` experimental feature (which `flakes`
includes/implies). This signals our commitment to the plan to stabilize
it first without waiting to go through the rest of Flakes, and also will
give users a "release candidate" when we get closer to stabilization.

This reverts commit 4112dd1fc9.
2023-11-08 23:23:56 -05:00
Robert Hensing 466271568b nix: Parse --file relative to shebang script 2023-11-07 13:24:01 -05:00
Robert Hensing 198bc22e3b nix: Add command baseDir to parse --expr relative to shebang script 2023-11-07 13:24:01 -05:00
Robert Hensing 20ff61ab25 nix: Reserve shebang line syntax and only parse double backtick quotes
Being restrictive about syntax leaves opportunity to improve the
syntax and functionality later.
2023-11-07 13:24:01 -05:00
Tom Bereknyei 06f3583b1c feat: break out of shebang processing for non-comments 2023-11-07 13:24:01 -05:00
Tom Bereknyei bfcbf3b5bf doc: shebang release notes, docs, tests
fix: release notes
2023-11-07 13:24:01 -05:00
Tom Bereknyei eea5a003d9 fix: test to ensure arguments are passed 2023-11-07 13:24:01 -05:00
Tom Bereknyei 74210c12fe Shellbang support with flakes
Enables shebang usage of nix shell. All arguments with `#! nix` get
added to the nix invocation. This implementation does NOT set any
additional arguments other than placing the script path itself as the
first argument such that the interpreter can utilize it.

Example below:

```
    #!/usr/bin/env nix
    #! nix shell --quiet
    #! nix nixpkgs#bash
    #! nix nixpkgs#shellcheck
    #! nix nixpkgs#hello
    #! nix --ignore-environment --command bash
    # shellcheck shell=bash
    set -eu
    shellcheck "$0" || exit 1
    function main {
        hello
        echo 0:"$0" 1:"$1" 2:"$2"
    }
    "$@"
```

fix: include programName usage

EDIT: For posterity I've changed shellwords to shellwords2 in order
      not to interfere with other changes during a rebase.
      shellwords2 is removed in a later commit. -- roberth
2023-11-07 13:24:01 -05:00
Bob van der Linden 9c0a09f09f allow ^ in URLs
Users may select specific outputs using the ^output syntax or selecting
any output using ^*.

URL parsing currently doesn't support these kinds of output references:
parsing will fail.

Currently `queryRegex` was reused for URL fragments, which didn't
include support for ^. Now queryRegex has been split from fragmentRegex,
where only the fragmentRegex supports ^.
2023-11-06 21:21:20 +01:00
Felix Uhl 257b768436 Enable using human-readable name in nix profile 2023-11-06 21:21:20 +01:00
Mel Zuser 61d6fe059e
Fix boost::bad_format_string exception in builtins.addErrorContext (#9291)
* Fix boost::bad_format_string exception in builtins.addErrorContext

The message passed to addTrace was incorrectly being used as a format
string and this this would cause an exception when the string contained
a '%', which can be hit in places where arbitrary file paths are
interpolated.

* add test
2023-11-06 19:13:40 +00:00
Théophane Hufschmitt 06d0d51895
Merge pull request #9242 from obsidiansystems/path-info-map
Make `nix path-info --json` return an object not array
2023-11-06 18:00:42 +01:00