1
0
Fork 0
mirror of https://github.com/NixOS/nix synced 2024-09-19 10:50:24 -04:00
Commit graph

11056 commits

Author SHA1 Message Date
Eelco Dolstra caf4e98f0c Log download durations 2024-07-24 20:10:41 +02:00
Ryan Hendrickson e086d5d899 libexpr: experimental pipe operators 2024-07-24 13:17:28 -04:00
Robert Hensing 3172e88af5 Make abort() call sites log first 2024-07-24 16:52:04 +02:00
Eelco Dolstra b8684eb447 sourceToSink(): Fix abort 2024-07-24 16:44:00 +02:00
Eelco Dolstra ca0f7db843 Remove hacks to support evaluation from coroutines
Since we're not doing this anymore.
2024-07-24 16:43:25 +02:00
Eelco Dolstra 609df83c01 sourceToSink(): Throw EndOfFile 2024-07-24 16:24:38 +02:00
Eelco Dolstra 0194f81587 addToStore(): Do evaluation on the main stack
This hopefully avoids the need for all our Boehm GC coroutine
workarounds, since the GC roots will be on the main stack of the
thread.

Fixes #11141.
2024-07-24 16:24:38 +02:00
Eelco Dolstra 3be7c0037e WorkerProto: Support fine-grained protocol feature negotiation
Currently, the worker protocol has a version number that we increment
whenever we change something in the protocol. However, this can cause
a collision between Nix PRs / forks that make protocol changes
(e.g. PR #9857 increments the version, which could collide with
another PR). So instead, the client and daemon now exchange a set of
protocol features (such as `auth-forwarding`). They will use the
intersection of the sets of features, i.e. the features they both
support.

Note that protocol features are completely distinct from
`ExperimentalFeature`s.
2024-07-24 16:23:37 +02:00
Robert Hensing 68693276f9 Update fixupBoehmStackPointer doc
lo might have made sense in the bdwgc code, maybe?, but not here.
2024-07-24 13:02:49 +02:00
Robert Hensing 907b0a371a Rename osStackLow -> osStackLimit
This is in accordance with ARM's naming convention.
"Low" is confusing, because it could refer to either the cold end
of the stack as an abstract data type, or a low address.
These are different places, because the stack grows down through
the address space.
2024-07-24 13:02:49 +02:00
Robert Hensing e48e0cbab0 markdown.hh: Improve includes 2024-07-24 12:54:40 +02:00
Robert Hensing 0bd2d36375 Document renderMarkdownToTerminal 2024-07-24 12:53:37 +02:00
Robert Hensing 7d4d34a27d eval-gc.cc: Fix warning 2024-07-24 12:48:38 +02:00
Robert Hensing ca2cc26e12 tests/functional/repl: Improve precision and readability
... as well as match buildReadlineNoMarkdown.

Unfortunately it doesn't support long inputs or multiline inputs
for now.
This needs to make better use of the interacter interface.
2024-07-24 12:48:38 +02:00
Robert Hensing 712ce2feac ReadlineLikeInteracter::getLine: Add _NIX_TEST_REPL_ECHO env var
... for testing
2024-07-24 12:48:38 +02:00
Robert Hensing 97b0114ab8 renderMarkdownToTerminal: Add _NIX_TEST_RAW_MARKDOWN env var
For testing only.
2024-07-24 12:48:38 +02:00
Robert Hensing 2b4e3f04a4 markdown.cc: Format
Slightly custom because the automated formatting messes up the
braced initializer with named fields.
2024-07-24 01:02:48 +02:00
Robert Hensing 5d6bc484be Hide getGCCycles when we have no GC
Alternatively, we could make it return 0, but we don't need it in
the first place because the caller exists conditionally too.
2024-07-23 16:24:43 +02:00
Robert Hensing eb89e50cbb Rejiggle getGCCycles() for buildNoGc 2024-07-23 16:21:43 +02:00
Eelco Dolstra 498eed0a25 ~FramedSource(): Don't throw an exception if the remote has disconnected
This would cause the daemon to crash with a call to terminate().
2024-07-23 10:56:29 +02:00
Eelco Dolstra babfd0cd99
Merge pull request #11150 from hercules-ci/issue-11141-broken-sp-corrector
Fix issue #11141 broken stack pointer corrector
2024-07-22 19:59:46 +02:00
John Ericson 046cc8cbb3
Merge pull request #11156 from obsidiansystems/fix-bsd-pthread
Fix some BSD builds missing pthread functions
2024-07-22 13:58:41 -04:00
John Ericson da625b8817
Merge pull request #11155 from obsidiansystems/build-system-fixes
Two Build system fixes
2024-07-22 12:16:56 -04:00
John Ericson d3cee8160c Add missing threads deps 2024-07-22 11:46:54 -04:00
Eelco Dolstra 9b9fc405d2
Merge pull request #11154 from DeterminateSystems/ping-store
nix ping-store: Redirect to 'nix store info'
2024-07-22 17:38:51 +02:00
Eelco Dolstra d08bb025e1
Merge pull request #11043 from hercules-ci/assert-eq
`assert`: Report why values aren't equal
2024-07-22 17:34:28 +02:00
John Ericson d7024ac9b7 Add S3 opt dep to Meson, and simplify build
Numeric version macros are now defined upstream, so we don't need roll
our own.
2024-07-22 11:11:38 -04:00
John Ericson 823baa25f3 Meson build: libstore check for statvfs 2024-07-22 11:09:53 -04:00
Eelco Dolstra 4878c31815 nix ping-store: Redirect to 'nix store info'
This avoids the double warning

   warning: 'ping-store' is a deprecated alias for 'store ping'
   warning: 'nix store ping' is a deprecated alias for 'nix store info'
2024-07-22 16:39:02 +02:00
Eelco Dolstra 40f80e1b5c
Merge pull request #11142 from detroyejr/fix-alias-flags
Allow flag aliases
2024-07-22 15:53:41 +02:00
Eelco Dolstra 56a2f7ab1d
Merge pull request #11102 from fzakaria/issue-10532
doc: Add comment for fetchurl for name & url
2024-07-22 15:49:41 +02:00
Robert Hensing 380becf0db Fix #11141 broken sp corrector 2024-07-22 14:52:08 +02:00
Robert Hensing b16861d82e libexpr: Track and show GC time and cycle number 2024-07-22 12:56:01 +02:00
poweredbypie 0ec5e3a1bc
Progress on Wine CI support, MinGW dev shell with Meson (#10975)
* Only build perl subproject on Linux

* Fix various Windows regressions

* Don't put the emulator hook in test builds

  We run the tests in a separate derivation. Only need it for the dev shell.

* Fix native dev shells

* Fix cross dev shells we don't know how to emulate

Co-authored-by: PoweredByPie <poweredbypie@users.noreply.github.com>
Co-authored-by: Joachim Schiele <js@lastlog.de>
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2024-07-21 22:03:04 +00:00
Robert Hensing 56757e15cf
Merge pull request #11123 from tweag/fix-non-det-parser-printing
Fix non-deterministic inherit printing
2024-07-20 19:12:48 +02:00
Robert Hensing 584f8cb065
Merge pull request #11120 from tweag/early-string-cutoff
parser: Remove empty multiline string parts earlier
2024-07-20 19:10:50 +02:00
detroyejr 74dccef004 addFlag: use aliases 2024-07-19 15:05:00 -04:00
Eelco Dolstra c4213f0e6c
Merge pull request #11125 from DeterminateSystems/basic-connection
Factor out commonality between WorkerProto::Basic{Client,Server}Connection
2024-07-19 14:26:04 +02:00
Eelco Dolstra 58a79b6943 performOp(): Take a WorkerProto::BasicServerConnection 2024-07-19 13:35:46 +02:00
Silvan Mosberger 0c91bb97e5 parser: Remove empty multiline string parts earlier
Makes parsing more consistent and is a super minor optimisation

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-19 00:43:44 +02:00
Silvan Mosberger a6dccae223 Fix non-deterministic parser printing
In _very_ rare cases (I had about 7 cases out of 32200 files!),
the order of how inherit-from bindings are printed when using
`nix-instantiate --parse` gets messed up.

The cause of this seems to be because the std::map the bindings are
placed in is keyed on a _pointer_, which then uses an
[implementation-defined strict total order](https://en.cppreference.com/w/cpp/language/operator_comparison#Pointer_total_order).

The fix here is to key the bindings on their displacement instead,
which maintains the same order as they appear in the file.

Unfortunately I wasn't able to make a reproducible test for this in the
source, there's something about the local environment that makes it
unreproducible for me.

However I was able to make a reproducible test in a Nix build on a Nix
version from a very recent master:

    nix build github:infinisil/non-det-nix-parsing-repro

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-19 00:42:00 +02:00
Emily 31e151386b libmain: add missing header include 2024-07-18 23:26:22 +01:00
Eelco Dolstra fa7aa0389a FdSource: Fix operator =
This wasn't moving the underlying buffer, so if the buffer was
non-empty, it could lose data.
2024-07-18 16:10:48 +02:00
Eelco Dolstra d231d802f5 Typo 2024-07-18 16:10:48 +02:00
Eelco Dolstra c1d5cf6f34 Factor out commonality between WorkerProto::Basic{Client,Server}Connection
This also renames clientVersion and daemonVersion to the more correct
protoVersion (since it's the version agreed to by both sides).
2024-07-18 16:10:48 +02:00
Robert Hensing 8ce4287409
Merge pull request #11086 from kognise/eval-cache-fixes
Eval cache: fix cache regressions
2024-07-18 14:57:07 +02:00
John Ericson 2aa9cf34dd Move uriSchemes to *StoreConfig
It is a property of the configuration of a store --- how a store URL is
parsed into a store config, not a store itself.

Progress towards #10766
2024-07-17 23:48:19 -04:00
Farid Zakaria 57399bfc0e
Refactor unix domain socket store config (#11109)
Following what is outlined in #10766 refactor the uds-remote-store such
that the member variables (state) don't live in the store itself but in
the config object.

Additionally, the config object includes a new necessary constructor
that takes a scheme & authority.

Tests are commented out because of linking errors with the current config system.
When there is a new config system we can reenable them.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2024-07-17 23:32:27 -04:00
Eelco Dolstra 1e1a8e8ad1
Merge pull request #11058 from hercules-ci/more-nix-shell
Make `#!nix-shell` arguments and options relative to script
2024-07-17 21:52:34 +02:00
John Ericson a915862450
Merge pull request #11126 from DeterminateSystems/dont-send-version-twice
BasicClientConnection::handshake(): Don't send our version twice
2024-07-17 15:50:17 -04:00
Eelco Dolstra b23da1ceca
Merge pull request #11100 from NixOS/pretty-print-idempotent
Pretty print idempotently
2024-07-17 21:35:27 +02:00
RTUnreal f0a1c130a1
doc: add example usage for Gitea in tarball fetcher (#11116)
Co-authored-by: Valentin Gagarin <valentin@gagarin.work>
2024-07-17 19:08:33 +00:00
Robert Hensing 83d585b423 C API: Make nix_err an enum
This generally gives a better experience with bindings generators,
possibly other tooling.

A possible risk is that some generators may not represent unknown
codes correctly.
Rust bindgen by default generates suitable code:
  * a type alias nix_err = c_int
  * individual constants for the known enum values
It does _not_ generate a closed type that can only hold the values
that were known at code generation time.

If this proves to be a problem, we could instead split the type:

`typedef int nix_err;` for return values
`enum nix_known_err` for code generation.

This would complicate the interface, so let's not do it unless it
is shown to be needed.
2024-07-17 16:54:30 +02:00
Eelco Dolstra 87f8ff23fe BasicClientConnection::handshake(): Don't send our version twice
This was accidentally introduced
in f71b4da0b3.  We didn't notice this
because the version got interpreted by the daemon as the obsolete "CPU
affinity will follow" field, and being non-zero, it would then read
another integer for the ignored CPU affinity.
2024-07-17 16:51:53 +02:00
Eelco Dolstra 621c23bbea
Merge pull request #11113 from NixOS/doc-comment-unordered-map
Doc comments: use std::unordered_map
2024-07-17 16:50:48 +02:00
Las Safin 464e5925cb
Avoid accessing uninitialized settings in own init (#11117)
The default value for the setting was evaluated by
calling a method on the object _being currently constructed_,
so we were using it before all fields were initialized.

This has been fixed by making the called method static,
and not using the previously used fields at all.

But functionality hasn't changed!
The fields were usually always zero (by chance?) anyway,
meaning the conditional path was always taken.

Thus the current logic has been kept, the code simplified,
and UB removed.

This was found with the helper of UBSan.
2024-07-17 14:10:01 +02:00
Robert Hensing da3eff60bc printList: Force item before determining whether to print multi-line 2024-07-17 13:42:05 +02:00
Robert Hensing a0635a80b2 printAttrs: Force item before determining whether to print multi-line 2024-07-17 13:42:05 +02:00
Robert Hensing f5ebaea277 Simplify PosIdx::hash()
In C++ we don't need to salt the hash.
2024-07-17 13:31:31 +02:00
Robert Hensing b230c01f73
Merge pull request #11014 from obsidiansystems/plugins-libmain
Move plugins infra to `libnixmain`
2024-07-17 09:42:09 +02:00
John Ericson 463256b9e8
Merge pull request #11122 from L-as/fix-ub
Fix ub
2024-07-16 18:51:17 -04:00
Las Safin a1f3f103bc
Check if drv is initialized in DerivationGoal::waiteeDone
It might not be set, in which case we shouldn't do anything.
Surprisingly, this somehow did not cause segfaults before?

Caught by UBSan.
2024-07-16 22:01:39 +00:00
Philip Taron 0a1a116f4b
builtins.genericClosure: fix documentation typo 2024-07-16 13:51:52 -07:00
Robert Hensing d0e9878389 Remove unused boost include and split out std-hash.hh
Splitting it out immediately answers questions like [this],
without increasing the number of compilation units.

I did consider using boost::hash_combine instead, but it doesn't seem
to be quite as capable, accepting only two arguments.

[this]: https://github.com/NixOS/nix/pull/11113#discussion_r1679991573
2024-07-16 22:31:25 +02:00
Robert Hensing 74698d54c8 Document builtins.derivation 2024-07-16 17:20:09 +02:00
Robert Hensing 64b46000ad Add std::hash<PosIdx> 2024-07-16 16:46:41 +02:00
Eelco Dolstra 3d8fa9f668 Pos::getSnippetUpTo(): Fix warning 2024-07-16 16:34:13 +02:00
Robert Hensing 6c9d62dceb Doc comments: use std::unordered_map
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-07-16 15:19:39 +02:00
Eelco Dolstra 9c6678da0e
Merge pull request #11092 from DeterminateSystems/hash-SourcePath
Use std::unordered_map for the EvalState caches
2024-07-16 11:06:43 +02:00
John Ericson 808082ea03 Ensure we can construct remote store configs in isolation
Progress towards #10766

I thought that #10768 achieved, but when I went to use this stuff (in
Hydra), turns out it did not. (Those `using FooConfig;` lines were not
working --- they are so finicky!) This PR gets the job done, and adds
some trivial unit tests to make sure I did what I intended.

I had to add add a header to expose `SSHStoreConfig`, after which the
preexisting `ssh-store-config.*` were very confusingly named files, so I
renamed them to `common-ssh-store-config.hh` to match the type defined
therein.
2024-07-15 17:32:49 -04:00
John Ericson 0feeab755a Move plugins infra to libnixmain
They are not actually part of the store layer, but instead part of the
Nix executable infra (libraries don't need plugins, executables do).

This is part of a larger project of moving all of our legacy settings
infra to libmain, and having the underlying libraries just have plain
configuration structs detached from any settings infra / UI layer.

Progress on #5638
2024-07-15 17:26:03 -04:00
John Ericson 1a273a623f Inline settings.pluginFiles.name
In theory the warning is more noisy now, but in practice this will not
happen unless the client is older than 2.14 (highly unlikely).
2024-07-15 16:50:57 -04:00
Las Safin 846869da0e
Make goals use C++20 coroutines (#11005)
undefined
2024-07-15 16:49:15 -04:00
Robert Hensing 61a4d3d45c getSnippetUpTo: Return optional
This makes it possible to certain discern failures from empty
snippets, which I think is an ok review comment.

Maybe it should do so for swapped column indexes too, but I'm not
sure.

I don't think it matters in the grand scheme. We don't even have
a real use case for `nullopt` now anyway.

Since we don't have a use case, I'm not applying this logic to
higher level functions yet.
2024-07-15 20:10:45 +02:00
Robert Hensing 03d33703ef Revert "Use HintFmt for doc comments"
Unfortunately these don't render correctly, because they go into the
markdown renderer, instead of the terminal.

```
nix-repl> :doc lib.version
Attribute '[35;1mversion[0m'

    … defined at [35;1m/home/user/h/nixpkgs/lib/default.nix:73:40[0m
```

We could switch that to go direct to the terminal, but then we should
do the same for the primops, to get a consistent look.

Reverting for now.

This reverts commit 3413e0338cbee1c7734d5cb614b5325e51815cde.
2024-07-15 19:56:40 +02:00
Robert Hensing ce31a0457f Use HintFmt for doc comments 2024-07-15 19:56:40 +02:00
Robert Hensing 6a125e65d0 Revert "Doc comments: use std::unordered_map"
hash<SourcePath> isn't implemented yet, and I can't cherry-pick
a bug-free commit yet.

This reverts commit 95529f31e3bbda99111c5ce98a33484dc6e7a462.
2024-07-15 19:56:40 +02:00
Robert Hensing ac89df815d libcmd/repl.cc: Explain evalString call and defend 2024-07-15 19:56:40 +02:00
Robert Hensing 21817473e8 Doc comments: use std::unordered_map
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-07-15 19:56:40 +02:00
Robert Hensing 131b6ccc71 nixexpr.hh: Avoid the warning and pragmas 2024-07-15 19:56:40 +02:00
Robert Hensing 6bbd493d49 libcmd/repl-interacter: INT_MAX -> numeric_limits 2024-07-15 19:56:40 +02:00
Robert Hensing 8a855296f5 tests/function/repl: Characterise the missing doc comment behavior 2024-07-15 19:56:40 +02:00
Robert Hensing 71cb8bf509 libexpr: Rename "column" fields to offset
... because that's what they are.
2024-07-15 19:56:40 +02:00
Robert Hensing 77e9f9ee82 libexpr: Get rid of unused line tracking fields 2024-07-15 19:56:40 +02:00
Robert Hensing cef11b23e8 Add missing .sh in _NIX_TEST_ACCEPT=1 message 2024-07-15 19:56:40 +02:00
Robert Hensing d4f576b0b2 nix repl: Render docs for attributes 2024-07-15 19:56:40 +02:00
Robert Hensing 491b9cf415 Refactor: extract DocComment::getInnerText(PosTable) 2024-07-15 19:56:40 +02:00
Robert Hensing e68234c4f9 libexpr: Rearrange lexer files so that yylex_init_extra can be found 2024-07-15 19:56:40 +02:00
Robert Hensing 7fae378835 Track doc comments and render them in :doc 2024-07-15 19:56:40 +02:00
Robert Hensing e5af7cbeb9 libutil: Add Pos::getSnippetUpTo(Pos) 2024-07-15 18:41:11 +02:00
Farid Zakaria 945fff5674
Apply suggestions from code review
Add @edolstra suggestion fixes.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-07-15 09:12:56 -07:00
Eelco Dolstra 550b3479cf Include the accessor in the SourcePath hash 2024-07-15 15:46:30 +02:00
Farid Zakaria a4ce96e5f1 doc: Add comment for fetchurl for name & url
fetchurl can be given a name and url aside from just the url.
Giving a name can be useful if the url has invalid characters such as
tilde for the store.
2024-07-14 19:07:18 -07:00
Robert Hensing 7e604f716c concatStrings: Give compiler access to definition for inlining
... at call sites that are may be in the hot path.

I do not know how clever the compiler gets at these sites.
My primary concern is to not regress performance and I am confident
that this achieves it the easy way.
2024-07-14 12:20:45 +02:00
Robert Hensing 97e01107ec dropEmptyInitThenConcatStringsSep -> concatStringSep: empty separator
When the separator is empty, no difference is observable.

Note that concatStringsSep has centralized definitions. This adds the
required definitions. Alternatively, `strings-inline.hh` could be
included at call sites.
2024-07-14 12:10:39 +02:00
Robert Hensing d40fdb5711 dropEmptyInitThenConcatStringsSep: Update doc and deprecate 2024-07-14 11:50:20 +02:00
Robert Hensing 1c97718146 dropEmptyInitThenConcatStringsSep: Allow it to drop items again
It's usually harmless, if it occurs at all.
2024-07-13 03:06:24 +02:00
Robert Hensing 6b2c277c36 dropEmptyInitThenConcatStringsSep -> concatStringSep: sigs are not empty
... but if they are, I'd like to see at least a hint of it so that
I'd know to fix it.
2024-07-13 03:06:24 +02:00
Robert Hensing 76b2d5ef3d dropEmptyInitThenConcatStringsSep -> concatStringSep: PATH handling
It's still wrong, but one step closer to correct.
Not that anyone should use "" or "." in their PATH, but that is not
for us to intervene.
2024-07-13 03:06:24 +02:00
Robert Hensing 9ca42d5da2 dropEmptyInitThenConcatStringsSep -> concatStringSep: setting value was already harmed
Considering that `value` was probably parsed with tokenizeString
prior, it's unlikely to contain empty strings, and we have no
reason to remove them either.
2024-07-13 03:06:24 +02:00
Robert Hensing 4029426ca8 dropEmptyInitThenConcatStringsSep -> concatStringSep: tokens from tokenizeString are not empty 2024-07-13 03:06:24 +02:00
Robert Hensing 0fe3525223 illegal configuration line -> syntax error in configuration line
The law has nothing to do with this, although I do feel like a
badass when I mess with the config.
I'm a conf artist.
2024-07-13 03:06:24 +02:00
Robert Hensing cf3c5cd189 dropEmptyInitThenConcatStringsSep -> concatStringSep: showVersions version is not empty 2024-07-13 03:06:24 +02:00
Robert Hensing d9043021df dropEmptyInitThenConcatStringsSep -> concatStringSep: break nix help "" "" "" build
Garbage in, error out. Experimental CLI. Zero derivations given.
2024-07-13 03:06:24 +02:00
Robert Hensing 062672b022 dropEmptyInitThenConcatStringsSep -> concatStringSep: CLI commands are not empty 2024-07-13 03:06:24 +02:00
Robert Hensing 0480bfe50b dropEmptyInitThenConcatStringsSep -> concatStringSep: do not drop attributes with empty names
Empty attributes are probably not well supported, but the least we
could do is leave a hint.
Attribute path rendering and parsing should be done according to
Nix expression syntax in my opinion.
2024-07-13 03:06:24 +02:00
Robert Hensing 4b34feb4c2 dropEmptyInitThenConcatStringsSep -> concatStringSep: system string should not be empty 2024-07-13 03:06:24 +02:00
Robert Hensing 837c3612d4 dropEmptyInitThenConcatStringsSep -> concatStringSep: escaped shell args are never empty 2024-07-13 03:06:24 +02:00
Robert Hensing 3b77f13451 dropEmptyInitThenConcatStringsSep -> concatStringSep: experimental features do not render as empty strings 2024-07-13 03:06:24 +02:00
Robert Hensing e64643bf63 dropEmptyInitThenConcatStringsSep -> concatStringSep: feature should not be empty
(System) features are unlikely to be empty strings, but when they
come in through structuredAttrs, they probably can.
I don't think this means we should drop them, but most likely they
will be dropped after this because next time they'll be parsed with
tokenizeString.

TODO: We should forbid empty features.
2024-07-13 03:06:24 +02:00
Robert Hensing f1966e22d9 dropEmptyInitThenConcatStringsSep -> concatStringSep: store paths are not empty 2024-07-13 03:06:24 +02:00
Robert Hensing 49d100ba8b dropEmptyInitThenConcatStringsSep -> concatStringSep: output name empty not feasible
I don't think it's completely impossible, but I can't construct
one easily as derivationStrict seems to (re)tokenize the outputs
attribute, dropping the empty output.

It's not a scenario we have to account for here.
2024-07-13 03:06:24 +02:00
Robert Hensing d3e49ac881 dropEmptyInitThenConcatStringsSep -> concatStringSep: shortRefs are not empty 2024-07-13 03:06:24 +02:00
Robert Hensing 608a425550 dropEmptyInitThenConcatStringsSep -> concatStringSep: diag 2024-07-13 03:06:24 +02:00
Robert Hensing 75dde71ff9 dropEmptyInitThenConcatStringsSep -> concatStringSep: sigs are non-empty
The sigs field is produced by tokenizeStrings, which does not return
empty strings.
2024-07-13 03:06:24 +02:00
Robert Hensing 3f37785afd NIX_REMOTE_SYSTEMS: actually support multiple :-separated entries
Bug not reported in 6 years, but here you go.

Also it is safe to switch to normal concatStringsSep behavior
because tokenizeString does not produce empty items.
2024-07-13 03:06:24 +02:00
Robert Hensing 39878c8979 dropEmptyInitThenConcatStringsSep -> concatStringSep: preserve empty attr
The empty attribute name should not be dropped from attribute paths.
Rendering attribute paths with concatStringsSep is lossy and wrong,
but this is just a first improvement while dealing with the
dropEmptyInitThenConcatStringsSep problem.
2024-07-13 03:06:24 +02:00
Robert Hensing ea966a70fc dropEmptyInitThenConcatStringsSep -> concatStringSep: diagnostics and docs
These are non-critical, so their behavior is ok to change.
Dropping empty items is not needed and usually not expected.
2024-07-13 03:06:24 +02:00
Robert Hensing a681d354e7 Add fresh concatStringsSep without bug
The buggy version was previously renamed to
dropEmptyInitThenConcatStringsSep
2024-07-13 03:06:24 +02:00
Robert Hensing 79eb0adf9d dropEmptyInitThenConcatStringSep: Check that we don't drop...
... initial empty strings.

The tests pass, which is encouraging.
2024-07-13 03:05:50 +02:00
Robert Hensing 1a8defd06f Refactor: rename C++ concatStringsSep -> dropEmptyInitThenConcatStringsSep 2024-07-13 03:05:50 +02:00
John Ericson bc83b9dc1f Remove comparator.hh and switch to <=> in a bunch of places
Known behavior changes:

- `MemorySourceAccessor`'s comparison operators no longer forget to
  compare the `SourceAccessor` base class.

Progress on #10832

What remains for that issue is hopefully much easier!
2024-07-12 14:54:18 -04:00
Lexi Mattick 6c4470ec2a Clean up cache for all commands 2024-07-12 11:54:12 -07:00
Lexi Mattick e764ed31f6 Eval cache: fix cache regressions
- Fix eval cache not being persisted in `nix develop` (since #10570)
- Don't attempt to commit cache transaction if there is no active transaction, which will spew errors in edge cases
- Drive-by: trivial typo fix
2024-07-12 09:45:35 -07:00
Eelco Dolstra 2a95a2d780
Merge pull request #11091 from NixOS/darwin-no-map-spaceship
Remove unused `operator<=>`'s that darwin can't generate
2024-07-12 18:40:20 +02:00
Eelco Dolstra cdc23b67a6 Provide std::hash<SourcePath> 2024-07-12 17:54:27 +02:00
Robert Hensing 11a6db5993 Remove unused operator<=>'s that darwin can't generate
It was complaining *a lot*, with dozens of MB of logs.
2024-07-12 17:37:27 +02:00
John Ericson 337a5a23b7
Merge pull request #11089 from NixOS/warnings-includes
Fix warnings and optimize includes
2024-07-12 10:29:26 -04:00
Robert Hensing 27eaeebc41 nar-accessor.cc: Silence unused variable warning 2024-07-12 15:38:17 +02:00
Robert Hensing 8df041cbc6 Solve unused header warnings reported by clangd 2024-07-12 15:37:54 +02:00
John Ericson 3fc77f281e No global settings in libnixfetchers and libnixflake
Progress on #5638

There are still a global fetcher and eval settings, but they are pushed
down into `libnixcmd`, which is a lot less bad a place for this sort of
thing.

Continuing process pioneered in
52bfccf8d8.
2024-07-12 08:50:28 -04:00
Robert Hensing f070d68c32 Add BaseError assignment operators
The move assignment was implicitly generated and used in

    src/libstore/build/goal.cc:90:22:
       90 |             this->ex = std::move(*ex);

Clang warns about this generated method being deprecated, so making
them explicit fixes the warning.
2024-07-12 14:25:16 +02:00
Eelco Dolstra 61080554ab SymbolStr: Remove std::string conversion
This refactoring allows the symbol table to be stored as something
other than std::strings.
2024-07-11 17:43:10 +02:00
Eelco Dolstra 87323a5689 Remove unused InstallableFlake::getFlakeOutputs() 2024-07-11 16:21:27 +02:00
Robert Hensing bb312a7174 Edit CompatibilitySettings 2024-07-11 13:59:49 +02:00
Robert Hensing 56bf39e905 eqValues/assertEqValues: Clean up assertions
It's still paranoid, and probably a waste of words, but at least
now it's consistent and readily identifyable from a log.
2024-07-11 12:19:29 +02:00
Robert Hensing 4fd8f19ecf Fix build to use CanonPath in new FSO sinks 2024-07-11 12:14:48 +02:00
Robert Hensing 86420753ec Merge remote-tracking branch 'upstream/master' into support-hardlinks-in-tarballs 2024-07-11 11:43:02 +02:00
Robert Hensing 61577402ba Add EvalErrorBuilder::panic()
An nicer alternative to printError + abort, or assert(false /* foo */)
2024-07-11 11:35:58 +02:00
Robert Hensing c4e3e2dc27 Soft-deprecate the compatibility settings 2024-07-10 16:24:31 +02:00
Robert Hensing d942d505ca Merge remote-tracking branch 'upstream/master' into more-nix-shell 2024-07-10 16:19:18 +02:00
Robert Hensing 13522229a9 assertEqValues: clarify potential bug error message 2024-07-10 16:08:10 +02:00
John Ericson 7a6269ba7b Package the Nix CLI with Meson
Co-Authored-By: Qyriad <qyriad@qyriad.me>
2024-07-08 17:49:44 -04:00
John Ericson f1d88e0f27
Merge pull request #11064 from obsidiansystems/header-generater
Use a meson "generator" to deduplicate `.gen.hh` creation
2024-07-08 12:03:59 -04:00
John Ericson 1439de874f
Merge pull request #11063 from obsidiansystems/config-expr
Forgot to include `config-expr.hh` in two places
2024-07-08 11:27:57 -04:00
John Ericson 6e5cec292b Use a meson "generator" to deduplicate .gen.hh creation 2024-07-08 11:13:11 -04:00
John Ericson c5284a84f3 Forgot to include config-expr.hh in some places 2024-07-08 10:37:34 -04:00
Robert Hensing cfe3ee3de8
nix-shell: look up shell.nix when argument is a directory (#11057)
* Refactor: rename runEnv -> isNixShell

* Refactor: rename left -> remainingArgs

* nix-build.cc: Refactor: extract baseDir variable

* nix-build.cc: Refactor: extract sourcePath, resolvedPath variables

* nix-shell: Look for shell.nix when directory is specified

* Add legacy setting: nix-shell-always-looks-for-shell-nix

* rl-next: Add note about shell.nix lookups

* tests/functional/shell.nix: Implement runHook for dummy stdenv
2024-07-08 14:36:36 +02:00
John Ericson d8850618b6
Merge pull request #11059 from rhendric/rhendric/reference-manual
docs: merge builtin-constants into builtins
2024-07-07 21:47:33 -04:00
Ryan Hendrickson 95890b3e1d docs: merge builtin-constants into builtins 2024-07-07 15:57:23 -04:00
Robert Hensing a9592077fb
Merge pull request #11055 from NixOS/packaging-for-nixpkgs
Indirections for packaging meson-based granular build for Nixpkgs
2024-07-07 21:33:01 +02:00
Robert Hensing fd4b17ab2c
Merge pull request #11056 from romain-neil/s3-allow-use-of-proxy
Configure aws s3 lib to use system defined proxy, if existent
2024-07-07 02:47:24 +02:00
Robert Hensing 63262e78c7 Add opt-out: nix-shell-shebang-arguments-relative-to-script 2024-07-07 00:55:33 +02:00
Robert Hensing 4c59d6e9f5 Merge branch 'nix-shell-lookup-shell-nix' into more-nix-shell 2024-07-07 00:27:07 +02:00
Robert Hensing 6c6d5263e2 Add legacy setting: nix-shell-always-looks-for-shell-nix 2024-07-07 00:25:56 +02:00
Robert Hensing f5b59fbc64 Fix and extend nix-shell baseDir test 2024-07-07 00:23:22 +02:00
Robert Hensing 8838f5c746 Merge remote-tracking branch 'matthewbauer/nix-shell-relative-shebang' into more-nix-shell 2024-07-07 00:18:03 +02:00
Robert Hensing b865625a8e nix-shell: Look for shell.nix when directory is specified 2024-07-06 23:05:34 +02:00
Robert Hensing 32fb127b9c Add legacy setting: nix-shell-always-looks-for-shell-nix 2024-07-06 23:05:34 +02:00
Romain NEIL 514062c227 feat: configure aws s3 lib to use system defined proxy, if existent 2024-07-06 21:46:58 +02:00
Robert Hensing 76245ffbeb nix-build.cc: Refactor: extract sourcePath, resolvedPath variables 2024-07-06 20:55:27 +02:00
Robert Hensing e9479b272f nix-build.cc: Refactor: extract baseDir variable 2024-07-06 20:51:45 +02:00
Robert Hensing 5c367ece89 Refactor: rename left -> remainingArgs 2024-07-06 20:03:30 +02:00
Robert Hensing 13181356fc Refactor: rename runEnv -> isNixShell 2024-07-06 20:01:46 +02:00
Robert Hensing 0729f0a113 packaging: Pass version directly 2024-07-06 17:52:57 +02:00
Robert Hensing efd5f50f5e nix-perl: Add deps, use mkMesonDerivation 2024-07-06 17:52:57 +02:00
Robert Hensing 4c014e238b nix-main: Add openssl 2024-07-06 17:52:57 +02:00
Robert Hensing 4d0c55ae55 api docs: Use mkMesonDerivation 2024-07-06 17:52:57 +02:00
John Ericson 3acf3fc746 Package libnixmain and libnixcmd with Meson
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-05 16:40:55 -04:00
Robert Hensing 0b901e10ee
Merge pull request #11050 from hercules-ci/issue-10677
Explain when `man` is missing
2024-07-05 22:25:38 +02:00
John Ericson ff9b6d0e1f
Merge pull request #11037 from fricklerhandwerk/document-config-parsing
use self-descriptive name for config file parser, document
2024-07-05 15:21:26 -04:00
Eelco Dolstra d5461b9009
Merge pull request #11051 from Mic92/fix-prefetch
src/nix/prefetch: fix prefetch containing current directory instead o…
2024-07-05 20:33:05 +02:00
Robert Hensing ddff76f667
Merge pull request #10973 from NixOS/meson-libexpr
Meson build for libexpr libflake, external C API, unit tests
2024-07-05 20:27:12 +02:00
Jörg Thalheim 05381c0b30
Update src/nix/prefetch.cc
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-07-05 19:45:03 +02:00
Jörg Thalheim 8cea1fbd97 src/nix/prefetch: fix prefetch containing current directory instead of tarball
When --unpack was used the nix would add the current directory to the
nix store instead of the content of unpacked.
The reason for this is that std::distance already consumes the iterator.
To fix this we re-instantiate the directory iterator in case the
directory only contains a single entry.
2024-07-05 19:18:49 +02:00
Robert Hensing 6ef00a503a Explain when man is missing
Have you seen this man?

Fixes #10677
2024-07-05 19:18:23 +02:00
Eelco Dolstra 8f280d72ff
Merge pull request #11019 from DeterminateSystems/fix-failed-to-open-archive
Tarball fetcher: Fix handling of cached tarballs
2024-07-05 17:10:02 +02:00
Robert Hensing a476383f46
Merge pull request #11031 from emilazy/push-xsrvoyspsvqx
libstore: fix sandboxed builds on macOS
2024-07-05 17:08:39 +02:00
Robert Hensing d63bd8295e assert: Report why values aren't equal 2024-07-05 16:43:48 +02:00
Eelco Dolstra 98bef7c38e
Merge pull request #11035 from siddhantk232/refactor
Factor duplicate code into util function `append`
2024-07-05 16:34:23 +02:00
Eelco Dolstra 61e1880847
Merge pull request #11041 from hercules-ci/trace-nix-env-attribute-names
getDerivations: add attributes to trace
2024-07-05 16:32:27 +02:00
Eelco Dolstra e1b6b3ce27
Merge pull request #11020 from DeterminateSystems/fix-tarball-caching
Tarball fetcher: Fix fetchToStore() and eval caching
2024-07-05 16:30:12 +02:00
Eelco Dolstra e7e070d36b Document 2024-07-05 16:29:16 +02:00
Robert Hensing 09763c7cad getDerivations: add attributes to trace
This improves the error message of nix-env -qa, among others, which
is crucial for understanding some ofborg eval error reports, such as
https://gist.github.com/GrahamcOfBorg/89101ca9c2c855d288178f1d3c78efef

After this change, it will report the same trace, but also start with

```
error:
       … while evaluating the attribute 'devShellTools'

       … while evaluating the attribute 'nixos'

       … while evaluating the attribute 'docker-tools-nix-shell'

       … while evaluating the attribute 'aarch64-darwin'

       … from call site
         at /home/user/h/nixpkgs/outpaths.nix:48:6:
           47|   tweak = lib.mapAttrs
           48|     (name: val:
             |      ^
           49|       if name == "recurseForDerivations" then true

<same>
```
2024-07-05 15:30:07 +02:00
John Ericson e4056b9afd
Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-04 17:48:27 -04:00
Emily af2e1142b1 libstore: fix sandboxed builds on macOS
The recent fix for CVE-2024-38531 broke the sandbox on macOS
completely. As it’s not practical to use `chroot(2)` on
macOS, the build takes place in the main filesystem tree, and the
world‐unreadable wrapper directory prevents the build from accessing
its `$TMPDIR` at all.

The macOS sandbox probably shouldn’t be treated as any kind of a
security boundary in its current state, but this specific vulnerability
wasn’t possible to exploit on macOS anyway, as creating `set{u,g}id`
binaries is blocked by sandbox policy.

Locking down the build sandbox further may be a good idea in future,
but it already has significant compatibility issues. For now, restore
the previous status quo on macOS.

Thanks to @alois31 for helping me come to a better understanding of
the vulnerability.

Fixes: 1d3696f0fb
Closes: #11002
2024-07-04 16:28:37 +01:00
Emily 76e4adfaac libstore: clean up the build directory properly
After the fix for CVE-2024-38531, this was only removing the nested
build directory, rather than the top‐level temporary directory.

Fixes: 1d3696f0fb
2024-07-04 16:22:02 +01:00
Valentin Gagarin c66079f1e8 use self-descriptive name for config file parser, document
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-04 10:36:48 +02:00
siddhantCodes 976c05879f factor duplicate code into util function append 2024-07-04 11:09:23 +05:30
Eelco Dolstra 5b4102c3b2 Tarball fetcher: Include revCount/lastModified in the fingerprint
This can influence the evaluation result so they should be included in
the fingerprint.
2024-07-03 22:05:45 +02:00
kn 10ccdb7a41 Use proper struct sockpeercred for SO_PEERCRED for OpenBSD
getsockopt(2) documents this;  ucred is wrong ("cr_" member prefix, no pid).
2024-07-03 11:16:39 -04:00
John Ericson a09360400b Ident some CPP in nix daemon
Makes it easier for me to read.
2024-07-03 11:15:56 -04:00
Eelco Dolstra 1ff186fc6e nix flake metadata: Show flake fingerprint
This is useful for testing/debugging and maybe for sharing eval caches
(since it tells you what file in ~/.cache/nix/eval-cache-v5 to copy).
2024-07-03 17:00:30 +02:00
Eelco Dolstra 9d95c228ee Tarball fetcher: Fix fetchToStore() and eval caching 2024-07-03 16:50:45 +02:00
John Ericson 30de61f16d
Merge pull request #11018 from siddhantk232/canonpath-fs-sink
Use `CanonPath` in `fs-sink.hh`
2024-07-03 10:36:18 -04:00
Eelco Dolstra 79ed3df8f8 Tarball fetcher: Fix handling of cached tarballs
Fixes a regression introduced in
5a9e1c0d20 where downloading a cached
file causes the error "Failed to open archive (Unrecognized archive
format)".
2024-07-03 14:14:20 +02:00
siddhantCodes 2cf24a2df0 fix tests and minor changes
- use the iterator in `CanonPath` to count `level`
- use the `CanonPath::basename` method
- use `CanonPath::root` instead of `CanonPath{""}`
- remove `Path` and `PathView`, use `std::filesystem::path` directly
2024-07-03 17:43:55 +05:30
Valentin Gagarin a713476790
docs: split types from syntax (#11013)
move together all syntactic and semantic information into one
page, and add a page on data types, which in turn links to the syntax and
semantics.

also split out the note on scoping rules into its own page.

Co-authored-by: Ryan Hendrickson <ryan.hendrickson@alum.mit.edu>
2024-07-03 09:03:41 +02:00
John Ericson 451f8a8c19 Put back files for now
We'll revert this sometime later
2024-07-02 09:26:22 -04:00
John Ericson 11946817f0 fileset for store unit test data 2024-07-02 09:26:22 -04:00