NixOS/nix
NixOS/nix
1
0
Fork 0
mirror of https://github.com/NixOS/nix synced 2024-09-20 23:28:26 -04:00
Code Activity
12263 commits 178 branches 170 tags 104 MiB
Commit graph

249 commits

Author SHA1 Message Date
Sergei Trofimovich d58453f72e gc: don't visit implicit referrers on garbage collection 
Before the change garbage collector was not considering
`.drv` and outputs as alive even if configuration says otherwise.
As a result `nix store gc --dry-run` could visit (and parse)
`.drv` files multiple times (worst case it's quadratic).

It happens because `alive` set was populating only runtime closure
without regard for actual configuration. The change fixes it.

Benchmark: my system has about 139MB, 40K `.drv` files.

Performance before the change:

    $ time nix store gc --dry-run
    real    4m22,148s

Performance after the change:

    $ time nix store gc --dry-run
    real    0m14,178s
 2022-03-17 18:47:29 +00:00
Sergei Trofimovich 6b1872312f nix store gc: account for auto-optimised store 
Before the change on a system with `auto-optimise-store = true`:

    $ nix store gc --verbose --max 1

deleted all the paths instead of one path (we requested 1 byte limit).

It happens because every file in `auto-optimise-store = true` has at
least 2 links: file itself and a link in /nix/store/.links/ directory.

The change conservatively assumes that any file that has one (as before)
or two links (assume auto-potimise mode) will free space.

Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
 2022-03-13 07:24:48 +00:00
Eelco Dolstra a949673a5b Fix Darwin build 
Fixes #6169
 2022-02-28 15:48:26 +01:00
Eelco Dolstra df552ff53e Remove std::string alias (for real this time) 
Also use std::string_view in a few more places.
 2022-02-25 16:13:02 +01:00
Eelco Dolstra d17d46cfc2 More debug info 2021-12-13 19:03:08 +01:00
Eelco Dolstra 05fcc681ac Retry on ECONNREFUSED 
https://hydra.nixos.org/build/161439235
 2021-12-13 19:00:19 +01:00
Eelco Dolstra 6d30f9e6fe Explicitly make GC roots client sockets blocking 
On macOS / BSD, these sockets inherit the non-blocking flag of the
server soocket, which is not what we want.

https://github.com/dotnet/runtime/issues/25069
https://bugs.python.org/issue7995

https://hydra.nixos.org/build/161439304
 2021-12-13 18:59:50 +01:00
Alex Shabalin db2e4489a5 Unify #if linux 2021-11-22 13:57:56 +01:00
Alex Shabalin a18d9269a5 Fix build warnings on MacOS 2021-11-19 15:22:31 +01:00
Eelco Dolstra 33d04e8a8d Use nix::connect() to connect to the garbage collector 2021-10-28 22:51:11 +02:00
Eelco Dolstra 22c35ea5b8 Remove unused variable 2021-10-28 22:51:11 +02:00
Eelco Dolstra 10f9a8e77d Add a test for the non-blocking GC 2021-10-15 16:52:37 +02:00
Eelco Dolstra ac54c6faa6 Fix main GC thread exiting 2021-10-15 16:36:48 +02:00
Eelco Dolstra e0936ae38f Fix crash when a GC client disconnects 
The client thread can't just delete its own thread object from
connections, it has to detach it.
 2021-10-15 16:12:21 +02:00
Eelco Dolstra 304180d0de Memoize queryReferrers() 2021-10-15 12:20:23 +02:00
Eelco Dolstra 17e6ebcc90 Speed up GC by marking entire closures as live 2021-10-14 14:13:57 +02:00
Eelco Dolstra 0154fa30cf Remove GCState 2021-10-14 13:52:49 +02:00
Eelco Dolstra 0317ffdad3 Move deleteFromStore() 2021-10-14 13:34:48 +02:00
Eelco Dolstra eab934cb2a Make the canReachRoots() traversal non-recursive 2021-10-14 12:34:32 +02:00
Eelco Dolstra 35c98a59c5 Fix GC when there are cycles in the referrers graph 
(where "referrers" includes the reverse of derivation outputs and
derivers). Now we do a full traversal to look if we can reach any
root. If not, all paths reached can be deleted.
 2021-10-13 12:12:44 +02:00
Eelco Dolstra 1785ba2980 Simplify 2021-10-13 12:12:44 +02:00
Eelco Dolstra 262520fcfe Use a thread per connection 2021-10-13 12:12:44 +02:00
Eelco Dolstra ff453b06f9 Fix auto-gc 2021-10-13 12:12:44 +02:00
Eelco Dolstra 8614cf1334 Non-blocking garbage collector 
The garbage collector no longer blocks other processes from
adding/building store paths or adding GC roots. To prevent the
collector from deleting store paths just added by another process,
processes need to connect to the garbage collector via a Unix domain
socket to register new temporary roots.
 2021-10-13 12:12:44 +02:00
Eelco Dolstra 9947f1646a Remove syncWithGC() 2021-10-13 12:12:44 +02:00
Eelco Dolstra 8eac7dfad4 Remove trash directory 2021-10-13 12:12:44 +02:00
Eelco Dolstra 4202a3bc4e UDSRemoteStore: Support the 'root' store parameter 
Useful when we're using a daemon with a chroot store, e.g.

  $ NIX_DAEMON_SOCKET_PATH=/tmp/chroot/nix/var/nix/daemon-socket/socket nix-daemon --store /tmp/chroot

Then the client can now connect with

  $ nix build --store unix:///tmp/chroot/nix/var/nix/daemon-socket/socket?root=/tmp/chroot nixpkgs#hello
 2021-06-18 17:04:11 +02:00
volth eee18f88dd
Handle amount of disk space saved by hard linking being negative 
Fixes bogus messages like "currently hard linking saves 17592186044416.00 MiB".
 2020-10-12 16:06:38 +00:00
John Ericson 6cc1541782 Split out local-fs-store.hh 
This matches the already-existing `local-fs-store.cc`.
 2020-10-09 20:18:08 +00:00
Eelco Dolstra 236d9ee7f7 lstat() cleanup 2020-09-23 19:17:28 +02:00
regnat 520895b1da Fix garbage collection of CA derivations 
Fix #4026
 2020-09-17 13:36:58 +02:00
Eelco Dolstra 8a945d6ddb
Remove gc-check-reachability 2020-09-03 11:30:15 +02:00
Eelco Dolstra 82b77a7726
addPermRoot(): Remove indirect flag 2020-09-03 11:29:34 +02:00
Eelco Dolstra 3f6e88a552 unsigned long long -> uint64_t 2020-07-30 13:34:04 +02:00
Eelco Dolstra c0dd05131e toStorePath(): Return a StorePath and the suffix 2020-07-13 16:25:48 +02:00
Eelco Dolstra 29542865ce Remove StorePath::clone() and related functions 2020-06-16 22:20:18 +02:00
Eelco Dolstra 1fb762d11f Get rid of explicit ErrorInfo constructors 2020-06-15 14:06:58 +02:00
Ben Burdette ef9dd9f9bc formatting and a few minor changes 2020-05-13 15:56:39 -06:00
Ben Burdette c79d4addab consistent capitalization 2020-05-13 10:02:18 -06:00
Ben Burdette bfca5fc395 change status messages to info level 2020-05-13 09:52:36 -06:00
Ben Burdette 59b1f5c701 Merge branch 'master' into errors-phase-2 2020-05-11 14:35:30 -06:00
Ben Burdette e3901638b5 todo removal 2020-05-06 15:01:13 -06:00
Eelco Dolstra 74a1bfdcab
Merge pull request #3546 from guibou/nix_readfile_on_0_sized_files 
builtins.readFile: do not truncate content
 2020-05-06 11:33:55 +02:00
Jude Taylor e2fc575c61 nix auto-gc: use fragment size 2020-05-04 14:42:06 -07:00
Ben Burdette 4b99c09f5c convert some errors 2020-05-01 14:32:06 -06:00
Guillaume Bouchard 7afcb5af98 Remove the drain argument from readFile 
Now it is always `drain` (see previous commit).
 2020-04-29 18:43:45 +02:00
Ben Burdette e4fb9a3849 remove 'format' from Error constructor calls 2020-04-21 17:07:07 -06:00
Philipp Middendorf 04bedda0b6 gc.cc: Ignore hidden files in temproots 2020-04-09 17:05:29 +02:00
Eelco Dolstra 22a754c091
Fix GC failures on bad store path names 
It failed on names like '/nix/store/9ip48nkc9rfy0a4yaw98lp6gipqlib1a-'.
 2020-02-28 18:07:10 +01:00
Daiderd Jordan 66fccd5832
build: fix sandboxing on darwin 
Starting ba87b08f85 getEnv now returns an
std::optional which means these getEnv() != "" conditions no longer happen
if the variables are not defined.
 2020-01-05 20:23:52 +01:00