Skip to content

Commit 49ee9a1

Browse files
committed
nix: try using sccache
Why: the original `rustBuildIncremental` actually produces bad output in some cases. (1) it purely relies on `cargo` incremental build correctness (2) nix flakes copies the repo source to the /nix/store. (3) for reproducibility, everything in /nix/store has mtime == 1970-01-01. (4) cargo's rebuild checking only looks at the file mtime, so we get false negatives (cargo believes the file is unchanged when it is actually different). Relevant issue: [cargo - fingerprint by hash instead of mtime](rust-lang/cargo#6529) Introducing sccache: `sccache` caches individual `rustc` invocations and stores the results in `$SCCACHE_DIR = /var/cache/lexe/sccache`. Using `sccache`: - cached builds are about 0.4-0.5x time (faster) - uncached builds are about 1.1-1.2x time (extra overhead) - sccache sadly can't cache proc-macro crates...
1 parent a6b0fe4 commit 49ee9a1

7 files changed

+244
-165
lines changed

README.md

+6-10
Original file line numberDiff line numberDiff line change
@@ -289,21 +289,20 @@ $ orb shell -m linux-builder
289289
bdd9eec1fbd625eec3b2a9e2a6072f60240c930b0867b47199730b320c148e8c
290290
```
291291

292-
## Fast incremental cargo builds in `nix`
292+
## (Optional) Faster `nix` cargo rebuilds
293293

294294
Follow these steps if:
295295

296296
- You're an engineer working on the rust+nix build.
297297
- You're running `nix build` a lot and want faster incremental cargo builds.
298298

299-
Once setup, [`buildRustIncremental`](./nix/pkgs/buildRustIncremental.nix) will
300-
make incremental Rust builds MUCH faster. However, avoid using
301-
`buildRustIncremental` for reproducibility-critical packages like the SGX
302-
enclaves.
299+
Once setup, [`buildRustSccache`](./nix/pkgs/buildRustSccache.nix) will make
300+
incremental Rust rebuilds faster. However, avoid using `buildRustSccache`
301+
for reproducibility-critical packages like the SGX enclaves.
303302

304303
To get this working, we give the nix build sandbox access to a shared, global
305-
cargo `target/` directory so we can fully reuse intermediate cargo build
306-
artifacts across `nix build` invocations.
304+
directory so that `sccache` can reuse already-built `rustc` compilation
305+
artifacts across different nix builds.
307306

308307
If you're using the `linux-builder` VM, then skip this; it's already setup for
309308
you.
@@ -323,9 +322,6 @@ $ echo "extra-sandbox-paths = /private/var/cache/lexe" | sudo tee -a /etc/nix/ni
323322
$ sudo launchctl kickstart -k system/org.nixos.nix-daemon
324323
```
325324

326-
NOTE: you'll want to run `sudo rm -rf /var/cache/lexe/target` periodically, just
327-
like `cargo clean`.
328-
329325
Here we're creating a new `/var/cache/lexe` directory that's only accessible to
330326
members of the `nixbld` group. We also set some special file settings so that
331327
all new files and directories created in `/var/cache/lexe` are automatically

nix/lib/default.nix

+21
Original file line numberDiff line numberDiff line change
@@ -70,4 +70,25 @@
7070
ref = elemAt matches 0;
7171
rev = elemAt matches 1;
7272
};
73+
74+
# In a string `inputStr`, replace all matches of `regex` with `replacement`.
75+
#
76+
# ```
77+
# > inputStr = "--package=run-sgx --bin=run-sgx --locked --offline"
78+
# > regexReplaceAll "--bin( |=)[^ ]+" "" inputStr
79+
# "--package=run-sgx --locked --offline"
80+
# ```
81+
regexReplaceAll = regex: replacement: inputStr: let
82+
inherit (builtins) concatStringsSep isString map split;
83+
84+
# ex: inputStr = "foo bar baz", regex = "bar" => [ "foo " ["bar"] " baz" ]
85+
splits = split regex inputStr;
86+
87+
matchesReplaced = map (s:
88+
if isString s
89+
then s
90+
else replacement)
91+
splits;
92+
in
93+
concatStringsSep "" matchesReplaced;
7394
}

nix/pkgs/buildRustIncremental.nix

-143
This file was deleted.

0 commit comments

Comments
 (0)