Skip to content

Commit da74a77

Browse files
authored
Rollup merge of rust-lang#82047 - the8472:fast-rename, r=davidtwco
bypass auto_da_alloc for metadata files This saves about 0.7% when rerunning the UI test suite. I.e. when the metadata files exist and will be overwritten. No improvements expected for a clean build. So it might show up in incr-patched perf results. ``` regular rename: Benchmark #1: touch src/tools/compiletest/src/main.rs ; RUSTC_WRAPPER="" schedtool -B -e ./x.py test src/test/ui Time (mean ± σ): 47.305 s ± 0.170 s [User: 1631.540 s, System: 412.648 s] Range (min … max): 47.125 s … 47.856 s 20 runs non-durable rename: Benchmark #1: touch src/tools/compiletest/src/main.rs ; RUSTC_WRAPPER="" schedtool -B -e ./x.py test src/test/ui Time (mean ± σ): 46.930 s ± 0.064 s [User: 1634.344 s, System: 396.038 s] Range (min … max): 46.759 s … 47.043 s 20 runs ``` There are more places that trigger auto_da_alloc behavior by overwriting existing files with O_TRUNC, but those are much harder to locate because `O_TRUNC` is set on `open()` but the writeback is triggered on `close()`. The latter is the part which shows up in profiles.
2 parents 27885a9 + 6a679ff commit da74a77

File tree

2 files changed

+19
-1
lines changed

2 files changed

+19
-1
lines changed

compiler/rustc_interface/src/passes.rs

+1-1
Original file line numberDiff line numberDiff line change
@@ -988,7 +988,7 @@ fn encode_and_write_metadata(
988988
.unwrap_or_else(|err| tcx.sess.fatal(&format!("couldn't create a temp dir: {}", err)));
989989
let metadata_tmpdir = MaybeTempDir::new(metadata_tmpdir, tcx.sess.opts.cg.save_temps);
990990
let metadata_filename = emit_metadata(tcx.sess, &metadata, &metadata_tmpdir);
991-
if let Err(e) = fs::rename(&metadata_filename, &out_filename) {
991+
if let Err(e) = util::non_durable_rename(&metadata_filename, &out_filename) {
992992
tcx.sess.fatal(&format!("failed to write {}: {}", out_filename.display(), e));
993993
}
994994
if tcx.sess.opts.json_artifact_notifications {

compiler/rustc_interface/src/util.rs

+18
Original file line numberDiff line numberDiff line change
@@ -694,6 +694,24 @@ pub fn build_output_filenames(
694694
}
695695
}
696696

697+
#[cfg(not(target_os = "linux"))]
698+
pub fn non_durable_rename(src: &Path, dst: &Path) -> std::io::Result<()> {
699+
std::fs::rename(src, dst)
700+
}
701+
702+
/// This function attempts to bypass the auto_da_alloc heuristic implemented by some filesystems
703+
/// such as btrfs and ext4. When renaming over a file that already exists then they will "helpfully"
704+
/// write back the source file before committing the rename in case a developer forgot some of
705+
/// the fsyncs in the open/write/fsync(file)/rename/fsync(dir) dance for atomic file updates.
706+
///
707+
/// To avoid triggering this heuristic we delete the destination first, if it exists.
708+
/// The cost of an extra syscall is much lower than getting descheduled for the sync IO.
709+
#[cfg(target_os = "linux")]
710+
pub fn non_durable_rename(src: &Path, dst: &Path) -> std::io::Result<()> {
711+
let _ = std::fs::remove_file(dst);
712+
std::fs::rename(src, dst)
713+
}
714+
697715
// Note: Also used by librustdoc, see PR #43348. Consider moving this struct elsewhere.
698716
//
699717
// FIXME: Currently the `everybody_loops` transformation is not applied to:

0 commit comments

Comments
 (0)