![]() Avoid no-op unlink+link dances in incr comp Incremental compilation scales quite poorly with the number of CGUs. This PR improves one reason for that. The incr comp process hard-links all the files from an old session into a new one, then it runs the backend, which may just hard-link the new session files into the output directory. Then codegen hard-links all the output files back to the new session directory. This PR (perhaps unimaginatively) fixes the silliness that ensues in the last step. The old `link_or_copy` implementation would be passed pairs of paths which are already the same inode, then it would blindly delete the destination and re-create the hard-link that it just deleted. This PR lets us skip both those operations. We don't skip the other two hard-links. `cargo +stage1 b && touch crates/core/main.rs && strace -cfw -elink,linkat,unlink,unlinkat cargo +stage1 b` before and then after on `ripgrep-13.0.0`: ``` % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 52.56 0.024950 25 978 485 unlink 34.38 0.016318 22 727 linkat 13.06 0.006200 24 249 unlinkat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.047467 24 1954 485 total ``` ``` % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 42.83 0.014521 57 252 unlink 38.41 0.013021 26 486 linkat 18.77 0.006362 25 249 unlinkat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.033904 34 987 total ``` This reduces the number of hard-links that are causing perf troubles, noted in https://github.com/rust-lang/rust/issues/64291 and https://github.com/rust-lang/rust/issues/137560 |
||
---|---|---|
.. | ||
.github | ||
.vscode | ||
.zed | ||
build_system | ||
docs | ||
example | ||
patches | ||
scripts | ||
src | ||
.cirrus.yml | ||
.gitattributes | ||
.gitignore | ||
Cargo.lock | ||
Cargo.toml | ||
clean_all.sh | ||
config.txt | ||
LICENSE-APACHE | ||
LICENSE-MIT | ||
Readme.md | ||
rust-toolchain | ||
rustfmt.toml | ||
test.sh | ||
y.cmd | ||
y.ps1 | ||
y.sh |
Cranelift codegen backend for rust
The goal of this project is to create an alternative codegen backend for the rust compiler based on Cranelift. This has the potential to improve compilation times in debug mode. If your project doesn't use any of the things listed under "Not yet supported", it should work fine. If not please open an issue.
Download using Rustup
The Cranelift codegen backend is distributed in nightly builds on Linux and x86_64 macOS. If you want to install it using Rustup, you can do that by running:
$ rustup component add rustc-codegen-cranelift-preview --toolchain nightly
Once it is installed, you can enable it with one of the following approaches:
CARGO_PROFILE_DEV_CODEGEN_BACKEND=cranelift cargo +nightly build -Zcodegen-backend
- Add the following to
.cargo/config.toml
:[unstable] codegen-backend = true [profile.dev] codegen-backend = "cranelift"
- Add the following to
Cargo.toml
:# This line needs to come before anything else in Cargo.toml cargo-features = ["codegen-backend"] [profile.dev] codegen-backend = "cranelift"
Precompiled builds
You can also download a pre-built version from the releases page.
Extract the dist
directory in the archive anywhere you want.
If you want to use cargo clif build
instead of having to specify the full path to the cargo-clif
executable, you can add the bin
subdirectory of the extracted dist
directory to your PATH
.
(tutorial for Windows, and for Linux/MacOS).
Building and testing
If you want to build the backend manually, you can download it from GitHub and build it yourself:
$ git clone https://github.com/rust-lang/rustc_codegen_cranelift
$ cd rustc_codegen_cranelift
$ ./y.sh prepare
$ ./y.sh build
To run the test suite replace the last command with:
$ ./test.sh
For more docs on how to build and test see build_system/usage.txt or the help message of ./y.sh
.
Platform support
OS \ architecture | x86_64 | AArch64 | Riscv64 | s390x (System-Z) |
---|---|---|---|---|
Linux | ✅ | ✅ | ✅1 | ✅1 |
FreeBSD | ✅1 | ❓ | ❓ | ❓ |
AIX | ❌2 | N/A | N/A | ❌2 |
Other unixes | ❓ | ❓ | ❓ | ❓ |
macOS | ✅ | ✅ | N/A | N/A |
Windows | ✅ | ❌ | N/A | N/A |
✅: Fully supported and tested ❓: Maybe supported, not tested ❌: Not supported at all
Not all targets are available as rustup component for nightly. See notes in the platform support matrix.
Usage
rustc_codegen_cranelift can be used as a near-drop-in replacement for cargo build
or cargo run
for existing projects.
Assuming $cg_clif_dir
is the directory you cloned this repo into and you followed the instructions (y.sh prepare
and y.sh build
or test.sh
).
In the directory with your project (where you can do the usual cargo build
), run:
$ $cg_clif_dir/dist/cargo-clif build
This will build your project with rustc_codegen_cranelift instead of the usual LLVM backend.
For additional ways to use rustc_codegen_cranelift like the JIT mode see usage.md.
Building and testing with changes in rustc code
See rustc_testing.md.
Not yet supported
- SIMD (tracked here,
std::simd
fully works,std::arch
is partially supported) - Unwinding on panics (no cranelift support,
-Cpanic=abort
is enabled by default)
License
Licensed under either of
- Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.
Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you shall be dual licensed as above, without any additional terms or conditions.