avoid stdout redirection on `curl` executions
Avoid redirecting the curl output directly to the stdout. This alteration affects the integrity of the file during the retry process, as it also redirects the logs from the retries. Consequently, this leads to the bootstrap process failing because of an invalid checksum.
For more information, see the [zulip thread](https://rust-lang.zulipchat.com/#narrow/stream/326414-t-infra.2Fbootstrap/topic/checksum.20errors)
Fixes#115275
Avoid redirecting the curl output directly to the stdout. This alteration
affects the integrity of the file during the retry process, as it also redirects
the logs from the retries. Consequently, this leads to the bootstrap process failing
because of an invalid checksum.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
In addition: Incorporated some review feedback (namely, removed a useless
initial assignment to True that was never read), and unified code a bit more
between bootstrap.py and download.rs (by using the same variable name for the
same concept).
1. Turned patch-binaries-for-nix from a boolean into a ternary flag: true,
false, and unset.
2. When patch-binaries-for-nix is unset, we continue with the existing NixOS
detection heuristic (look for nixos in /etc/os-release, if present), but if
we are not atop NixOS, then issue a note if we see the IN_NIX_SHELL
environment variable telling the user to consider setting
patch-binaries-for-nix explicitly.
[Two users over on zulip](https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/Bootstrapping.20on.20NixOS) bumped into issues where NixOS wasn't being properly detected.
I believe this was caused by the presence of `/lib` on their machines. `/lib` is not standard on NixOS but can still be created by users or scripts.
We are already checking `/etc/os-release`. The presence of `ID=nixos` in it's output should be trustworthy and we shouldn't then go on to also check for `/lib`.
- Remove unneeded imports in 'fuscia-test-runner.py'
- Add explicit stacklevel to 'x.py'
- Fix mutable types as default args in `bootstrap.py` and `bootstrap_test.py`
This moves a lot of code around, but the logic itself is not too terribly complicated.
- Move almost all logic in `def bootstrap` to the `RustBuild` class, to avoid mixing setting configuration with running commands
- Update various doctests to the new (more complete) RustBuild config. In particular, don't pretend that `bin_root` supports `build` being unset.
- Change `parse_args` not to use a global, to allow testing it
- Set BUILD_DIR appropriately so bootstrap.py doesn't panic because cargo isn't found
Add `--warnings warn` flag to `x.py`
So that bootstrap itself can be built with warnings not being treated as errors.
Fixes: https://github.com/rust-lang/rust/issues/76805
r? ```@jyn514```
Respect CARGOFLAGS in bootstrap.py
This makes it possible to pass flags to bootstrap itself, for consistency with std/rustc: 04265621f9/src/bootstrap/builder.rs (L1446-L1447)
Like RUSTFLAGS, this doesn't support CARGOFLAGS_BOOTSTRAP: 6674dcda7a/src/bootstrap/bootstrap.py (L883-L884)
I found this useful recently when I wanted to pass `-Zsparse-registry` to an old checkout of the compiler from before it was stabilized in cargo.
Improve startup time of bootstrap
~~If the user has a `build/host` symlink set up, we can determine the target triple by reading it rather than invoking rustc. This significantly reduces startup time of bootstrap once any kind of build has been done~~
New approach explained below
```
➜ hyperfine -p 'git checkout -q master' -N './x.py -h' -r 50
Benchmark 1: ./x.py -h
Time (mean ± σ): 140.7 ms ± 2.6 ms [User: 99.9 ms, System: 39.3 ms]
Range (min … max): 136.8 ms … 149.6 ms 50 runs
➜ rust git:(master) hyperfine -p 'git checkout -q speedup-bootstrap-py' -N './x.py -h' -r 50
Benchmark 1: ./x.py -h
Time (mean ± σ): 95.2 ms ± 1.5 ms [User: 67.7 ms, System: 26.7 ms]
Range (min … max): 92.9 ms … 99.6 ms 50 runs
```
Also a small microoptimisation in using string splitting rather than regex when reading toml, which saves a few more milliseconds (2-5 testing locally), but less important.
Profiling shows the remaining runtime is around half setting up the Python runtime, and the vast majority of the remaining time is spent in subprocess building and running bootstrap itself, so probably can't be improved much further.
... using server-reported timestamp.
This allows us to track changes to the downloaded artifact more easily
and in a more reproducible manner.
Co-authored-by: Zixing Liu <zixing.liu@canonical.com>
Parallelize initial Rust download in bootstrap
Parallelize the initial download of Rust in `bootstrap.py`
`time ./x.py --help` after `rm -r build`
Before: 33s
After: 27s
This fixes the following recurring error on windows:
```
Traceback (most recent call last):
File "C:\Users\jyn\src\rust\x.py", line 29, in <module>
bootstrap.main()
File "C:\Users\jyn\src\rust\src\bootstrap\bootstrap.py", line 963, in main
bootstrap(args)
File "C:\Users\jyn\src\rust\src\bootstrap\bootstrap.py", line 927, in bootstrap
build.download_toolchain()
File "C:\Users\jyn\src\rust\src\bootstrap\bootstrap.py", line 437, in download_toolchain
shutil.rmtree(bin_root)
File "C:\Users\jyn\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 759, in rmtree
return _rmtree_unsafe(path, onerror)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\jyn\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 617, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "C:\Users\jyn\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 622, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "C:\Users\jyn\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 620, in _rmtree_unsafe
os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\jyn\\src\\rust\\build\\x86_64-pc-windows-msvc\\stage0\\bin\\rust-analyzer-proc-macro-srv.exe'
```
This also
* bumps cargo to the latest in rust-lang/cargo.
* adds 0BSD to allowed list of licenses
Co-authored-by: Scott Schafer <schaferjscott@gmail.com>
Co-authored-by: Eric Huss <eric@huss.org>
Initial support for loongarch64-unknown-linux-gnu
Hi, We hope to add a new port in rust for LoongArch.
LoongArch intro
LoongArch is a RISC style ISA which is independently designed by Loongson
Technology in China. It is divided into two versions, the 32-bit version (LA32)
and the 64-bit version (LA64). LA64 applications have application-level
backward binary compatibility with LA32 applications. LoongArch is composed of
a basic part (Loongson Base) and an expanded part. The expansion part includes
Loongson Binary Translation (LBT), Loongson VirtualiZation (LVZ), Loongson SIMD
EXtension (LSX) and Loongson Advanced SIMD EXtension(LASX).
Currently the LA464 processor core supports LoongArch ISA and the Loongson
3A5000 processor integrates 4 64-bit LA464 cores. LA464 is a four-issue 64-bit
high-performance processor core. It can be used as a single core for high-end
embedded and desktop applications, or as a basic processor core to form an
on-chip multi-core system for server and high-performance machine applications.
Documentations:
ISA:
https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html
ABI:
https://loongson.github.io/LoongArch-Documentation/LoongArch-ELF-ABI-EN.html
More docs can be found at:
https://loongson.github.io/LoongArch-Documentation/README-EN.html
Since last year, we have locally adapted two versions of rust, rust1.41 and rust1.57, and completed the test locally.
I'm not sure if I'm submitting all the patches at once, so I split up the patches and here's one of the commits
