run-make: drop `os_pipe` workaround now that `anonymous_pipe` is stable on beta
Follow-up to #137537 where I had to include a temporary dep on `os_pipe` before `anonymous_pipe` was stabilized. Now that `anonymous_pipe` is stable on beta, we can get rid of this workaround.
Closes#137532. (Final cleanup item)
r? `@Kobzol`
update ```miniz_oxide``` to 0.8.8
I would normally let the auto actions handle this but it turns out 0.8.7 can trigger a panic when debug assertions are enabled in a few cases so I feel it's important it gets sorted more quickly. (and I would ideally like to yank that version but was worried that could cause some issues had been pulled in as a dependency by this repo already before I discovered the problem)
As it can only happen when debug assertions are enabled (the overflow results in the intended result so it doesn't cause any issue in release mode) and using the wrapping buffer mode when decompressing it is very unlikely to cause any issues here but I would like to get it sorted just to be safe. ```miniz_oxide``` is used in the standard library (and some tools) via ```backtrace-rs ``` which doesn't use a wrapping buffer, and thus won't trigger this condition. There does however seem like there are some tools that do dependency on ```flate2``` which does use ```miniz_oxide``` decompression using a a wrapping buffer and could in theory trigger it if they are run when compiled with debug assertions enabled.
It's kinda unclear what version what tool uses though as several of them specify older versions of flate2 which depended on ```miniz_oxide``` 0.7.x in cargo.toml, and ```miniz_oxide```, and not all have a cargo.lock and due to an older version of ```backtrace``` being in the root Cargo.lock which still depended on ```miniz_oxide``` 0.7.4, so that version is also pulled in alongside the newer version.
Update windows-bindgen to 0.61.0
This updates the automatically generate Windows API bindings. Not much changed this time:
- There's now `Default` implementations for many types, which is convenient. It does however conflict with one place where we implemented a non-zeroed default (to set the length field). But that's no big problem.
- The `--no-core` flag has been renamed to `--no-deps` to more accurately reflect its meaning (i.e. generate all necessary code without requiring additional dependencies).
- The `--link` flag allows us to set the location of the `link!` macro. Currently we use our workspace's `windows_targets` crate but we could move it into library/std using `--link`. However, this would need to be co-ordinated with the `backtrace` crate (which is a separate crate but included in std using `#[path]`). So I've left that for another time.
Switch `time` to `jiff` for time formatting in ICE dumps
Due to https://github.com/jhpratt/deranged/issues/21, Clippy, R-A and Miri currently fail to build if we bump to 0.4.1, pulled in via `time`. ~~Add some specific type annotations so we don't have to just pin it.~~
~~I can open 3 PRs to the tool repos if preferred, but I thought it might be easier to do this than to pin the transitive dep and go back and remove it once the changes are synced back.~~
Avoid a reverse map that is only used in diagnostics paths
r? `@petrochenkov`
iterating a map until a value matches and returning the key is bad obviously, but it happens very rarely and only on diagnostics paths. It would also be a lot cheaper with https://github.com/rust-lang/rust/pull/138995. Which is actually why I'm trying this out, that PR adds a new entry in `create_def`, which makes `create_def` show up in cachegrind. So I'm trying out if removing adding an entry in `create_def` is a perf improvement
Prepend temp files with per-invocation random string to avoid temp filename conflicts
https://github.com/rust-lang/rust/issues/139407 uncovered a very subtle unsoundness with incremental codegen, failing compilation sessions (due to assembler errors), and the "prefer hard linking over copying files" strategy we use in the compiler for file management.
Specifically, imagine we're building a single file 3 times, all with `-Csave-temps -Cincremental=...`. Let's call the object file we're building for the codegen unit for `main` "`XXX.o`" just for clarity since it's probably some gigantic hash name:
```
#[inline(never)]
#[cfg(any(rpass1, rpass3))]
fn a() -> i32 {
0
}
#[cfg(any(cfail2))]
fn a() -> i32 {
1
}
fn main() {
evil::evil();
assert_eq!(a(), 0);
}
mod evil {
#[cfg(any(rpass1, rpass3))]
pub fn evil() {
unsafe {
std::arch::asm!("/* */");
}
}
#[cfg(any(cfail2))]
pub fn evil() {
unsafe {
std::arch::asm!("missing");
}
}
}
```
Session 1 (`rpass1`):
* Type-check, borrow-check, etc.
* Serialize the dep graph to the incremental working directory `.../s-...-working/`.
* Codegen object file to a temp file `XXX.rcgu.o` which is spit out in the cwd.
* Hard-link[^1] `XXX.rcgu.o` to the incremental working directory `.../s-...-working/XXX.o`.
* Save-temps option means we don't delete `XXX.rgcu.o`.
* Link the binary and stuff.
* Finalize[^2] the working incremental session by renaming `.../s-...-working` to ` s-...-asjkdhsjakd` (some other finalized incr comp session dir name).
Session 2 (`cfail2`):
* Load artifacts from the previous *finalized* incremental session, namely the dep graph.
* Type-check, borrow-check, etc. since the file has changed, so most dep graph nodes are red.
* Serialize the dep graph to the incremental working directory `.../s-...-working/`.
* Codegen object file to a temp file `XXX.rcgu.o`. **HERE IS THE PROBLEM**: The hard-link is still set up to point to the inode from `XXX.o` from the first session, so this also modifies the `XXX.o` in the previous finalized session directory.
* Codegen emits an error b/c `missing` is not an instruction, so we abort before finalizing the incremental session. Specifically, this means that the *previous* session is the last finalized session.
Session 3 (`rpass3`):
* Load artifacts from the previous *finalized* incremental session, namely the dep graph. NOTE that this is from session 1.
* All the dep graph nodes are green since we are basically replaying session 1.
* codegen object file `XXX.o`, which is detected as *reused* from session 1 since dep nodes were green. That means we **reuse** `XXX.o` which had been dirtied from session 2.
* Link the binary and stuff.
This results in a binary which reuses some of the build artifacts from session 2, but thinks it's from session 1.
At this point, I hope it's clear to see that the incremental results from session 1 were dirtied from session 2, but we reuse them as if session 1 was the previous (finalized) incremental session we ran. This is at best really buggy, and at worst **unsound**.
This isn't limited to `-C save-temps`, since there are other combinations of flags that may keep around temporary files (hard linked) in the working directory (like `-C debuginfo=1 -C split-debuginfo=unpacked` on darwin, for example).
---
This PR implements a fix which is to prepend temp filenames with a random string that is generated per invocation of rustc. This string is not *deterministic*, but temporary files are transient anyways, so I don't believe this is a problem.
That means that temp files are now something like... `{crate-name}.{cgu}.{invocation_temp}.rcgu.o`, where `{invocation_temp}` is the new temporary string we generate per invocation of rustc.
Fixes https://github.com/rust-lang/rust/issues/139407
[^1]: 175dcc7773/compiler/rustc_fs_util/src/lib.rs (L60)
[^2]: 175dcc7773/compiler/rustc_incremental/src/persist/fs.rs (L1-L40)
compiletest maintenance: sort deps and drop dep on `anyhow`
Two changes:
1. Sort compiletest deps alphabetically because it was annoying me (harder to quickly glance what deps compiletest is using).
2. Drop dependency on `anyhow`. There's only one usage of `anyhow`, which is for `with_context` on sth that would immediately panic anyway.
Currently `compiletest` panics all over the place but doesn't really use
`anyhow` anyway. I'd like to introduce some more principled error
handling and disciplined diagnostic reporting in the near future.
Make the UnifyKey and UnifyValue imports non-nightly
Explicitly depend on ena in rustc_type_ir and import types from there.
This is required for rust-analyzer to use the new solver.
r? types
Add `*_value` methods to proc_macro lib
This is the (re-)implementation of https://github.com/rust-lang/libs-team/issues/459.
It allows to get the actual value (unescaped) of the different string literals.
It was originally done in https://github.com/rust-lang/rust/pull/136355 but it broke the artifacts build so we decided to move the crate to crates.io to go around this limitation.
Part of https://github.com/rust-lang/rust/issues/136652.
Considering this is a copy-paste of the originally approved PR, no need to go through the whole process again. \o/
r? `@Urgau`
rustc_resolve: fix instability in lib.rmeta contents
rust-lang/rust@23032f31c9 accidentally introduced some nondeterminism in the ordering of lib.rmeta files, which we caught in our bazel-based builds only recently due to being further behind than normal. In my testing, this fixes the issue.
Optimize hash map operations in the query system
This optimizes hash map operations in the query system by explicitly passing hashes and using more optimal operations. `find_or_find_insert_slot` in particular saves a hash table lookup over `entry`. It's not yet available in a safe API, but will be in https://github.com/rust-lang/hashbrown/pull/466.
<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check</td><td align="right">1.6189s</td><td align="right">1.6129s</td><td align="right"> -0.37%</td></tr><tr><td>🟣 <b>hyper</b>:check</td><td align="right">0.2353s</td><td align="right">0.2337s</td><td align="right"> -0.67%</td></tr><tr><td>🟣 <b>regex</b>:check</td><td align="right">0.9344s</td><td align="right">0.9289s</td><td align="right"> -0.59%</td></tr><tr><td>🟣 <b>syn</b>:check</td><td align="right">1.4693s</td><td align="right">1.4652s</td><td align="right"> -0.28%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check</td><td align="right">5.6606s</td><td align="right">5.6439s</td><td align="right"> -0.30%</td></tr><tr><td>Total</td><td align="right">9.9185s</td><td align="right">9.8846s</td><td align="right"> -0.34%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9956s</td><td align="right"> -0.44%</td></tr></table>
r? `@cjgillot`
jsondocck: Replace `jsonpath_lib` with `jsonpath-rust`
The current jsonpath implementation we use isn't spec-compliant, and is buggy. See https://github.com/freestrings/jsonpath/issues/91
To solve it, it's replaced with https://github.com/besok/jsonpath-rust. This is spec-compiant, and doesn't have a really awkward bug we need to always dance around.
Unfortunately, this requires rewriting almost every test, as the behaviour of `[?(```@`,``` which is *extremely* common was changed. (But the new behaviour makes way more sense, and isn't buggy with tripply nested selectors)
Unblocks #110406. Makes #100515 much easier as we don't need to explain the broken JSONPath implementation
Best reviewed commit-by-commit. The first does the replacement. The next two rewrite the test-suite mechanically. The last rewrites the test-suite by hand.
r? ```@GuillaumeGomez```
