Remove `prev_index_to_index` field from `CurrentDepGraph`
The dep graph currently has 2 ways to map a previous index into a current index. The `prev_index_to_index` map stores the current index equivalent of a previous index. For indices which are marked green, we also store the same information in the `DepNodeColorMap`. We actually only need to known the mapping for green nodes however, so this PR removes `prev_index_to_index` and instead makes use of the `DepNodeColorMap`.
To avoid racing when promoting a node from the previous session, the encoder lock is now used to ensure only one thread encodes the promoted node. This was previously done by the lock in `prev_index_to_index`.
This also changes `nodes_newly_allocated_in_current_session` used to detect duplicate dep nodes to contain both new and previous nodes, which is simpler and can better catch duplicates.
The dep node index encoding used in `DepNodeColorMap` is tweak to avoid subtraction / addition to optimize accessing the current equivalent of a previous index.
r? `@oli-obk`
Batch mark waiters as unblocked when resuming in the deadlock handler
This fixes a race when resuming multiple threads to resolve query cycles. This now marks all threads as unblocked before resuming any of them. Previously if one was resumed and marked as unblocked at a time. The first thread resumed could fall asleep then Rayon would detect a second false deadlock. Later the initial deadlock handler thread would resume further threads.
This also reverts the workaround added in https://github.com/rust-lang/rust/pull/137731.
cc `@SparrowLii` `@lqd`
Abort in deadlock handler if we fail to get a query map
Resolving query cycles requires the complete active query map, or it may miss query cycles. We did not check that the map is completely constructed before. If there is some error collecting the map, something has gone wrong already. This adds a check to abort/panic if we fail to construct the complete map.
This can help differentiate errors from the `deadlock detected` case if constructing query map has errors in practice.
An `Option` is not used for `collect_active_jobs` as the panic handler can still make use of a partial map.
It's very useful. There are some false positives involving integration
tests in `rustc_pattern_analysis` and `rustc_serialize`. There is also a
false positive involving `rustc_driver_impl`'s
`rustc_randomized_layouts` feature. And I removed a `rustc_span` mention
in a doc comment in `rustc_log` because it wasn't integral to the
comment but caused a dev-dependency.
The `hash_raw_entry` feature has finished fcp-close, so the compiler
should stop using it to allow its removal. Several `Sharded` maps were
using raw entries to avoid re-hashing between shard and map lookup, and
we can do that with `hashbrown::HashTable` instead.
Revert <https://github.com/rust-lang/rust/pull/138084> to buy time to
consider options that avoids breaking downstream usages of cargo on
distributed `rustc-src` artifacts, where such cargo invocations fail due
to inability to inherit `lints` from workspace root manifest's
`workspace.lints` (this is only valid for the source rust-lang/rust
workspace, but not really the distributed `rustc-src` artifacts).
This breakage was reported in
<https://github.com/rust-lang/rust/issues/138304>.
This reverts commit 48caf81484, reversing
changes made to c6662879b2.
Rollup of 12 pull requests
Successful merges:
- #136127 (Allow `*const W<dyn A> -> *const dyn A` ptr cast)
- #136968 (Turn order dependent trait objects future incompat warning into a hard error)
- #137319 (Stabilize `const_vec_string_slice`)
- #137885 (tidy: add triagebot checks)
- #138040 (compiler: Use `size_of` from the prelude instead of imported)
- #138084 (Use workspace lints for crates in `compiler/`)
- #138158 (Move more layouting logic to `rustc_abi`)
- #138160 (depend more on attr_data_structures and move find_attr! there)
- #138192 (crashes: couple more tests)
- #138216 (bootstrap: Fix stack printing when a step cycle is detected)
- #138232 (Reduce verbosity of GCC build log)
- #138242 (Revert "Don't test new error messages with the stage 0 compiler")
r? `@ghost`
`@rustbot` modify labels: rollup
compiler: Use `size_of` from the prelude instead of imported
Use `std::mem::{size_of, size_of_val, align_of, align_of_val}` from the prelude instead of importing or qualifying them. Apply this change across the compiler.
These functions were added to all preludes in Rust 1.80.
r? ``@compiler-errors``
Change TaskDeps to start preallocated with 128 capacity
This is a tiny change that makes `TaskDeps::read_set` start preallocated with capacity for 128 elements.
From local profiling, it looks like `TaskDeps::read_set` is one of the most-often resized hash-sets in `rustc`.
By naming them in `[workspace.lints.rust]` in the top-level
`Cargo.toml`, and then making all `compiler/` crates inherit them with
`[lints] workspace = true`. (I omitted `rustc_codegen_{cranelift,gcc}`,
because they're a bit different.)
The advantages of this over the current approach:
- It uses a standard Cargo feature, rather than special handling in
bootstrap. So, easier to understand, and less likely to get
accidentally broken in the future.
- It works for proc macro crates.
It's a shame it doesn't work for rustc-specific lints, as the comments
explain.
Use `std::mem::{size_of, size_of_val, align_of, align_of_val}` from the
prelude instead of importing or qualifying them.
These functions were added to all preludes in Rust 1.80.
Resume one waiter at once in deadlock handler
When multiple query loop errors occur in the code, only one waiter should be resumed at a time to avoid waking up multiple waiters at the same time and causing deadlock due to thread grabbing.
This fixes the UI failures in #132051
cc `@Zoxc` `@cjgillot` `@nnethercote` `@bjorn3` `@Kobzol`
Zulip discussion [here](https://rust-lang.zulipchat.com/#narrow/channel/187679-t-compiler.2Fwg-parallel-rustc/topic/Deadlocks.20and.20Rayon)
Edit: We can't reproduce these bugs with the existing test suits, so we keep them until we merge #132051
UPDATES #129912
UPDATES #120757
UPDATES #129911
Parallel-compiler-related cleanup
Parallel-compiler-related cleanup
I carefully split changes into commits. Commit messages are self-explanatory. Squashing is not recommended.
cc "Parallel Rustc Front-end" https://github.com/rust-lang/rust/issues/113349
r? SparrowLii
``@rustbot`` label: +WG-compiler-parallel
tree-wide: parallel: Fully removed all `Lrc`, replaced with `Arc`
tree-wide: parallel: Fully removed all `Lrc`, replaced with `Arc`
This is continuation of https://github.com/rust-lang/rust/pull/132282 .
I'm pretty sure I did everything right. In particular, I searched all occurrences of `Lrc` in submodules and made sure that they don't need replacement.
There are other possibilities, through.
We can define `enum Lrc<T> { Rc(Rc<T>), Arc(Arc<T>) }`. Or we can make `Lrc` a union and on every clone we can read from special thread-local variable. Or we can add a generic parameter to `Lrc` and, yes, this parameter will be everywhere across all codebase.
So, if you think we should take some alternative approach, then don't merge this PR. But if it is decided to stick with `Arc`, then, please, merge.
cc "Parallel Rustc Front-end" ( https://github.com/rust-lang/rust/issues/113349 )
r? SparrowLii
`@rustbot` label WG-compiler-parallel
