Refactor Apple version handling in the compiler
Move various Apple version handling code in the compiler out `rustc_codegen_ssa` and into a place where it can be accessed by `rustc_attr_parsing`, which I found to be necessary when doing https://github.com/rust-lang/rust/pull/136867. Thought I'd split it out to make it easier to land, and to make further changes like https://github.com/rust-lang/rust/pull/131477 have fewer conflicts / PR dependencies.
There should be no functional changes in this PR.
`@rustbot` label O-apple
r? rust-lang/compiler
Autodiff batching
Enzyme supports batching, which is especially known from the ML side when training neural networks.
There we would normally have a training loop, where in each iteration we would pass in some data (e.g. an image), and a target vector. Based on how close we are with our prediction we compute our loss, and then use backpropagation to compute the gradients and update our weights.
That's quite inefficient, so what you normally do is passing in a batch of 8/16/.. images and targets, and compute the gradients for those all at once, allowing better optimizations.
Enzyme supports batching in two ways, the first one (which I implemented here) just accepts a Batch size,
and then each Dual/Duplicated argument has not one, but N shadow arguments. So instead of
```rs
for i in 0..100 {
df(x[i], y[i], 1234);
}
```
You can now do
```rs
for i in 0..100.step_by(4) {
df(x[i+0],x[i+1],x[i+2],x[i+3], y[i+0], y[i+1], y[i+2], y[i+3], 1234);
}
```
which will give the same results, but allows better compiler optimizations. See the testcase for details.
There is a second variant, where we can mark certain arguments and instead of having to pass in N shadow arguments, Enzyme assumes that the argument is N times longer. I.e. instead of accepting 4 slices with 12 floats each, we would accept one slice with 48 floats. I'll implement this over the next days.
I will also add more tests for both modes.
For any one preferring some more interactive explanation, here's a video of Tim's llvm dev talk, where he presents his work. https://www.youtube.com/watch?v=edvaLAL5RqU
I'll also add some other docs to the dev guide and user docs in another PR.
r? ghost
Tracking:
- https://github.com/rust-lang/rust/issues/124509
- https://github.com/rust-lang/rust/issues/135283
add `TypingMode::Borrowck`
Shares the first commit with #138499, doesn't really matter which PR to land first 😊😁
Introduces `TypingMode::Borrowck` which unlike `TypingMode::Analysis`, uses the hidden type computed by HIR typeck as the initial value of opaques instead of an unconstrained infer var. This is a part of https://github.com/rust-lang/types-team/issues/129.
Using this new `TypingMode` is unfortunately a breaking change for now, see tests/ui/impl-trait/non-defining-uses/as-projection-term.rs. Using an inference variable as the initial value results in non-defining uses in the defining scope. We therefore only enable it if with `-Znext-solver=globally` or `-Ztyping-mode-borrowck`
To do that the PR contains the following changes:
- `TypeckResults::concrete_opaque_type` are already mapped to the definition of the opaque type
- writeback now checks that the non-lifetime parameters of the opaque are universal
- for this, `fn check_opaque_type_parameter_valid` is moved from `rustc_borrowck` to `rustc_trait_selection`
- we add a new `query type_of_opaque_hir_typeck` which, using the same visitors as MIR typeck, attempts to merge the hidden types from HIR typeck from all defining scopes
- done by adding a `DefiningScopeKind` flag to toggle between using borrowck and HIR typeck
- the visitors stop checking that the MIR type matches the HIR type. This is trivial as the HIR type are now used as the initial hidden types of the opaque. This check is useful as a safeguard when not using `TypingMode::Borrowck`, but adding it to the new structure is annoying and it's not soundness critical, so I intend to not add it back.
- add a `TypingMode::Borrowck` which behaves just like `TypingMode::Analysis` except when normalizing opaque types
- it uses `type_of_opaque_hir_typeck(opaque)` as the initial value after replacing its regions with new inference vars
- it uses structural lookup in the new solver
fixes#112201, fixes#132335, fixes#137751
r? `@compiler-errors` `@oli-obk`
Initial support for auto traits with default bounds
This PR is part of ["MCP: Low level components for async drop"](https://github.com/rust-lang/compiler-team/issues/727)
Tracking issue: #138781
Summary: https://github.com/rust-lang/rust/pull/120706#issuecomment-1934006762
### Intro
Sometimes we want to use type system to express specific behavior and provide safety guarantees. This behavior can be specified by various "marker" traits. For example, we use `Send` and `Sync` to keep track of which types are thread safe. As the language develops, there are more problems that could be solved by adding new marker traits:
- to forbid types with an async destructor to be dropped in a synchronous context a trait like `SyncDrop` could be used [Async destructors, async genericity and completion futures](https://sabrinajewson.org/blog/async-drop).
- to support [scoped tasks](https://without.boats/blog/the-scoped-task-trilemma/) or in a more general sense to provide a [destruction guarantee](https://zetanumbers.github.io/book/myosotis.html) there is a desire among some users to see a `Leak` (or `Forget`) trait.
- Withoutboats in his [post](https://without.boats/blog/changing-the-rules-of-rust/) reflected on the use of `Move` trait instead of a `Pin`.
All the traits proposed above are supposed to be auto traits implemented for most types, and usually implemented automatically by compiler.
For backward compatibility these traits have to be added implicitly to all bound lists in old code (see below). Adding new default bounds involves many difficulties: many standard library interfaces may need to opt out of those default bounds, and therefore be infected with confusing `?Trait` syntax, migration to a new edition may contain backward compatibility holes, supporting new traits in the compiler can be quite difficult and so forth. Anyway, it's hard to evaluate the complexity until we try the system on a practice.
In this PR we introduce new optional lang items for traits that are added to all bound lists by default, similarly to existing `Sized`. The examples of such traits could be `Leak`, `Move`, `SyncDrop` or something else, it doesn't matter much right now (further I will call them `DefaultAutoTrait`'s). We want to land this change into rustc under an option, so it becomes available in bootstrap compiler. Then we'll be able to do standard library experiments with the aforementioned traits without adding hundreds of `#[cfg(not(bootstrap))]`s. Based on the experiments, we can come up with some scheme for the next edition, in which such bounds are added in a more targeted way, and not just everywhere.
Most of the implementation is basically a refactoring that replaces hardcoded uses of `Sized` with iterating over a list of traits including both `Sized` and the new traits when `-Zexperimental-default-bounds` is enabled (or just `Sized` as before, if the option is not enabled).
### Default bounds for old editions
All existing types, including generic parameters, are considered `Leak`/`Move`/`SyncDrop` and can be forgotten, moved or destroyed in generic contexts without specifying any bounds. New types that cannot be, for example, forgotten and do not implement `Leak` can be added at some point, and they should not be usable in such generic contexts in existing code.
To both maintain this property and keep backward compatibility with existing code, the new traits should be added as default bounds _everywhere_ in previous editions. Besides the implicit `Sized` bound contexts that includes supertrait lists and trait lists in trait objects (`dyn Trait1 + ... + TraitN`). Compiler should also generate implicit `DefaultAutoTrait` implementations for foreign types (`extern { type Foo; }`) because they are also currently usable in generic contexts without any bounds.
#### Supertraits
Adding the new traits as supertraits to all existing traits is potentially necessary, because, for example, using a `Self` param in a trait's associated item may be a breaking change otherwise:
```rust
trait Foo: Sized {
fn new() -> Option<Self>; // ERROR: `Option` requires `DefaultAutoTrait`, but `Self` is not `DefaultAutoTrait`
}
// desugared `Option`
enum Option<T: DefaultAutoTrait + Sized> {
Some(T),
None,
}
```
However, default supertraits can significantly affect compiler performance. For example, if we know that `T: Trait`, the compiler would deduce that `T: DefaultAutoTrait`. It also implies proving `F: DefaultAutoTrait` for each field `F` of type `T` until an explicit impl is be provided.
If the standard library is not modified, then even traits like `Copy` or `Send` would get these supertraits.
In this PR for optimization purposes instead of adding default supertraits, bounds are added to the associated items:
```rust
// Default bounds are generated in the following way:
trait Trait {
fn foo(&self) where Self: DefaultAutoTrait {}
}
// instead of this:
trait Trait: DefaultAutoTrait {
fn foo(&self) {}
}
```
It is not always possible to do this optimization because of backward compatibility:
```rust
pub trait Trait<Rhs = Self> {}
pub trait Trait1 : Trait {} // ERROR: `Rhs` requires `DefaultAutoTrait`, but `Self` is not `DefaultAutoTrait`
```
or
```rust
trait Trait {
type Type where Self: Sized;
}
trait Trait2<T> : Trait<Type = T> {} // ERROR: `???` requires `DefaultAutoTrait`, but `Self` is not `DefaultAutoTrait`
```
Therefore, `DefaultAutoTrait`'s are still being added to supertraits if the `Self` params or type bindings were found in the trait header.
#### Trait objects
Trait objects requires explicit `+ Trait` bound to implement corresponding trait which is not backward compatible:
```rust
fn use_trait_object(x: Box<dyn Trait>) {
foo(x) // ERROR: `foo` requires `DefaultAutoTrait`, but `dyn Trait` is not `DefaultAutoTrait`
}
// implicit T: DefaultAutoTrait here
fn foo<T>(_: T) {}
```
So, for a trait object `dyn Trait` we should add an implicit bound `dyn Trait + DefaultAutoTrait` to make it usable, and allow relaxing it with a question mark syntax `dyn Trait + ?DefaultAutoTrait` when it's not necessary.
#### Foreign types
If compiler doesn't generate auto trait implementations for a foreign type, then it's a breaking change if the default bounds are added everywhere else:
```rust
// implicit T: DefaultAutoTrait here
fn foo<T: ?Sized>(_: &T) {}
extern "C" {
type ExternTy;
}
fn forward_extern_ty(x: &ExternTy) {
foo(x); // ERROR: `foo` requires `DefaultAutoTrait`, but `ExternTy` is not `DefaultAutoTrait`
}
```
We'll have to enable implicit `DefaultAutoTrait` implementations for foreign types at least for previous editions:
```rust
// implicit T: DefaultAutoTrait here
fn foo<T: ?Sized>(_: &T) {}
extern "C" {
type ExternTy;
}
impl DefaultAutoTrait for ExternTy {} // implicit impl
fn forward_extern_ty(x: &ExternTy) {
foo(x); // OK
}
```
### Unresolved questions
New default bounds affect all existing Rust code complicating an already complex type system.
- Proving an auto trait predicate requires recursively traversing the type and proving the predicate for it's fields. This leads to a significant performance regression. Measurements for the stage 2 compiler build show up to 3x regression.
- We hope that fast path optimizations for well known traits could mitigate such regressions at least partially.
- New default bounds trigger some compiler bugs in both old and new trait solver.
- With new default bounds we encounter some trait solver cycle errors that break existing code.
- We hope that these cases are bugs that can be addressed in the new trait solver.
Also migration to a new edition could be quite ugly and enormous, but that's actually what we want to solve. For other issues there's a chance that they could be solved by a new solver.
Target modifiers fix for bool flags without value
Fixed support of boolean flags without values: `-Zbool-flag` is now consistent with `-Zbool-flag=true` in another crate.
When flag is explicitly set to default value, target modifier will not be set in crate metainfo (`-Zflag=false` when `false` is a default value for the flag).
Improved error notification when target modifier flag is absent in a crate ("-Zflag unset").
Example:
```
note: `-Zreg-struct-return=true` in this crate is incompatible with unset `-Zreg-struct-return` in dependency `default_reg_struct_return`
```
add FCW to warn about wasm ABI transition
See https://github.com/rust-lang/rust/issues/122532 for context: the "C" ABI on wasm32-unk-unk will change. The goal of this lint is to warn about any function definition and calls whose behavior will be affected by the change. My understanding is the following:
- scalar arguments are fine
- including 128 bit types, they get passed as two `i64` arguments in both ABIs
- `repr(C)` structs (recursively) wrapping a single scalar argument are fine (unless they have extra padding due to over-alignment attributes)
- all return values are fine
`@bjorn3` `@alexcrichton` `@Manishearth` is that correct?
I am making this a "show up in future compat reports" lint to maximize the chances people become aware of this. OTOH this likely means warnings for most users of Diplomat so maybe we shouldn't do this?
IIUC, wasm-bindgen should be unaffected by this lint as they only pass scalar types as arguments.
Tracking issue: https://github.com/rust-lang/rust/issues/138762
Transition plan blog post: https://github.com/rust-lang/blog.rust-lang.org/pull/1531
try-job: dist-various-2
Cache current_dll_path output
Computing the current dll path is somewhat expensive relative to other work when compiling `fn main() {}` as `dladdr` needs to iterate over the symbol table of librustc_driver.so until it finds a match.
Computing the current dll path is somewhat expensive relative to other
work when compiling `fn main() {}` as `dladdr` needs to iterate over the
symbol table of librustc_driver.so until it finds a match.
Represent diagnostic side effects as dep nodes
This changes diagnostic to be tracked as a special dep node (`SideEffect`) instead of having a list of side effects associated with each dep node. `SideEffect` is always red and when forced, it emits the diagnostic and marks itself green. Each emitted diagnostic generates a new `SideEffect` with an unique dep node index.
Some implications of this:
- Diagnostic may now be emitted more than once as they can be emitted once when the `SideEffect` gets marked green and again if the task it depends on needs to be re-executed due to another node being red. It relies on deduplicating of diagnostics to avoid that.
- Anon tasks which emits diagnostics will no longer *incorrectly* be merged with other anon tasks.
- Reusing a CGU will now emit diagnostics from the task generating it.
To correspond to their actual print request names, `target-spec-json`
and `all-target-specs-json`, and for consistency with other print name
<-> print kind mappings.
Revert <https://github.com/rust-lang/rust/pull/138084> to buy time to
consider options that avoids breaking downstream usages of cargo on
distributed `rustc-src` artifacts, where such cargo invocations fail due
to inability to inherit `lints` from workspace root manifest's
`workspace.lints` (this is only valid for the source rust-lang/rust
workspace, but not really the distributed `rustc-src` artifacts).
This breakage was reported in
<https://github.com/rust-lang/rust/issues/138304>.
This reverts commit 48caf81484, reversing
changes made to c6662879b2.
compiler: Use `size_of` from the prelude instead of imported
Use `std::mem::{size_of, size_of_val, align_of, align_of_val}` from the prelude instead of importing or qualifying them. Apply this change across the compiler.
These functions were added to all preludes in Rust 1.80.
r? ``@compiler-errors``
By naming them in `[workspace.lints.rust]` in the top-level
`Cargo.toml`, and then making all `compiler/` crates inherit them with
`[lints] workspace = true`. (I omitted `rustc_codegen_{cranelift,gcc}`,
because they're a bit different.)
The advantages of this over the current approach:
- It uses a standard Cargo feature, rather than special handling in
bootstrap. So, easier to understand, and less likely to get
accidentally broken in the future.
- It works for proc macro crates.
It's a shame it doesn't work for rustc-specific lints, as the comments
explain.
Use `std::mem::{size_of, size_of_val, align_of, align_of_val}` from the
prelude instead of importing or qualifying them.
These functions were added to all preludes in Rust 1.80.
`-Zteach` is perma-unstable, barely used, the highlighting logic buggy and the flag being passed around is tech-debt. We should likely remove `-Zteach` in its entirely.
