Pointers for variables all need to be in the same address space for
correct compilation. Therefore ensure that even if a global variable is
created in a different address space, it is casted to the default
address space before its value is used.
This is necessary for the amdgpu target and others where the default
address space for global variables is not 0.
For example `core` does not compile in debug mode when not casting the
address space to the default one because it tries to emit the following
(simplified) LLVM IR, containing a type mismatch:
```llvm
@alloc_0 = addrspace(1) constant <{ [6 x i8] }> <{ [6 x i8] c"bit.rs" }>, align 1
@alloc_1 = addrspace(1) constant <{ ptr }> <{ ptr addrspace(1) @alloc_0 }>, align 8
; ^ here a struct containing a `ptr` is needed, but it is created using a `ptr addrspace(1)`
```
For this to compile, we need to insert a constant `addrspacecast` before
we use a global variable:
```llvm
@alloc_0 = addrspace(1) constant <{ [6 x i8] }> <{ [6 x i8] c"bit.rs" }>, align 1
@alloc_1 = addrspace(1) constant <{ ptr }> <{ ptr addrspacecast (ptr addrspace(1) @alloc_0 to ptr) }>, align 8
```
As vtables are global variables as well, they are also created with an
`addrspacecast`. In the SSA backend, after a vtable global is created,
metadata is added to it. To add metadata, we need the non-casted global
variable. Therefore we strip away an addrspacecast if there is one, to
get the underlying global.
Autodiff Upstreaming - rustc_codegen_llvm changes
Now that the autodiff/Enzyme backend is merged, this is an upstream PR for the `rustc_codegen_llvm` changes.
It also includes small changes to three files under `compiler/rustc_ast`, which overlap with my frontend PR (https://github.com/rust-lang/rust/pull/129458).
Here I only include minimal definitions of structs and enums to be able to build this backend code.
The same goes for minimal changes to `compiler/rustc_codegen_ssa`, the majority of changes there will be in another PR, once either this or the frontend gets merged.
We currently have 68 files left to merge, 19 in the frontend PR, 21 (+3 from the frontend) in this PR, and then ~30 in the middle-end.
This PR is large because it includes two of my three large files (~800 loc each). I could also first only upstream enzyme_ffi.rs, but I think people might want to see some use of these bindings in the same PR?
To already highlight the things which reviewers might want to discuss:
1) `enzyme_ffi.rs`: I do have a fallback module to make sure that we don't link rustc against Enzyme when we build rustc without autodiff support.
2) `add_panic_msg_to_global` was a pain to write and I currently can't even use it. Enzyme writes gradients into shadow memory. Pass in one float scalar? We'll allocate and return an extra float telling you how this float affected the output. Pass in a slice of floats? We'll let you allocate the vector and pass in a mutable reference to a float slice, we'll then write the gradient into that slice. It should be at least as large as your original slice, so we check that and panic if not. Currently we panic silently, but I already generate a nicer panic message with this function. I just don't know how to print it to the user. yet. I discussed this with a few rustc devs and the best we could come up with (for now), was to look for mangled panic calls in the IR and pick one, which works surprisingly reliably. If someone knows a good way to clean this up and print the panic message I'm all in, otherwise I can remove the code that writes the nicer panic message and keep the silent panic, since it's enough for soundness. Especially since this PR is already a bit larger.
3) `SanitizeHWAddress`: When differentiating C++, Enzyme can use TBAA to "understand" enums/unions, but for Rust we don't have this information. LLVM might to speculative loads which (without TBAA) confuse Enzyme, so we disable those with this attribute. This attribute is only set during the first opt run before Enzyme differentiates code. We then remove it again once we are done with autodiff and run the opt pipeline a second time. Since enums are everywhere in Rust, support for them is crucial, but if this looks too cursed I can remove these ~100 lines and keep them in my fork for now, we can then discuss them separately to make this PR simpler?
4) Duplicated llvm-opt runs: Differentiating already optimized code (and being able to do additional optimizations on the fly, e.g. for GPU code) is _the_ reason why Enzyme is so fast, so the compile time is acceptable for autodiff users: https://enzyme.mit.edu/talks/Publications/ (There are also algorithmic issues in Enzyme core which are more serious than running opt twice).
5) I assume that if we merge these minimal cg_ssa changes here already, I also need to fix the other backends (GCC and cliff) to have dummy implementations, correct?
6) *I'm happy to split this PR up further if reviewers have recommendations on how to.*
For the full implementation, see: https://github.com/rust-lang/rust/pull/129175
Tracking:
- https://github.com/rust-lang/rust/issues/124509
Provide structured suggestion for `impl Default` of type where all fields have defaults
```
error: `Default` impl doesn't use the declared default field values
--> $DIR/manual-default-impl-could-be-derived.rs:28:1
|
LL | / impl Default for B {
LL | | fn default() -> Self {
LL | | B {
LL | | x: s(),
| | --- this field has a default value
LL | | y: 0,
| | - this field has a default value
... |
LL | | }
| |_^
|
help: to avoid divergence in behavior between `Struct { .. }` and `<Struct as Default>::default()`, derive the `Default`
|
LL ~ #[derive(Default)] struct B {
|
```
Note that above the structured suggestion also includes completely removing the manual `impl`, but the rendering doesn't.
Some small nits to the borrowck suggestions for mutating a map through index
1. Suggesting users to either use `.insert` or `.get_mut` (which do totally different things) can be a bit of a footgun, so let's make that a bit more nuanced.
2. I find the suggestion of `.get_mut(|val| { *val = whatever; })` to be a bit awkward. I changed this to be an if-let instead.
3. Fix a bug which was suppressing the structured suggestion for some mutations via the index operator on `HashMap`/`BTreeMap`.
r? estebank or reassign
E0277: suggest dereferencing function arguments in more cases
This unifies and generalizes some of the logic in `TypeErrCtxt::suggest_dereferences` so that it will suggest dereferencing arguments to function/method calls in order to satisfy trait bounds in more cases.
Previously it would only fire on reference types, and it had two separate cases (one specifically to get through custom `Deref` impls when passing by-reference, and one specifically to catch #87437). I've based the new checks loosely on what's done for `E0308` in `FnCtxt::suggest_deref_or_ref`: it will suggest dereferences to satisfy trait bounds whenever the referent is `Copy`, is boxed (& so can be moved out of the boxes), or is being passed by reference.
This doesn't make the suggestion fire in contexts other than function arguments or binary operators (which are in a separate case that this doesn't touch), and doesn't make it suggest a combination of `&`-removal and dereferences. Those would require a bit more restructuring, so I figured just doing this would be a decent first step.
Closes#90997
borrowck diagnostics: make `add_move_error_suggestions` use the HIR rather than `SourceMap`
This PR aims to fix#132806 by rewriting `add_move_error_suggestions`[^1]. Previously, it manually scanned the source text to find a leading `&`, which isn't always going to produce a correct result (see: that issue). Admittedly, the HIR visitor in this PR introduces a lot of boilerplate, but hopefully the logic at its core isn't too complicated (I go over it in the comments). I also tried a simpler version that didn't use a HIR visitor and suggested adding `ref` always, but the `&ref x` suggestions really didn't look good. As a bonus for the added complexity though, it's now able to produce nice `&`-removing suggestions in more cases.
I tried to do this such that it avoids edition-dependent checks and its suggestions can be applied together with those from the match ergonomics 2024 migration lint. I haven't added tests for that since the details of match ergonomics 2024 are still being sorted out, but I can try if desired once that's finalized.
[^1]: In brief, it fires on patterns where users try to bind by-value in such a way that moves out of a reference to a non-Copy type (including slice references with non-copy elements). The suggestions are to change the binding's mode to be by-reference, either by removing[^2] an enclosing `&`/`&mut` or adding `ref` to the binding.
[^2]: Incidentally, I find the terminology of "consider removing the borrow" a bit confusing for a suggestion to remove a `&` pattern in order to make bindings borrow rather than move. I'm not sure what a good, concise way to explain that would be though, and that should go in a separate PR anyway.
```
error: `Default` impl doesn't use the declared default field values
--> $DIR/manual-default-impl-could-be-derived.rs:28:1
|
LL | / impl Default for B {
LL | | fn default() -> Self {
LL | | B {
LL | | x: s(),
| | --- this field has a default value
LL | | y: 0,
| | - this field has a default value
... |
LL | | }
| |_^
|
help: to avoid divergence in behavior between `Struct { .. }` and `<Struct as Default>::default()`, derive the `Default`
|
LL ~ #[derive(Default)] struct B {
|
```
Note that above the structured suggestion also includes completely removing the manual `impl`, but the rendering doesn't.
Account for C string literals and `format_args` in `HiddenUnicodeCodepoints` lint
This is stacked on #134955, and either that can land first or both of them can land together here. I split this out because this is a bit more involved of an impl.
Fixes#94945
compiler: Add a statement-of-intent to `rustc_abi`
This just documents the most basic idea of what the crate is even for in my view, rather than leaving that strewn about GitHub issues, PR reviews, and Zulip streams. In particular, I hope to make it clearer what code should go in `rustc_abi` and what should not, which is of immediate relevance to contributors.
I considered going even further and explaining ideas like "ABI compatibility", prologues, and so on. However, because of the cross-cutting nature of ABI, I think such explanations should probably live in the place for cross-cutting documents: the rustc dev guide. This is only meant to be a quick "by the way".
explicitly set float ABI for all ARM targets
We currently always set the `FloatABIType` field in the LLVM target machine to `Default`, which means LLVM infers the ARM float ABI (hard vs soft) from the LLVM target triple. This causes problems such as having to set the LLVM triple to `*-gnueabi` for our `musleabi` targets to ensure they get correctly inferred as soft-float targets. It also means rustc doesn't really know which float ABI ends up being used, which is a blocker for https://github.com/rust-lang/rust/pull/134794. So I think we should stop doing that and instead explicitly control that value. That's what this PR implements.
See [Zulip](https://rust-lang.zulipchat.com/#narrow/channel/187780-t-compiler.2Fwg-llvm/topic/Softfloat.20ABI.2C.20hardfloat.20instructions) for more context.
Best reviewed commit-by-commit. I hope I got all those `llvm_floatabi` values right...
stabilize const_swap
libs-api FCP passed in https://github.com/rust-lang/rust/issues/83163.
However, I only just realized that this actually involves an intrinsic. The intrinsic could be implemented entirely with existing stable const functionality, but we choose to make it a primitive to be able to detect more UB. So nominating for `@rust-lang/lang` to make sure they are aware; I leave it up to them whether they want to FCP this.
While at it I also renamed the intrinsic to make the "nonoverlapping" constraint more clear.
Fixes#83163
A couple datalog/borrowck cleanups
As discussed on zulip, here's a chill one in between slightly more interesting PRs:
- I hadn't noticed there still were a couple of datalog-related modules outside of their dedicated `polonius` module (go to horn-clause jail, bonk!).
- there somehow was both a `diags` module and a `diagnostics` module.
- a couple other tiny things being renamed -- let me know what you think.
As requested I've tried to have somewhat granular commits to ease review, but the last two or three could be squashed together, since they're all related to the `diags` module (but moving its contents is less tedious to check in its own commit).
r? `@jackh726`
rustc_codegen_ssa: Buffer file writes in link_rlib
This makes this step take ~25ms on my machine (M3 Max 64GB) for Zed repo instead of ~150ms (on editor crate). Additionally it takes down the time needed for a clean cargo build of ripgrep from ~6.1s to 5.9s.
This change is mostly relevant for dev builds of crates with multiple large CGUs.
I imagine it could be quite relevant for dev scenarios on Windows, but sadly I have no way to measure that myself.
Compute liveness constraints in location-sensitive polonius
This continues the location-sensitive prototype. In this episode, we build the liveness constraints.
Reminder of the approach we're taking: we need variance data to create liveness edges in the forward/backward/both directions (respectively in the cases of covariance, contravariance, invariance) in the localized constraint graph.
This PR:
- introduces the holder for that, and for the liveness data in the correct shape: the transpose of what we're using today, "live regions per points".
- records use/drop live region variance during tracing
- records regular live region variance at the end of liveness
- records the correctly shaped live region per point matrix
- uses all of the above to compute the liveness constraints
(There's still technically one tiny part of the liveness owl left to do, but I'll leave it for a future PR: we also need to disable the NLL optimization that avoids computing liveness for locals whose types contain a region outliving a free region -- the existing constraints make it effectively live at all points; this doesn't work under polonius)
r? `@jackh726` cc `@matthewjasper`
Rollup of 4 pull requests
Successful merges:
- #134870 (Fix sentence fragment in `pin` module docs)
- #134884 (Fix typos)
- #134892 (Added codegen test for elidings bounds check when indexes are manually checked)
- #134894 (Document how to run the split Docker pipelines)
r? `@ghost`
`@rustbot` modify labels: rollup
