Asserts the maximum value that can be returned from `Vec::len`
Currently, casting `Vec<i32>` to `Vec<u32>` takes O(1) time:
```rust
// See <https://godbolt.org/z/hxq3hnYKG> for assembly output.
pub fn cast(vec: Vec<i32>) -> Vec<u32> {
vec.into_iter().map(|e| e as _).collect()
}
```
But the generated assembly is not the same as the identity function, which prevents us from casting `Vec<Vec<i32>>` to `Vec<Vec<u32>>` within O(1) time:
```rust
// See <https://godbolt.org/z/7n48bxd9f> for assembly output.
pub fn cast(vec: Vec<Vec<i32>>) -> Vec<Vec<u32>> {
vec.into_iter()
.map(|e| e.into_iter().map(|e| e as _).collect())
.collect()
}
```
This change tries to fix the problem. You can see the comparison here: <https://godbolt.org/z/jdManrKvx>.
Optimize `is_ascii` for `str` and `[u8]` further
Replace the existing optimized function with one that enables auto-vectorization.
This is especially beneficial on x86-64 as `pmovmskb` can be emitted with careful structuring of the code. The instruction can detect non-ASCII characters one vector register width at a time instead of the current `usize` at a time check.
The resulting implementation is completely safe.
`case00_libcore` is the current implementation, `case04_while_loop` is this PR.
```
benchmarks:
ascii::is_ascii_slice::long::case00_libcore 22.25/iter +/- 1.09
ascii::is_ascii_slice::long::case04_while_loop 6.78/iter +/- 0.92
ascii::is_ascii_slice::medium::case00_libcore 2.81/iter +/- 0.39
ascii::is_ascii_slice::medium::case04_while_loop 1.56/iter +/- 0.78
ascii::is_ascii_slice::short::case00_libcore 5.55/iter +/- 0.85
ascii::is_ascii_slice::short::case04_while_loop 3.75/iter +/- 0.22
ascii::is_ascii_slice::unaligned_both_long::case00_libcore 26.59/iter +/- 0.66
ascii::is_ascii_slice::unaligned_both_long::case04_while_loop 5.78/iter +/- 0.16
ascii::is_ascii_slice::unaligned_both_medium::case00_libcore 2.97/iter +/- 0.32
ascii::is_ascii_slice::unaligned_both_medium::case04_while_loop 2.41/iter +/- 0.10
ascii::is_ascii_slice::unaligned_head_long::case00_libcore 23.71/iter +/- 0.79
ascii::is_ascii_slice::unaligned_head_long::case04_while_loop 7.83/iter +/- 1.31
ascii::is_ascii_slice::unaligned_head_medium::case00_libcore 3.69/iter +/- 0.54
ascii::is_ascii_slice::unaligned_head_medium::case04_while_loop 7.05/iter +/- 0.32
ascii::is_ascii_slice::unaligned_tail_long::case00_libcore 24.44/iter +/- 1.41
ascii::is_ascii_slice::unaligned_tail_long::case04_while_loop 5.12/iter +/- 0.18
ascii::is_ascii_slice::unaligned_tail_medium::case00_libcore 3.24/iter +/- 0.40
ascii::is_ascii_slice::unaligned_tail_medium::case04_while_loop 2.86/iter +/- 0.14
```
`unaligned_head_medium` is the main regression in the benchmarks. It is a 32 byte string being sliced `bytes[1..]`.
The first commit can be used to run the benchmarks against the current core implementation.
Previous implementation was done in #74066
---
Two potential drawbacks of this implementation are that it increases instruction count and may regress other platforms/architectures. The benches here may also be too artificial to glean much insight from.
https://rust.godbolt.org/z/G9znGfY36
This was fragile as it was based on host target passed to compiletest,
but the user could cross-compile and run test for a different target
(e.g. cross from linux to msvc, but msvc won't be set on the target).
Furthermore, it was also very surprising as normally revision names
(other than `CHECK`) was accepted as FileCheck prefixes.
Field init shorthand allows writing initializers like `tcx: tcx` as
`tcx`. The compiler already uses it extensively. Fix the last few places
where it isn't yet used.
We don't need `NonNull::as_ptr` debuginfo
In order to stop pessimizing the use of local variables in core, skip debug info for MIR temporaries in tiny (single-BB) functions.
For functions as simple as this -- `Pin::new`, etc -- nobody every actually wants debuginfo for them in the first place. They're more like intrinsics than real functions, and stepping over them is good.
codegen `#[naked]` functions using global asm
tracking issue: https://github.com/rust-lang/rust/issues/90957Fixes#124375
This implements the approach suggested in the tracking issue: use the existing global assembly infrastructure to emit the body of `#[naked]` functions. The main advantage is that we now have full control over what gets generated, and are no longer dependent on LLVM not sneakily messing with our output (inlining, adding extra instructions, etc).
I discussed this approach with `@Amanieu` and while I think the general direction is correct, there is probably a bunch of stuff that needs to change or move around here. I'll leave some inline comments on things that I'm not sure about.
Combined with https://github.com/rust-lang/rust/pull/127853, if both accepted, I think that resolves all steps from the tracking issue.
r? `@Amanieu`
we get these declarations
```
; opt level 0
declare x86_intrcc void @page_fault_handler(ptr byval([8 x i8]) align 8, i64) unnamed_addr #1
; opt level > 0
declare x86_intrcc void @page_fault_handler(ptr noalias nocapture noundef byval([8 x i8]) align 8 dereferenceable(8), i64 noundef) unnamed_addr #1
```
The space after `i64` in the original regex made the regex not match for
opt level 0. Removing the space fixes the issue.
```
declare x86_intrcc void @page_fault_handler(ptr {{.*}}, i64 {{.*}}){{.*}}#[[ATTRS:[0-9]+]]
```
Stop pessimizing the use of local variables in core by skipping debug info for MIR temporaries in tiny (single-BB) functions.
For functions as simple as this -- `Pin::new`, etc -- nobody every actually wants debuginfo for them in the first place. They're more like intrinsics than real functions, and stepping over them is good.
rust_for_linux: -Zreg-struct-return commandline flag for X86 (#116973)
Command line flag `-Zreg-struct-return` for X86 (32-bit) for rust-for-linux.
This flag enables the same behavior as the `abi_return_struct_as_int` target spec key.
- Tracking issue: https://github.com/rust-lang/rust/issues/116973
Gate async fn trait bound modifier on `async_trait_bounds`
This PR moves `async Fn()` trait bounds into a new feature gate: `feature(async_trait_bounds)`. The general vibe is that we will most likely stabilize the `feature(async_closure)` *without* the `async Fn()` trait bound modifier, so we need to gate that separately.
We're trying to work on the general vision of `async` trait bound modifier general in: https://github.com/rust-lang/rfcs/pull/3710, however that RFC still needs more time for consensus to converge, and we've decided that the value that users get from calling the bound `async Fn()` is *not really* worth blocking landing async closures in general.
Support input/output in vector registers of PowerPC inline assembly
This extends currently clobber-only vector registers (`vreg`) support to allow passing `#[repr(simd)]` types as input/output.
| Architecture | Register class | Target feature | Allowed types |
| ------------ | -------------- | -------------- | -------------- |
| PowerPC | `vreg` | `altivec` | `i8x16`, `i16x8`, `i32x4`, `f32x4` |
| PowerPC | `vreg` | `vsx` | `f32`, `f64`, `i64x2`, `f64x2` |
In addition to floats and `core::simd` types listed above, `core::arch` types and custom `#[repr(simd)]` types of the same size and type are also allowed. All allowed types and relevant target features are currently unstable.
r? `@Amanieu`
`@rustbot` label +O-PowerPC +A-inline-assembly
Support `clobber_abi` in AVR inline assembly
This PR implements the `clobber_abi` part necessary to eventually stabilize the inline assembly for AVR. This is tracked in #93335.
This is heavily inspired by the sibling-PR #131310 for the MSP430. I've explained my reasoning in the first commit message in detail, which is reproduced below for easier reviewing:
This follows the [ABI documentation] of AVR-GCC:
> The [...] call-clobbered general purpose registers (GPRs) are registers that might be destroyed (clobbered) by a function call.
>
> - **R18–R27, R30, R31**
>
> These GPRs are call clobbered. An ordinary function may use them without restoring the contents. [...]
>
> - **R0, T-Flag**
>
> The temporary register and the T-flag in SREG are also call-clobbered, but this knowledge is not exposed explicitly to the compiler (R0 is a fixed register).
Therefore this commit lists the aforementioned registers `r18–r27`, `r30` and `r31` as clobbered registers. Since the `r0` register (listed above as well) is not available in inline assembly at all (potentially because the AVR-GCC considers it a fixed register causing the register to never be used in register allocation and LLVM adopting this), there is no need to list it in the clobber list (the `r0`-variant is not even available). A comment was added to ensure, that the `r0` gets added to the clobber-list once the register gets usable in inline ASM.
Since the SREG is normally considered clobbered anyways (unless the user supplies the `preserve_flags`-option), there is no need to explicitly list a bit in this register (which is not possible to list anyways).
Note, that this commit completely ignores the case of interrupts (that are described in the ABI-specification), since every register touched in an ISR need to be saved anyways.
[ABI documentation]: https://gcc.gnu.org/wiki/avr-gcc#Call-Used_Registers
r? ``@Amanieu``
``@rustbot`` label +O-AVR
Enable -Zshare-generics for inline(never) functions
This avoids inlining cross-crate generic items when possible that are
already marked inline(never), implying that the author is not intending
for the function to be inlined by callers. As such, having a local copy
may make it easier for LLVM to optimize but mostly just adds to binary
bloat and codegen time. In practice our benchmarks indicate this is
indeed a win for larger compilations, where the extra cost in dynamic
linking to these symbols is diminished compared to the advantages in
fewer copies that need optimizing in each binary.
It might also make sense it expand this with other heuristics (e.g.,
`#[cold]`) in the future, but this seems like a good starting point.
FWIW, I expect that doing cleanup in where we make the decision
what should/shouldn't be shared is also a good idea. Way too
much code needed to be tweaked to check this. But I'm hoping
to leave that for a follow-up PR rather than blocking this on it.
Compiletest: add proc-macro header
This adds a `proc-macro` header to simplify using proc-macros, and to reduce boilerplate. This header works similar to the `aux-build` header where you pass a path for a proc-macro to be built.
This allows the `force-host`, `no-prefer-dynamic` headers, and `crate_type` attribute to be removed. Additionally it uses `--extern` like `aux_crate` (allows implicit `extern crate` in 2018) and `--extern proc_macro` (to place in the prelude in 2018).
~~This also includes a secondary change which defaults the edition of proc-macros to 2024. This further reduces boilerplate (removing `extern crate proc_macro;`), and allows using modern Rust syntax. I was a little on the fence including this. I personally prefer it, but I can imagine it might be confusing to others.~~ EDIT: Removed
Some tests were changed so that when there is a chain of dependencies A→B→C, that the `@ proc-macro` is placed in `B` instead of `A` so that the `--extern` flag works correctly (previously it depended on `-L` to find `C`). I think this is better to make the dependencies more explicit. None of these tests looked like the were actually testing this behavior.
There is one test that had an unexplained output change: `tests/ui/macros/same-sequence-span.rs`. I do not know why it changed, but it didn't look like it was particularly important. Perhaps there was a normalization issue?
This is currently not compatible with the rustdoc `build-aux-docs` header. It can probably be fixed, I'm just not feeling motivated to do that right now.
### Implementation steps
- [x] Document this new behavior in rustc-dev-guide once we figure out the specifics. https://github.com/rust-lang/rustc-dev-guide/pull/2149
This reduces code sizes and better respects programmer intent when
marking inline(never). Previously such a marking was essentially ignored
for generic functions, as we'd still inline them in remote crates.
Fix clobber_abi in RV32E and RV64E inline assembly
Currently clobber_abi in RV32E and RV64E inline assembly is implemented using InlineAsmClobberAbi::RiscV, but broken since x16-x31 cannot be used in RV32E and RV64E.
```
error: cannot use register `x16`: register can't be used with the `e` target feature
--> <source>:42:14
|
42 | asm!("", clobber_abi("C"), options(nostack, nomem, preserves_flags));
| ^^^^^^^^^^^^^^^^
error: cannot use register `x17`: register can't be used with the `e` target feature
--> <source>:42:14
|
42 | asm!("", clobber_abi("C"), options(nostack, nomem, preserves_flags));
| ^^^^^^^^^^^^^^^^
error: cannot use register `x28`: register can't be used with the `e` target feature
--> <source>:42:14
|
42 | asm!("", clobber_abi("C"), options(nostack, nomem, preserves_flags));
| ^^^^^^^^^^^^^^^^
error: cannot use register `x29`: register can't be used with the `e` target feature
--> <source>:42:14
|
42 | asm!("", clobber_abi("C"), options(nostack, nomem, preserves_flags));
| ^^^^^^^^^^^^^^^^
error: cannot use register `x30`: register can't be used with the `e` target feature
--> <source>:42:14
|
42 | asm!("", clobber_abi("C"), options(nostack, nomem, preserves_flags));
| ^^^^^^^^^^^^^^^^
error: cannot use register `x31`: register can't be used with the `e` target feature
--> <source>:42:14
|
42 | asm!("", clobber_abi("C"), options(nostack, nomem, preserves_flags));
| ^^^^^^^^^^^^^^^^
```
r? `@Amanieu`
`@rustbot` label O-riscv +A-inline-assembly
The test was checking for two `ptr` arguments by matching commas (or
non-commas), however after
https://github.com/llvm/llvm-project/pull/117104 LLVM adds an
`initializes((0, 16))` attribute, which includes a comma.
So instead, we make the test check for two LLVM values, i.e. something
prefixed by %.
(See also https://crbug.com/380707238)
When labels are present, the `noreturn` option really means that asm block
won't fallthrough -- if labels are present, then outputs can still be
meaningfully used.
Allow disabling ASan instrumentation for globals
AddressSanitizer adds instrumentation to global variables unless the [`no_sanitize_address`](https://llvm.org/docs/LangRef.html#global-attributes) attribute is set on them.
This commit extends the existing `#[no_sanitize(address)]` attribute to set this; previously it only had the desired effect on functions.
(cc https://github.com/rust-lang/rust/issues/39699)
aarch64 softfloat target: always pass floats in int registers
This is a part of https://github.com/rust-lang/rust/issues/131058: on softfloat aarch64 targets, the float registers may be unavailable. And yet, LLVM will happily use them to pass float types if the corresponding target features are enabled. That's a problem as it means enabling/disabling `neon` instructions can change the ABI.
Other targets have a `soft-float` target feature that forces the use of the soft-float ABI no matter whether float registers are enabled or not; aarch64 has nothing like that.
So we follow the aarch64 [softfloat ABI](https://github.com/rust-lang/rust/issues/131058#issuecomment-2385027423) and treat floats like integers for `extern "C"` functions. For the "Rust" ABI, we do the same for scalars, and then just do something reasonable for ScalarPair that avoids the pointer indirection.
Cc ```@workingjubilee```
