Construct OutputType using macro and print [=FILENAME] help info
Closes#139805
Use define_output_types to define variants of OutputType, as well as refactor all of its methods for clarity. This way no variant is missed when pattern matching or output help messages.
On top of that, I optimized for `emit` help messages.
r? ```@jieyouxu```
Don't canonicalize crate paths
When printing paths in diagnostic we should favour printing the paths that were passed in rather than resolving all symlinks.
This PR changes the form of the crate path but it should only really affect diagnostics as filesystem functions won't care which path is used. The uncanonicalized path was already used as a fallback for when canonicalization failed.
This is a partial alternative to #139823.
run-make: drop `os_pipe` workaround now that `anonymous_pipe` is stable on beta
Follow-up to #137537 where I had to include a temporary dep on `os_pipe` before `anonymous_pipe` was stabilized. Now that `anonymous_pipe` is stable on beta, we can get rid of this workaround.
Closes#137532. (Final cleanup item)
r? `@Kobzol`
Hide unstable print kinds within emit_unknown_print_request_help in stable channel
Fixes#138698
We need to get the channel from `matches`. However, since `matches`(Line 1169) is constructed after `rustc_optgroups` (Line1165, where `RustcOptGroup::value_hint` is generated, i.e. what `rustc --print print` prints), I've left it unchanged here for now.
2da29dbe8f/compiler/rustc_driver_impl/src/lib.rs (L1161-L1169)
There is actually a way to manually parse the `--crate-name` parameter, but I'm afraid that's an unorthodox practice. So I conservatively just modified `emit_unknown_print_request_help` to print different parameters depending on whether they are nightly or not when passing the error parameter.
r? ```@jieyouxu```
Stabilize `-Zdwarf-version` as `-Cdwarf-version`
I propose stabilizing `-Zdwarf-version` as `-Cdwarf-version`. This PR adds a new `-Cdwarf-version` flag, leaving the unstable `-Z` flag as is to ease the transition period. The `-Z` flag will be removed in the future.
# `-Zdwarf-version` stabilization report
## What is the RFC for this feature and what changes have occurred to the user-facing design since the RFC was finalized?
No RFC/MCP, this flag was added in https://github.com/rust-lang/rust/pull/98350 and was not deemed large enough to require additional process.
The tracking issue for this feature is #103057.
## What behavior are we committing to that has been controversial? Summarize the major arguments pro/con.
None that has been extensively debated but there are a few questions that could have been chosen differently:
1. What should the flag name be?
The current flag name is very specific to DWARF. Other debuginfo formats exist (msvc's CodeView format or https://en.wikipedia.org/wiki/Stabs) so we could have chosen to generalize the flag name (`-{C,Z} debuginfo-version=dwarf-5` for example). While this would extend cleanly to support formats other than DWARF, there are some downsides to this design. Neither CodeView nor Stabs have specification or format versions so it's not clear what values would be supported beyond `dwarf-{2,3,4,5}` or `codeview`. We would also need to take care to ensure the name does not lead users to think they can pick a format other than one supported by the target. For instance, what would `--target x86_64-pc-windows-msvc -Cdebuginfo-version=dwarf-5` do?
2. What is the behavior when flag is used on targets that do not support DWARF?
Currently, passing `-{C,Z} dwarf-version` on targets like `*-windows-msvc` does not do anything. It may be preferable to emit a warning alerting the user that the flag has no effect on the target platform. Alternatively, we could emit an error but this could be annoying since it would require the use of target specific RUSTFLAGS to use the flag correctly (and there isn't a way to target "any platform that uses DWARF" using cfgs).
3. Does the precompiled standard library potentially using a different version of DWARF a problem?
I don't believe this is an issue as debuggers (and other such tools) already must deal with the possibility that an application uses different DWARF versions across its statically or dynamically linked libraries.
## Are there extensions to this feature that remain unstable? How do we know that we are not accidentally committing to those.
No extensions per se, although future DWARF versions could be considered as such. At present, we validate the requested DWARF version is between 2 and 5 (inclusive) so new DWARF versions will not automatically be supported until the validation logic is adjusted.
## Summarize the major parts of the implementation and provide links into the code (or to PRs)
- Targets define their preferred or default DWARF version: 34a5ea911c/compiler/rustc_target/src/spec/mod.rs (L2369)
- We use the target default but this can be overriden by `-{C,Z} dwarf-version` 34a5ea911c/compiler/rustc_session/src/session.rs (L738)
- The flag is validated 34a5ea911c/compiler/rustc_session/src/session.rs (L1253-L1258)
- When debuginfo is generated, we tell LLVM to use the requested value or the target default 34a5ea911c/compiler/rustc_codegen_llvm/src/debuginfo/mod.rs (L106)
## Summarize existing test coverage of this feature
- Test that we actually generate the appropriate DWARF version
- https://github.com/rust-lang/rust/blob/master/tests/assembly/dwarf5.rs
- https://github.com/rust-lang/rust/blob/master/tests/assembly/dwarf4.rs
- Test that LTO with different DWARF versions picks the highest version
- https://github.com/rust-lang/rust/blob/master/tests/assembly/dwarf-mixed-versions-lto.rs
- Test DWARF versions 2-5 are valid while 0, 1 and 6 report an error
- https://github.com/rust-lang/rust/blob/master/tests/ui/debuginfo/dwarf-versions.rs
- Ensure LLVM does not report a warning when LTO'ing different DWARF versions together
- https://github.com/rust-lang/rust/blob/master/tests/ui/lto/dwarf-mixed-versions-lto.rs
## Has a call-for-testing period been conducted? If so, what feedback was received?
No call-for-testing has been conducted but Rust for Linux has been using this flag without issue.
## What outstanding bugs in the issue tracker involve this feature? Are they stabilization-blocking?
All reported bugs have been resolved.
## Summarize contributors to the feature by name for recognition and assuredness that people involved in the feature agree with stabilization
- Initial implementation in https://github.com/rust-lang/rust/pull/98350 by `@pcwalton`
- Stop emitting `.debug_pubnames` and `.debug_pubtypes` when using DWARF 5 in https://github.com/rust-lang/rust/pull/117962 by `@weihanglo.`
- Refactoring & cleanups (#135739), fix LLVM warning on LTO with different DWARF versions (#136659) and argument validation (#136746) by `@wesleywiser`
## What FIXMEs are still in the code for that feature and why is it ok to leave them there?
No FIXMEs related to this feature.
## What static checks are done that are needed to prevent undefined behavior?
This feature cannot cause undefined behavior.
We ensure the DWARF version is one of the supported values [here](34a5ea911c/compiler/rustc_session/src/session.rs (L1255-L1257)).
## In what way does this feature interact with the reference/specification, and are those edits prepared?
No changes to reference/spec, unstable rustc docs are moved to the stable book as part of the stabilization PR.
## Does this feature introduce new expressions and can they produce temporaries? What are the lifetimes of those temporaries?
No.
## What other unstable features may be exposed by this feature?
`-Zembed-source` requires use of DWARF 5 extensions but has its own feature gate.
## What is tooling support like for this feature, w.r.t rustdoc, clippy, rust-analzyer, rustfmt, etc.?
No support needed for rustdoc, clippy, rust-analyzer, rustfmt or rustup.
Cargo could expose this as an option in build profiles but I would expect the decision as to what version should be used would be made for the entire crate graph at build time rather than by individual package authors.
cc-rs has support for detecting the presence of `-{C,Z} dwarf-version` in `RUSTFLAGS` and providing the corresponding flag to Clang/gcc (https://github.com/rust-lang/cc-rs/pull/1395).
---
Closes#103057
Some tests fail on cross-compiled targets due to various linker problems
on cross-compiled target, and having test coverage for these against
cross-compiled targets is nice but not necessary.
Prepend temp files with per-invocation random string to avoid temp filename conflicts
https://github.com/rust-lang/rust/issues/139407 uncovered a very subtle unsoundness with incremental codegen, failing compilation sessions (due to assembler errors), and the "prefer hard linking over copying files" strategy we use in the compiler for file management.
Specifically, imagine we're building a single file 3 times, all with `-Csave-temps -Cincremental=...`. Let's call the object file we're building for the codegen unit for `main` "`XXX.o`" just for clarity since it's probably some gigantic hash name:
```
#[inline(never)]
#[cfg(any(rpass1, rpass3))]
fn a() -> i32 {
0
}
#[cfg(any(cfail2))]
fn a() -> i32 {
1
}
fn main() {
evil::evil();
assert_eq!(a(), 0);
}
mod evil {
#[cfg(any(rpass1, rpass3))]
pub fn evil() {
unsafe {
std::arch::asm!("/* */");
}
}
#[cfg(any(cfail2))]
pub fn evil() {
unsafe {
std::arch::asm!("missing");
}
}
}
```
Session 1 (`rpass1`):
* Type-check, borrow-check, etc.
* Serialize the dep graph to the incremental working directory `.../s-...-working/`.
* Codegen object file to a temp file `XXX.rcgu.o` which is spit out in the cwd.
* Hard-link[^1] `XXX.rcgu.o` to the incremental working directory `.../s-...-working/XXX.o`.
* Save-temps option means we don't delete `XXX.rgcu.o`.
* Link the binary and stuff.
* Finalize[^2] the working incremental session by renaming `.../s-...-working` to ` s-...-asjkdhsjakd` (some other finalized incr comp session dir name).
Session 2 (`cfail2`):
* Load artifacts from the previous *finalized* incremental session, namely the dep graph.
* Type-check, borrow-check, etc. since the file has changed, so most dep graph nodes are red.
* Serialize the dep graph to the incremental working directory `.../s-...-working/`.
* Codegen object file to a temp file `XXX.rcgu.o`. **HERE IS THE PROBLEM**: The hard-link is still set up to point to the inode from `XXX.o` from the first session, so this also modifies the `XXX.o` in the previous finalized session directory.
* Codegen emits an error b/c `missing` is not an instruction, so we abort before finalizing the incremental session. Specifically, this means that the *previous* session is the last finalized session.
Session 3 (`rpass3`):
* Load artifacts from the previous *finalized* incremental session, namely the dep graph. NOTE that this is from session 1.
* All the dep graph nodes are green since we are basically replaying session 1.
* codegen object file `XXX.o`, which is detected as *reused* from session 1 since dep nodes were green. That means we **reuse** `XXX.o` which had been dirtied from session 2.
* Link the binary and stuff.
This results in a binary which reuses some of the build artifacts from session 2, but thinks it's from session 1.
At this point, I hope it's clear to see that the incremental results from session 1 were dirtied from session 2, but we reuse them as if session 1 was the previous (finalized) incremental session we ran. This is at best really buggy, and at worst **unsound**.
This isn't limited to `-C save-temps`, since there are other combinations of flags that may keep around temporary files (hard linked) in the working directory (like `-C debuginfo=1 -C split-debuginfo=unpacked` on darwin, for example).
---
This PR implements a fix which is to prepend temp filenames with a random string that is generated per invocation of rustc. This string is not *deterministic*, but temporary files are transient anyways, so I don't believe this is a problem.
That means that temp files are now something like... `{crate-name}.{cgu}.{invocation_temp}.rcgu.o`, where `{invocation_temp}` is the new temporary string we generate per invocation of rustc.
Fixes https://github.com/rust-lang/rust/issues/139407
[^1]: 175dcc7773/compiler/rustc_fs_util/src/lib.rs (L60)
[^2]: 175dcc7773/compiler/rustc_incremental/src/persist/fs.rs (L1-L40)
Greatly simplify doctest parsing and information extraction
The original process was pretty terrible, as it tried to extract information such as attributes by performing matches over tokens like `#!`, which doesn't work very well considering you can have `# ! [`, which is valid.
Also, it now does it in one pass: if the parser is happy, then we try to extract information, otherwise we return early.
r? `@fmease`
Reintroduce remote-test support in run-make tests
The old Makefile-based infrastructure included support for executing binaries with remote-test-client if configured, but that didn't get ported to run_make_support as part of the rmake migration.
This PR re-introduces back that support, with the same implementation (and limitations) of the original Makefile-based support.
[Old Makefile-based implementation of this](9b8accbeb6/tests/run-make/tools.mk (L65-L74))
try-job: armhf-gnu
I'm not entirely sure this is correct, but it doesn't feel
obviously-wrong so I figured I'd just start by sending a PR rather than
filing a bug and letting it linger.
@rustbot label llvm-main
Mangle rustc_std_internal_symbols functions
This reduces the risk of issues when using a staticlib or rust dylib compiled with a different rustc version in a rust program. Currently this will either (in the case of staticlib) cause a linker error due to duplicate symbol definitions, or (in the case of rust dylibs) cause rustc_std_internal_symbols functions to be silently overridden. As rust gets more commonly used inside the implementation of libraries consumed with a C interface (like Spidermonkey, Ruby YJIT (curently has to do partial linking of all rust code to hide all symbols not part of the C api), the Rusticl OpenCL implementation in mesa) this is becoming much more of an issue. With this PR the only symbols remaining with an unmangled name are rust_eh_personality (LLVM doesn't allow renaming it) and `__rust_no_alloc_shim_is_unstable`.
Helps mitigate https://github.com/rust-lang/rust/issues/104707
try-job: aarch64-gnu-debug
try-job: aarch64-apple
try-job: x86_64-apple-1
try-job: x86_64-mingw-1
try-job: i686-mingw-1
try-job: x86_64-msvc-1
try-job: i686-msvc-1
try-job: test-various
try-job: armhf-gnu
Update the standard library to Rust 2024
This updates the standard library to Rust 2024. This includes the following notable changes:
- Macros are updated to use new expression fragment specifiers. This PR includes a test to illustrate the changes, primarily allowing `const {...}` expressions now.
- Some tests show a change in MIR drop order. We do not believe this will be an observable change ([see zulip discussion](500972873)).
Fixes https://github.com/rust-lang/rust/issues/133081
strip `-Wlinker-messages` wrappers from `rust-lld` rmake test
The `tests/run-make/rust-lld` rmake test is failing locally on my M1, due to linker messages being in a different shape than the test expects: it asserts that the LLD version is the first linker message, which is seemingly not always the case on osx I guess.
```console
thread 'main' panicked at /Users/lqd/rust/lqd-rust/tests/run-make/rust-lld/rmake.rs:24:5:
the LLD version string should be present in the output logs:
warning: linker stderr: rust-lld: directory not found for option -L/usr/local/lib
LLD 20.1.0 (https://github.com/rust-lang/llvm-project.git 1c3bb96fdb6db7b8e8f24edb016099c223fdd27e)
Library search paths:
/Users/lqd/rust/lqd-rust/build/aarch64-apple-darwin/test/run-make/rust-lld/rmake_out
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib
Framework search paths:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks
```
This PR normalizes away the `-Wlinker-messages` wrappers around the linker output, to remove the requirement that the linker version is the first linker message / is prefixed with the warning wrapper in the regex.
(also another strange thing to explain the pre-existing regex: it seems the LLD version is sometimes output on stderr sometimes on stdout cool stuff)
We could do this for the other lld rmake tests, but they're only enabled on x64 linux so less likely to have random linker messages appearing without anyone noticing.
This prevents the included test case from getting at nightly-only
features when run on stable. The harness builds with
RUSTC_BOOTSTRAP, but the bundle doesn't.
