Store LLVM bitcode in object files, not compressed
This commit is an attempted resurrection of #70458 where LLVM bitcode
emitted by rustc into rlibs is stored into object file sections rather
than in a separate file. The main rationale for doing this is that when
rustc emits bitcode it will no longer use a custom compression scheme
which makes it both easier to interoperate with existing tools and also
cuts down on compile time since this compression isn't happening.
The blocker for this in #70458 turned out to be that native linkers
didn't handle the new sections well, causing the sections to either
trigger bugs in the linker or actually end up in the final linked
artifact. This commit attempts to address these issues by ensuring that
native linkers ignore the new sections by inserting custom flags with
module-level inline assembly.
Note that this does not currently change the API of the compiler at all.
The pre-existing `-C bitcode-in-rlib` flag is co-opted to indicate
whether the bitcode should be present in the object file or not.
Finally, note that an important consequence of this commit, which is also
one of its primary purposes, is to enable rustc's `-Clto` bitcode
loading to load rlibs produced with `-Clinker-plugin-lto`. The goal here
is that when you're building with LTO Cargo will tell rustc to skip
codegen of all intermediate crates and only generate LLVM IR. Today
rustc will generate both object code and LLVM IR, but the object code is
later simply thrown away, wastefully.
This commit is an attempted resurrection of #70458 where LLVM bitcode
emitted by rustc into rlibs is stored into object file sections rather
than in a separate file. The main rationale for doing this is that when
rustc emits bitcode it will no longer use a custom compression scheme
which makes it both easier to interoperate with existing tools and also
cuts down on compile time since this compression isn't happening.
The blocker for this in #70458 turned out to be that native linkers
didn't handle the new sections well, causing the sections to either
trigger bugs in the linker or actually end up in the final linked
artifact. This commit attempts to address these issues by ensuring that
native linkers ignore the new sections by inserting custom flags with
module-level inline assembly.
Note that this does not currently change the API of the compiler at all.
The pre-existing `-C bitcode-in-rlib` flag is co-opted to indicate
whether the bitcode should be present in the object file or not.
Finally, note that an important consequence of this commit, which is also
one of its primary purposes, is to enable rustc's `-Clto` bitcode
loading to load rlibs produced with `-Clinker-plugin-lto`. The goal here
is that when you're building with LTO Cargo will tell rustc to skip
codegen of all intermediate crates and only generate LLVM IR. Today
rustc will generate both object code and LLVM IR, but the object code is
later simply thrown away, wastefully.
[experiment] Support linking from a .rlink file
Flag `-Z no-link` was previously introduced, which allows creating an `.rlink` file to perform compilation without linking. This change enables linking from an `.rlink` file.
Part of Issue #64191
Flag `-Z no-link` was previously introduced, which allows creating
an `.rlink` file to perform compilation without linking.
This change enables linking from an `.rlink` file.
This commit builds on #65501 continue to simplify the build system and
compiler now that we no longer have multiple LLVM backends to ship by
default. Here this switches the compiler back to what it once was long
long ago, which is linking LLVM directly to the compiler rather than
dynamically loading it at runtime. The `codegen-backends` directory of
the sysroot no longer exists and all relevant support in the build
system is removed. Note that `rustc` still supports a dynamically loaded
codegen backend as it did previously, it just no longer supports
dynamically loaded codegen backends in its own sysroot.
Additionally as part of this the `librustc_codegen_llvm` crate now once
again explicitly depends on all of its crates instead of implicitly
loading them through the sysroot. This involved filling out its
`Cargo.toml` and deleting all the now-unnecessary `extern crate`
annotations in the header of the crate. (this in turn required adding a
number of imports for names of macros too).
The end results of this change are:
* Rustbuild's build process for the compiler as all the "oh don't forget
the codegen backend" checks can be easily removed.
* Building `rustc_codegen_llvm` is much simpler since it's simply
another compiler crate.
* Managing the dependencies of `rustc_codegen_llvm` is much simpler since
it's "just another `Cargo.toml` to edit"
* The build process should be a smidge faster because there's more
parallelism in the main rustc build step rather than splitting
`librustc_codegen_llvm` out to its own step.
* The compiler is expected to be slightly faster by default because the
codegen backend does not need to be dynamically loaded.
* Disabling LLVM as part of rustbuild is still supported, supporting
multiple codegen backends is still supported, and dynamic loading of a
codegen backend is still supported.
SelfProfiler API refactoring and part one of event review
This PR refactors the `SelfProfiler` a little bit so that most profiling methods are RAII-based. The codegen backend code already had something similar, this refactoring pulls this functionality up into `SelfProfiler` itself, for general use.
The second commit of this PR is a review and update of the existing events we are already recording. Names have been made more consistent. CGU names have been removed from event names. They will be added back in when function parameter recording is implemented.
There is still some work to be done for adding new events, especially around trait resolution and the incremental system.
r? @wesleywiser
When performing a "fat" LTO the compiler has a whole mess of codegen
units that it links together. To do this it needs to select one module
as a "base" module and then link everything else into this module.
Previously LTO passes assume that there's at least one module in-memory
to link into, but nowadays that's not always true! With incremental
compilation modules may actually largely be cached and it may be
possible that there's no in-memory modules to work with.
This commit updates the logic of the LTO backend to handle modules a bit
more uniformly during a fat LTO. This commit immediately splits them
into two lists, one serialized and one in-memory. The in-memory list is
then searched for the largest module and failing that we simply
deserialize the first serialized module and link into that. This
refactoring avoids juggling three lists, two of which are serialized
modules and one of which is half serialized and half in-memory.
Closes#63349
This commit removes the crates.io dependency of `rustc-demangle` from
`rustc_codegen_llvm`. This crate is actually already pulled in to part
of the `librustc_driver` build and with the upcoming pipelining
implementation in Cargo it causes build issues if `rustc-demangle` is
left to its own devices.
This is not currently required, but once pipelining is enabled for
rustc's own build it will be required to build correctly.
