bjoernager/rust
bjoernager/rust
1
Fork 0
Code Issues Pull requests Activity

rustc_codegen_llvm: add support for writing summary bitcode

Browse source 
Typical uses of ThinLTO don't have any use for this as a standalone
file, but distributed ThinLTO uses this to make the linker phase more
efficient. With clang you'd do something like `clang -flto=thin
-fthin-link-bitcode=foo.indexing.o -c foo.c` and then get both foo.o
(full of bitcode) and foo.indexing.o (just the summary or index part of
the bitcode). That's then usable by a two-stage linking process that's
more friendly to distributed build systems like bazel, which is why I'm
working on this area.

I talked some to @teresajohnson about naming in this area, as things
seem to be a little confused between various blog posts and build
systems. "bitcode index" and "bitcode summary" tend to be a little too
ambiguous, and she tends to use "thin link bitcode" and "minimized
bitcode" (which matches the descriptions in LLVM). Since the clang
option is thin-link-bitcode, I went with that to try and not add a new
spelling in the world.

Per @dtolnay, you can work around the lack of this by using `lld
--thinlto-index-only` to do the indexing on regular .o files of
bitcode, but that is a bit wasteful on actions when we already have all
the information in rustc and could just write out the matching minimized
bitcode. I didn't test that at all in our infrastructure, because by the
time I learned that I already had this patch largely written.
This commit is contained in:
Augie Fackler 2024-01-19 14:42:43 -05:00
parent e8fbd99128
commit aa91871539
9 changed files with 85 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

14
compiler/rustc_llvm/llvm-wrapper/PassWrapper.cpp
View file

@ -1488,6 +1488,7 @@ LLVMRustPrepareThinLTOImport(const LLVMRustThinLTOData *Data, LLVMModuleRef M,
// a ThinLTO summary attached.
struct LLVMRustThinLTOBuffer {
std::string data;
std::string thin_link_data;
};
extern "C" LLVMRustThinLTOBuffer*
@ -1495,6 +1496,7 @@ LLVMRustThinLTOBufferCreate(LLVMModuleRef M, bool is_thin) {
auto Ret = std::make_unique<LLVMRustThinLTOBuffer>();
{
auto OS = raw_string_ostream(Ret->data);
auto ThinLinkOS = raw_string_ostream(Ret->thin_link_data);
{
if (is_thin) {
PassBuilder PB;
@ -1508,7 +1510,7 @@ LLVMRustThinLTOBufferCreate(LLVMModuleRef M, bool is_thin) {
PB.registerLoopAnalyses(LAM);
PB.crossRegisterProxies(LAM, FAM, CGAM, MAM);
ModulePassManager MPM;
MPM.addPass(ThinLTOBitcodeWriterPass(OS, nullptr));
MPM.addPass(ThinLTOBitcodeWriterPass(OS, &ThinLinkOS));
MPM.run(*unwrap(M), MAM);
} else {
WriteBitcodeToFile(*unwrap(M), OS);
@ -1533,6 +1535,16 @@ LLVMRustThinLTOBufferLen(const LLVMRustThinLTOBuffer *Buffer) {
return Buffer->data.length();
}
extern "C" const void*
LLVMRustThinLTOBufferThinLinkDataPtr(const LLVMRustThinLTOBuffer *Buffer) {
return Buffer->thin_link_data.data();
}
extern "C" size_t
LLVMRustThinLTOBufferThinLinkDataLen(const LLVMRustThinLTOBuffer *Buffer) {
return Buffer->thin_link_data.length();
}
// This is what we used to parse upstream bitcode for actual ThinLTO
// processing. We'll call this once per module optimized through ThinLTO, and
// it'll be called concurrently on many threads.