bjoernager/rust
bjoernager/rust
1
Fork 0
Code Issues Pull requests Activity

rustc: Persist LLVM's Linker in Fat LTO

Browse source 
This commit updates our Fat LTO logic to tweak our custom wrapper around LLVM's
"link modules" functionality. Previously whenever the
`LLVMRustLinkInExternalBitcode` function was called it would call LLVM's
`Linker::linkModules` wrapper. Internally this would crate an instance of a
`Linker` which internally creates an instance of an `IRMover`. Unfortunately for
us the creation of `IRMover` is somewhat O(n) with the input module. This means
that every time we linked a module it was O(n) with respect to the entire module
we had built up!

Now the modules we build up during LTO are quite large, so this quickly started
creating an O(n^2) problem for us! Discovered in #48025 it turns out this has
always been a problem and we just haven't noticed it. It became particularly
worse recently though due to most libraries having 16x more object files than
they previously did (1 -> 16).

This commit fixes this performance issue by preserving the `Linker` instance
across all links into the main LLVM module. This means we only create one
`IRMover` and allows LTO to progress much speedier.

From the `cargo-cache` project in #48025 a **full build** locally when from
5m15s to 2m24s. Looking at the timing logs each object file was linked in in
single-digit millisecond rather than hundreds, clearly being a nice improvement!

Closes #48025
This commit is contained in:
Alex Crichton 2018-02-12 08:38:46 -08:00
parent 16362c737f
commit 43e8ac27d9
5 changed files with 114 additions and 49 deletions
Download patch file Download diff file Expand all files Collapse all files

40
src/rustllvm/RustWrapper.cpp
View file

@ -916,46 +916,6 @@ extern "C" void LLVMRustWriteValueToString(LLVMValueRef V,
}
}
extern "C" bool LLVMRustLinkInExternalBitcode(LLVMModuleRef DstRef, char *BC,
size_t Len) {
Module *Dst = unwrap(DstRef);
std::unique_ptr<MemoryBuffer> Buf =
MemoryBuffer::getMemBufferCopy(StringRef(BC, Len));
#if LLVM_VERSION_GE(4, 0)
Expected<std::unique_ptr<Module>> SrcOrError =
llvm::getLazyBitcodeModule(Buf->getMemBufferRef(), Dst->getContext());
if (!SrcOrError) {
LLVMRustSetLastError(toString(SrcOrError.takeError()).c_str());
return false;
}
auto Src = std::move(*SrcOrError);
#else
ErrorOr<std::unique_ptr<Module>> Src =
llvm::getLazyBitcodeModule(std::move(Buf), Dst->getContext());
if (!Src) {
LLVMRustSetLastError(Src.getError().message().c_str());
return false;
}
#endif
std::string Err;
raw_string_ostream Stream(Err);
DiagnosticPrinterRawOStream DP(Stream);
#if LLVM_VERSION_GE(4, 0)
if (Linker::linkModules(*Dst, std::move(Src))) {
#else
if (Linker::linkModules(*Dst, std::move(Src.get()))) {
#endif
LLVMRustSetLastError(Err.c_str());
return false;
}
return true;
}
// Note that the two following functions look quite similar to the
// LLVMGetSectionName function. Sadly, it appears that this function only
// returns a char* pointer, which isn't guaranteed to be null-terminated. The