bjoernager/rust
bjoernager/rust
1
Fork 0
Code Activity

Auto merge of #133832 - madsmtm:apple-symbols.o, r=DianQK

Browse source 
Make `#[used]` work when linking with `ld64`

To make `#[used]` work in static libraries, we use the `symbols.o` trick introduced in https://github.com/rust-lang/rust/pull/95604.

However, the linker shipped with Xcode, ld64, works a bit differently from other linkers; in particular, [it completely ignores undefined symbols by themselves](https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/src/ld/parsers/macho_relocatable_file.cpp#L2455-L2468), and only consider them if they have relocations (something something atoms something fixups, I don't know the details).

So to make the `symbols.o` file work on ld64, we need to actually insert a relocation. That's kinda cumbersome to do though, since the relocation must be valid, and hence must point to a valid piece of machine code, and is hence very architecture-specific.

Fixes https://github.com/rust-lang/rust/issues/133491, see that for investigation.

---

Another option would be to pass `-u _foo` to the final linker invocation. This has the problem that `-u` causes the linker to not be able to dead-strip the symbol, which is undesirable. (If we did this, we would possibly also want to do it by putting the arguments in a file by itself, and passing that file via ``@`,` e.g. ``@undefined_symbols.txt`,` similar to https://github.com/rust-lang/rust/issues/52699, though that [is only supported since Xcode 12](https://developer.apple.com/documentation/xcode-release-notes/xcode-12-release-notes#Linking), and I'm not sure we wanna bump that).

Various other options that are probably all undesirable as they affect link time performance:
- Pass `-all_load` to the linker.
- Pass `-ObjC` to the linker (the Objective-C support in the linker has different code paths that load more of the binary), and instrument the binaries that contain `#[used]` symbols.
- Pass `-force_load` to libraries that contain `#[used]` symbols.

Failed attempt: Embed `-u _foo` in the object file with `LC_LINKER_OPTION`, akin to https://github.com/rust-lang/rust/issues/121293. Doesn't work, both because `ld64` doesn't read that from archive members unless it already has a reason to load the member (which is what this PR is trying to make it do), and because `ld64` only support the `-l`, `-needed-l`, `-framework` and `-needed_framework` flags in there.

---

TODO:
- [x] Support all Apple architectures.
- [x] Ensure that this works regardless of the actual type of the symbol.
- [x] Write up more docs.
- [x] Wire up a few proper tests.

`@rustbot` label O-apple
This commit is contained in:
bors 2025-02-25 11:59:11 +00:00
commit c51b9b6d52
7 changed files with 202 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

84
compiler/rustc_codegen_ssa/src/back/apple.rs
View file

@ -2,6 +2,7 @@ use std::env;
use std::fmt::{Display, from_fn};
use std::num::ParseIntError;
use rustc_middle::middle::exported_symbols::SymbolExportKind;
use rustc_session::Session;
use rustc_target::spec::Target;
@ -26,6 +27,89 @@ pub(super) fn macho_platform(target: &Target) -> u32 {
}
}
/// Add relocation and section data needed for a symbol to be considered
/// undefined by ld64.
///
/// The relocation must be valid, and hence must point to a valid piece of
/// machine code, and hence this is unfortunately very architecture-specific.
///
///
/// # New architectures
///
/// The values here are basically the same as emitted by the following program:
///
/// ```c
/// // clang -c foo.c -target $CLANG_TARGET
/// void foo(void);
///
/// extern int bar;
///
/// void* foobar[2] = {
/// (void*)foo,
/// (void*)&bar,
/// // ...
/// };
/// ```
///
/// Can be inspected with:
/// ```console
/// objdump --macho --reloc foo.o
/// objdump --macho --full-contents foo.o
/// ```
pub(super) fn add_data_and_relocation(
file: &mut object::write::Object<'_>,
section: object::write::SectionId,
symbol: object::write::SymbolId,
target: &Target,
kind: SymbolExportKind,
) -> object::write::Result<()> {
let authenticated_pointer =
kind == SymbolExportKind::Text && target.llvm_target.starts_with("arm64e");
let data: &[u8] = match target.pointer_width {
_ if authenticated_pointer => &[0, 0, 0, 0, 0, 0, 0, 0x80],
32 => &[0; 4],
64 => &[0; 8],
pointer_width => unimplemented!("unsupported Apple pointer width {pointer_width:?}"),
};
if target.arch == "x86_64" {
// Force alignment for the entire section to be 16 on x86_64.
file.section_mut(section).append_data(&[], 16);
} else {
// Elsewhere, the section alignment is the same as the pointer width.
file.section_mut(section).append_data(&[], target.pointer_width as u64);
}
let offset = file.section_mut(section).append_data(data, data.len() as u64);
let flags = if authenticated_pointer {
object::write::RelocationFlags::MachO {
r_type: object::macho::ARM64_RELOC_AUTHENTICATED_POINTER,
r_pcrel: false,
r_length: 3,
}
} else if target.arch == "arm" {
// FIXME(madsmtm): Remove once `object` supports 32-bit ARM relocations:
// https://github.com/gimli-rs/object/pull/757
object::write::RelocationFlags::MachO {
r_type: object::macho::ARM_RELOC_VANILLA,
r_pcrel: false,
r_length: 2,
}
} else {
object::write::RelocationFlags::Generic {
kind: object::RelocationKind::Absolute,
encoding: object::RelocationEncoding::Generic,
size: target.pointer_width as u8,
}
};
file.add_relocation(section, object::write::Relocation { offset, addend: 0, symbol, flags })?;
Ok(())
}
/// Deployment target or SDK version.
///
/// The size of the numbers in here are limited by Mach-O's `LC_BUILD_VERSION`.

59
compiler/rustc_codegen_ssa/src/back/link.rs
View file

@ -2058,8 +2058,8 @@ fn add_post_link_args(cmd: &mut dyn Linker, sess: &Session, flavor: LinkerFlavor
/// linker, and since they never participate in the linking, using `KEEP` in the linker scripts
/// can't keep them either. This causes #47384.
///
/// To keep them around, we could use `--whole-archive` and equivalents to force rlib to
/// participate in linking like object files, but this proves to be expensive (#93791). Therefore
/// To keep them around, we could use `--whole-archive`, `-force_load` and equivalents to force rlib
/// to participate in linking like object files, but this proves to be expensive (#93791). Therefore
/// we instead just introduce an undefined reference to them. This could be done by `-u` command
/// line option to the linker or `EXTERN(...)` in linker scripts, however they does not only
/// introduce an undefined reference, but also make them the GC roots, preventing `--gc-sections`
@ -2101,8 +2101,20 @@ fn add_linked_symbol_object(
file.set_mangling(object::write::Mangling::None);
}
// ld64 requires a relocation to load undefined symbols, see below.
// Not strictly needed if linking with lld, but might as well do it there too.
let ld64_section_helper = if file.format() == object::BinaryFormat::MachO {
Some(file.add_section(
file.segment_name(object::write::StandardSegment::Data).to_vec(),
"__data".into(),
object::SectionKind::Data,
))
} else {
None
};
for (sym, kind) in symbols.iter() {
file.add_symbol(object::write::Symbol {
let symbol = file.add_symbol(object::write::Symbol {
name: sym.clone().into(),
value: 0,
size: 0,
@ -2116,6 +2128,47 @@ fn add_linked_symbol_object(
section: object::write::SymbolSection::Undefined,
flags: object::SymbolFlags::None,
});
// The linker shipped with Apple's Xcode, ld64, works a bit differently from other linkers.
//
// Code-wise, the relevant parts of ld64 are roughly:
// 1. Find the `ArchiveLoadMode` based on commandline options, default to `parseObjects`.
// https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/src/ld/Options.cpp#L924-L932
// https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/src/ld/Options.h#L55
//
// 2. Read the archive table of contents (__.SYMDEF file).
// https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/src/ld/parsers/archive_file.cpp#L294-L325
//
// 3. Begin linking by loading "atoms" from input files.
// https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/doc/design/linker.html
// https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/src/ld/InputFiles.cpp#L1349
//
// a. Directly specified object files (`.o`) are parsed immediately.
// https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/src/ld/parsers/macho_relocatable_file.cpp#L4611-L4627
//
// - Undefined symbols are not atoms (`n_value > 0` denotes a common symbol).
// https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/src/ld/parsers/macho_relocatable_file.cpp#L2455-L2468
// https://maskray.me/blog/2022-02-06-all-about-common-symbols
//
// - Relocations/fixups are atoms.
// https://github.com/apple-oss-distributions/ld64/blob/ce6341ae966b3451aa54eeb049f2be865afbd578/src/ld/parsers/macho_relocatable_file.cpp#L2088-L2114
//
// b. Archives are not parsed yet.
// https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/src/ld/parsers/archive_file.cpp#L467-L577
//
// 4. When a symbol is needed by an atom, parse the object file that contains the symbol.
// https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/src/ld/InputFiles.cpp#L1417-L1491
// https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/src/ld/parsers/archive_file.cpp#L579-L597
//
// All of the steps above are fairly similar to other linkers, except that **it completely
// ignores undefined symbols**.
//
// So to make this trick work on ld64, we need to do something else to load the relevant
// object files. We do this by inserting a relocation (fixup) for each symbol.
if let Some(section) = ld64_section_helper {
apple::add_data_and_relocation(&mut file, section, symbol, &sess.target, *kind)
.expect("failed adding relocation");
}
}
let path = tmpdir.join("symbols.o");