bjoernager/rust
bjoernager/rust
1
Fork 0
Code Activity

Auto merge of #124686 - saethlin:rust-file-footer, r=fmease

Browse source 
Add a footer in FileEncoder and check for it in MemDecoder

We have a few reports of ICEs due to decoding failures, where the fault does not lie with the compiler. The goal of this PR is to add some very lightweight and on-by-default validation to the compiler's outputs. If validation fails, we emit a fatal error for rmeta files in general that mentions the path that didn't load, and for incremental compilation artifacts we emit a verbose warning that tries to explain the situation and treat the artifacts as outdated.

The validation currently implemented here is very crude, and yet I think we have 11 ICE reports currently open (you can find them by searching issues for `1002111927320821928687967599834759150`) which this simple validation would have detected. The structure of the code changes here should permit the addition of further validation code, such as a checksum, if it is merited. I would like to have code to detect corruption such as reported in https://github.com/rust-lang/rust/issues/124719, but I'm not yet sure how to do that efficiently, and this PR is already a good size.

The ICE reports I have in mind that this PR would have smoothed over are:
https://github.com/rust-lang/rust/issues/124469
https://github.com/rust-lang/rust/issues/123352
https://github.com/rust-lang/rust/issues/123376 [^1]
https://github.com/rust-lang/rust/issues/99763
https://github.com/rust-lang/rust/issues/93900.

---

[^1]: This one might be a compiler bug, but even if it is I think the workflow described is pushing the envelope of what we can support. This issue is one of the reasons this warning still asks people to file an issue.
This commit is contained in:
bors 2024-05-22 15:59:56 +00:00
commit 22f5bdc42b
16 changed files with 122 additions and 52 deletions
Download patch file Download diff file Expand all files Collapse all files

38
compiler/rustc_middle/src/query/on_disk_cache.rs
View file

@ -154,24 +154,25 @@ impl EncodedSourceFileId {
impl<'sess> OnDiskCache<'sess> {
/// Creates a new `OnDiskCache` instance from the serialized data in `data`.
pub fn new(sess: &'sess Session, data: Mmap, start_pos: usize) -> Self {
debug_assert!(sess.opts.incremental.is_some());
///
/// The serialized cache has some basic integrity checks, if those checks indicate that the
/// on-disk data is corrupt, an error is returned.
pub fn new(sess: &'sess Session, data: Mmap, start_pos: usize) -> Result<Self, ()> {
assert!(sess.opts.incremental.is_some());
// Wrap in a scope so we can borrow `data`.
let footer: Footer = {
let mut decoder = MemDecoder::new(&data, start_pos);
let mut decoder = MemDecoder::new(&data, start_pos)?;
// Decode the *position* of the footer, which can be found in the
// last 8 bytes of the file.
let footer_pos = decoder
.with_position(decoder.len() - IntEncodedWithFixedSize::ENCODED_SIZE, |decoder| {
IntEncodedWithFixedSize::decode(decoder).0 as usize
});
// Decode the file footer, which contains all the lookup tables, etc.
decoder.with_position(footer_pos, |decoder| decode_tagged(decoder, TAG_FILE_FOOTER))
};
// Decode the *position* of the footer, which can be found in the
// last 8 bytes of the file.
let footer_pos = decoder
.with_position(decoder.len() - IntEncodedWithFixedSize::ENCODED_SIZE, |decoder| {
IntEncodedWithFixedSize::decode(decoder).0 as usize
});
// Decode the file footer, which contains all the lookup tables, etc.
let footer: Footer =
decoder.with_position(footer_pos, |decoder| decode_tagged(decoder, TAG_FILE_FOOTER));
Self {
Ok(Self {
serialized_data: RwLock::new(Some(data)),
file_index_to_stable_id: footer.file_index_to_stable_id,
file_index_to_file: Default::default(),
@ -184,7 +185,7 @@ impl<'sess> OnDiskCache<'sess> {
expn_data: footer.expn_data,
foreign_expn_data: footer.foreign_expn_data,
hygiene_context: Default::default(),
}
})
}
pub fn new_empty(source_map: &'sess SourceMap) -> Self {
@ -437,7 +438,8 @@ impl<'sess> OnDiskCache<'sess> {
let serialized_data = self.serialized_data.read();
let mut decoder = CacheDecoder {
tcx,
opaque: MemDecoder::new(serialized_data.as_deref().unwrap_or(&[]), pos.to_usize()),
opaque: MemDecoder::new(serialized_data.as_deref().unwrap_or(&[]), pos.to_usize())
.unwrap(),
source_map: self.source_map,
file_index_to_file: &self.file_index_to_file,
file_index_to_stable_id: &self.file_index_to_stable_id,
@ -558,7 +560,7 @@ impl<'a, 'tcx> TyDecoder for CacheDecoder<'a, 'tcx> {
{
debug_assert!(pos < self.opaque.len());
let new_opaque = MemDecoder::new(self.opaque.data(), pos);
let new_opaque = self.opaque.split_at(pos);
let old_opaque = mem::replace(&mut self.opaque, new_opaque);
let r = f(self);
self.opaque = old_opaque;