bjoernager/rust
bjoernager/rust
1
Fork 0
Code Issues Pull requests Activity
rust/compiler/rustc_serialize/tests/leb128.rs

67 lines
2.5 KiB
Rust
Raw Normal View History

Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
#![feature(maybe_uninit_slice)]
#![feature(maybe_uninit_uninit_array)]
libserialize: Unconfigure tests during normal build
2019-08-01 02:47:06 +03:00
use rustc_serialize::leb128::*;
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
use std::mem::MaybeUninit;
libserialize: Unconfigure tests during normal build
2019-08-01 02:47:06 +03:00
macro_rules! impl_test_unsigned_leb128 {
Format the world
2019-12-22 17:42:04 -05:00
($test_name:ident, $write_fn_name:ident, $read_fn_name:ident, $int_ty:ident) => {
libserialize: Unconfigure tests during normal build
2019-08-01 02:47:06 +03:00
#[test]
fn $test_name() {
let mut stream = Vec::new();
for x in 0..62 {
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
let mut buf = MaybeUninit::uninit_array();
stream.extend($write_fn_name(&mut buf, (3u64 << x) as $int_ty));
libserialize: Unconfigure tests during normal build
2019-08-01 02:47:06 +03:00
}
let mut position = 0;
for x in 0..62 {
let expected = (3u64 << x) as $int_ty;
Format the world
2019-12-22 17:42:04 -05:00
let (actual, bytes_read) = $read_fn_name(&stream[position..]);
libserialize: Unconfigure tests during normal build
2019-08-01 02:47:06 +03:00
assert_eq!(expected, actual);
position += bytes_read;
}
assert_eq!(stream.len(), position);
}
Format the world
2019-12-22 17:42:04 -05:00
};
libserialize: Unconfigure tests during normal build
2019-08-01 02:47:06 +03:00
}
impl_test_unsigned_leb128!(test_u16_leb128, write_u16_leb128, read_u16_leb128, u16);
impl_test_unsigned_leb128!(test_u32_leb128, write_u32_leb128, read_u32_leb128, u32);
impl_test_unsigned_leb128!(test_u64_leb128, write_u64_leb128, read_u64_leb128, u64);
impl_test_unsigned_leb128!(test_u128_leb128, write_u128_leb128, read_u128_leb128, u128);
impl_test_unsigned_leb128!(test_usize_leb128, write_usize_leb128, read_usize_leb128, usize);
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
macro_rules! impl_test_signed_leb128 {
($test_name:ident, $write_fn_name:ident, $int_ty:ident) => {
#[test]
fn $test_name() {
let values: Vec<_> = (-500..500)
.map(|i| ((i as $int_ty).wrapping_mul(0x12345789ABCDEF_i64 as $int_ty)))
.collect();
let mut stream = Vec::new();
for &x in &values {
let mut buf = MaybeUninit::uninit_array();
stream.extend($write_fn_name(&mut buf, x));
}
let mut position = 0;
for &x in &values {
let expected = x as i128;
let (actual, bytes_read) = read_signed_leb128(&mut stream, position);
assert_eq!(expected, actual);
position += bytes_read;
}
assert_eq!(stream.len(), position);
}
};
libserialize: Unconfigure tests during normal build
2019-08-01 02:47:06 +03:00
}
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
impl_test_signed_leb128!(test_i16_leb128, write_i16_leb128, i16);
impl_test_signed_leb128!(test_i32_leb128, write_i32_leb128, i32);
impl_test_signed_leb128!(test_i64_leb128, write_i64_leb128, i64);
impl_test_signed_leb128!(test_i128_leb128, write_i128_leb128, i128);
impl_test_signed_leb128!(test_isize_leb128, write_isize_leb128, isize);
Reference in a new issue Copy permalink