We needed a more efficient way to zerofill the vector in read_to_end.
This to reduce the memory intialization overhead to a minimum.
Use the implementation of `std::vec::from_elem` (used for the vec![]
macro) for Vec::resize as well. For simple element types like u8, this
compiles to memset, so it makes Vec::resize much more efficient.
This takes the cases from InvalidInput where a data format error
was encountered. This is different from the documented semantics
of InvalidInput, which more likely indicate a programming error.
This patch
1. renames libunicode to librustc_unicode,
2. deprecates several pieces of libunicode (see below), and
3. removes references to deprecated functions from
librustc_driver and libsyntax. This may change pretty-printed
output from these modules in cases involving wide or combining
characters used in filenames, identifiers, etc.
The following functions are marked deprecated:
1. char.width() and str.width():
--> use unicode-width crate
2. str.graphemes() and str.grapheme_indices():
--> use unicode-segmentation crate
3. str.nfd_chars(), str.nfkd_chars(), str.nfc_chars(), str.nfkc_chars(),
char.compose(), char.decompose_canonical(), char.decompose_compatible(),
char.canonical_combining_class():
--> use unicode-normalization crate
This introduces no functional changes except for reducing a few unnecessary operations and variables. Vec has the behavior that, if you request space past the capacity with reserve(), it will round up to the nearest power of 2. What that effectively means is that after the first call to reserve(16), we are doubling our capacity every time. So using the DEFAULT_BUF_SIZE and doubling cap_size() here is meaningless and has no effect on the call to reserve().
Note that with #23842 implemented this will hopefully have a clearer API and less of a need for commenting. If #23842 is not implemented then the most clear implementation would be to call reserve_exact(buf.capacity()) at every step (and making sure that buf.capacity() is not zero at the beginning of the function of course).
Edit- functional change now introduced. We will now zero 16 bytes of the vector first, then double to 32, then 64, etc. until we read 64kB. This stops us from zeroing the entire vector when we double it, some of which may be wasted work. Reallocation still follows the doubling strategy, but the responsibility has been moved to vec.extend(), which calls reserve() and push_back().
This commit stabilizes a few remaining bits of the `io::Error` type:
* The `Error::new` method is now stable. The last `detail` parameter was removed
and the second `desc` parameter was generalized to `E: Into<Box<Error>>` to
allow creating an I/O error from any form of error. Currently there is no form
of downcasting, but this will be added in time.
* An implementation of `From<&str> for Box<Error>` was added to liballoc to
allow construction of errors from raw strings.
* The `Error::raw_os_error` method was stabilized as-is.
* Trait impls for `Clone`, `Eq`, and `PartialEq` were removed from `Error` as it
is not possible to use them with trait objects.
This is a breaking change due to the modification of the `new` method as well as
the removal of the trait implementations for the `Error` type.
[breaking-change]
* The `io::Seek` trait, and `SeekFrom` enum.
* The `Iterator::{partition, unsip}` methods.
* The `Vec::into_boxed_slice` method.
* The `LinkedList::append` method.
* The `{or_insert, or_insert_with` methods in the `Entry` APIs.
with_end_to_cap is enormously expensive now that it's initializing
memory since it involves 64k allocation + memset on every call. This is
most noticable when calling read_to_end on very small readers, where the
new version if **4 orders of magnitude** faster.
BufReader also depended on with_end_to_cap so I've rewritten it in its
original form.
As a bonus, converted the buffered IO struct Debug impls to use the
debug builders.
I first came across this in sfackler/rust-postgres#106 where a user reported a 10x performance regression. A call to read_to_end turned out to be the culprit: 9cd413d42c.
The new version differs from the old in a couple of ways. The buffer size used is now adaptive. It starts at 32 bytes and doubles each time EOF hasn't been reached up to a limit of 64k. In addition, the buffer is only truncated when EOF or an error has been reached, rather than after every call to read as was the case for the old implementation.
I wrote up a benchmark to compare the old version and new version: https://gist.github.com/sfackler/e979711b0ee2f2063462
It tests a couple of different cases: a high bandwidth reader, a low bandwidth reader, and a low bandwidth reader that won't return more than 10k per call to `read`. The high bandwidth reader should be analagous to use cases when reading from e.g. a `BufReader` or `Vec`, and the low bandwidth readers should be analogous to reading from something like a `TcpStream`.
Of special note, reads from a high bandwith reader containing 4 bytes are now *4,495 times faster*.
```
~/foo ❯ cargo bench
Compiling foo v0.0.1 (file:///home/sfackler/foo)
Running target/release/foo-7498d7dd7faecf5c
running 13 tests
test test_new ... ignored
test new_delay_4 ... bench: 230768 ns/iter (+/- 14812)
test new_delay_4_cap ... bench: 231421 ns/iter (+/- 7211)
test new_delay_5m ... bench: 14495370 ns/iter (+/- 4008648)
test new_delay_5m_cap ... bench: 73127954 ns/iter (+/- 59908587)
test new_nodelay_4 ... bench: 83 ns/iter (+/- 2)
test new_nodelay_5m ... bench: 12527237 ns/iter (+/- 335243)
test std_delay_4 ... bench: 373095 ns/iter (+/- 12613)
test std_delay_4_cap ... bench: 374190 ns/iter (+/- 19611)
test std_delay_5m ... bench: 17356012 ns/iter (+/- 15906588)
test std_delay_5m_cap ... bench: 883555035 ns/iter (+/- 205559857)
test std_nodelay_4 ... bench: 144937 ns/iter (+/- 2448)
test std_nodelay_5m ... bench: 16095893 ns/iter (+/- 3315116)
test result: ok. 0 passed; 0 failed; 1 ignored; 12 measured
```
r? @alexcrichton
with_end_to_cap is enormously expensive now that it's initializing
memory since it involves 64k allocation + memset on every call. This is
most noticable when calling read_to_end on very small readers, where the
new version if **4 orders of magnitude** faster.
BufReader also depended on with_end_to_cap so I've rewritten it in its
original form.
As a bonus, converted the buffered IO struct Debug impls to use the
debug builders.
Fixes#23815
Now that `<[_]>::split` is an inherent method, it will trump `BufRead::split`
when `BufRead` is in scope, so there is no longer a conflict. As a result,
calling `slice.split()` will probably always give you precisely what you want!
Main motivation was to update docs for the removal or "demotion" of certain extension traits. The update to the slice docs was larger, since the text was largely outdated.
This commit alters the behavior of the `Read::read_to_end()` method to zero all
memory instead of passing an uninitialized buffer to `read`. This change is
motivated by the [discussion on the internals forum][discuss] where the
conclusion has been that the standard library will not expose uninitialized
memory.
[discuss]: http://internals.rust-lang.org/t/uninitialized-memory/1652Closes#20314
This commit alters the behavior of the `Read::read_to_end()` method to zero all
memory instead of passing an uninitialized buffer to `read`. This change is
motivated by the [discussion on the internals forum][discuss] where the
conclusion has been that the standard library will not expose uninitialized
memory.
[discuss]: http://internals.rust-lang.org/t/uninitialized-memory/1652Closes#20314
The [associated RFC][rfc] for possibly splitting out `flush` has been closed and
as a result there are no more blockers for stabilizing this method, so this
commit marks the method as such.
[rfc]: https://github.com/rust-lang/rfcs/pull/950
