This introduces no functional changes except for reducing a few unnecessary operations and variables. Vec has the behavior that, if you request space past the capacity with reserve(), it will round up to the nearest power of 2. What that effectively means is that after the first call to reserve(16), we are doubling our capacity every time. So using the DEFAULT_BUF_SIZE and doubling cap_size() here is meaningless and has no effect on the call to reserve().
Note that with #23842 implemented this will hopefully have a clearer API and less of a need for commenting. If #23842 is not implemented then the most clear implementation would be to call reserve_exact(buf.capacity()) at every step (and making sure that buf.capacity() is not zero at the beginning of the function of course).
Edit- functional change now introduced. We will now zero 16 bytes of the vector first, then double to 32, then 64, etc. until we read 64kB. This stops us from zeroing the entire vector when we double it, some of which may be wasted work. Reallocation still follows the doubling strategy, but the responsibility has been moved to vec.extend(), which calls reserve() and push_back().
This commit stabilizes a few remaining bits of the `io::Error` type:
* The `Error::new` method is now stable. The last `detail` parameter was removed
and the second `desc` parameter was generalized to `E: Into<Box<Error>>` to
allow creating an I/O error from any form of error. Currently there is no form
of downcasting, but this will be added in time.
* An implementation of `From<&str> for Box<Error>` was added to liballoc to
allow construction of errors from raw strings.
* The `Error::raw_os_error` method was stabilized as-is.
* Trait impls for `Clone`, `Eq`, and `PartialEq` were removed from `Error` as it
is not possible to use them with trait objects.
This is a breaking change due to the modification of the `new` method as well as
the removal of the trait implementations for the `Error` type.
[breaking-change]
* The `io::Seek` trait, and `SeekFrom` enum.
* The `Iterator::{partition, unsip}` methods.
* The `Vec::into_boxed_slice` method.
* The `LinkedList::append` method.
* The `{or_insert, or_insert_with` methods in the `Entry` APIs.
with_end_to_cap is enormously expensive now that it's initializing
memory since it involves 64k allocation + memset on every call. This is
most noticable when calling read_to_end on very small readers, where the
new version if **4 orders of magnitude** faster.
BufReader also depended on with_end_to_cap so I've rewritten it in its
original form.
As a bonus, converted the buffered IO struct Debug impls to use the
debug builders.
I first came across this in sfackler/rust-postgres#106 where a user reported a 10x performance regression. A call to read_to_end turned out to be the culprit: 9cd413d42c.
The new version differs from the old in a couple of ways. The buffer size used is now adaptive. It starts at 32 bytes and doubles each time EOF hasn't been reached up to a limit of 64k. In addition, the buffer is only truncated when EOF or an error has been reached, rather than after every call to read as was the case for the old implementation.
I wrote up a benchmark to compare the old version and new version: https://gist.github.com/sfackler/e979711b0ee2f2063462
It tests a couple of different cases: a high bandwidth reader, a low bandwidth reader, and a low bandwidth reader that won't return more than 10k per call to `read`. The high bandwidth reader should be analagous to use cases when reading from e.g. a `BufReader` or `Vec`, and the low bandwidth readers should be analogous to reading from something like a `TcpStream`.
Of special note, reads from a high bandwith reader containing 4 bytes are now *4,495 times faster*.
```
~/foo ❯ cargo bench
Compiling foo v0.0.1 (file:///home/sfackler/foo)
Running target/release/foo-7498d7dd7faecf5c
running 13 tests
test test_new ... ignored
test new_delay_4 ... bench: 230768 ns/iter (+/- 14812)
test new_delay_4_cap ... bench: 231421 ns/iter (+/- 7211)
test new_delay_5m ... bench: 14495370 ns/iter (+/- 4008648)
test new_delay_5m_cap ... bench: 73127954 ns/iter (+/- 59908587)
test new_nodelay_4 ... bench: 83 ns/iter (+/- 2)
test new_nodelay_5m ... bench: 12527237 ns/iter (+/- 335243)
test std_delay_4 ... bench: 373095 ns/iter (+/- 12613)
test std_delay_4_cap ... bench: 374190 ns/iter (+/- 19611)
test std_delay_5m ... bench: 17356012 ns/iter (+/- 15906588)
test std_delay_5m_cap ... bench: 883555035 ns/iter (+/- 205559857)
test std_nodelay_4 ... bench: 144937 ns/iter (+/- 2448)
test std_nodelay_5m ... bench: 16095893 ns/iter (+/- 3315116)
test result: ok. 0 passed; 0 failed; 1 ignored; 12 measured
```
r? @alexcrichton
with_end_to_cap is enormously expensive now that it's initializing
memory since it involves 64k allocation + memset on every call. This is
most noticable when calling read_to_end on very small readers, where the
new version if **4 orders of magnitude** faster.
BufReader also depended on with_end_to_cap so I've rewritten it in its
original form.
As a bonus, converted the buffered IO struct Debug impls to use the
debug builders.
Fixes#23815
Now that `<[_]>::split` is an inherent method, it will trump `BufRead::split`
when `BufRead` is in scope, so there is no longer a conflict. As a result,
calling `slice.split()` will probably always give you precisely what you want!
Main motivation was to update docs for the removal or "demotion" of certain extension traits. The update to the slice docs was larger, since the text was largely outdated.
This commit alters the behavior of the `Read::read_to_end()` method to zero all
memory instead of passing an uninitialized buffer to `read`. This change is
motivated by the [discussion on the internals forum][discuss] where the
conclusion has been that the standard library will not expose uninitialized
memory.
[discuss]: http://internals.rust-lang.org/t/uninitialized-memory/1652Closes#20314
This commit alters the behavior of the `Read::read_to_end()` method to zero all
memory instead of passing an uninitialized buffer to `read`. This change is
motivated by the [discussion on the internals forum][discuss] where the
conclusion has been that the standard library will not expose uninitialized
memory.
[discuss]: http://internals.rust-lang.org/t/uninitialized-memory/1652Closes#20314
The [associated RFC][rfc] for possibly splitting out `flush` has been closed and
as a result there are no more blockers for stabilizing this method, so this
commit marks the method as such.
[rfc]: https://github.com/rust-lang/rfcs/pull/950
The [associated RFC][rfc] for possibly splitting out `flush` has been closed and
as a result there are no more blockers for stabilizing this method, so this
commit marks the method as such.
[rfc]: https://github.com/rust-lang/rfcs/pull/950
r? @alexcrichton or @aturon
This still needs to somehow figure out how to avoid unstable warnings arising from the use of unstable functions. I tried to use `#[allow_internal_unstable]` but it still spits out warnings as far as I can see. @huonw (I think you implemented it) does `#[allow_internal_unstable]` not work for some reason or am I using it incorrectly?
The new `std::io` module has had some time to bake now, and this commit
stabilizes its functionality. There are still portions of the module which
remain unstable, and below contains a summart of the actions taken.
This commit also deprecates the entire contents of the `old_io` module in a
blanket fashion. All APIs should now have a reasonable replacement in the
new I/O modules.
Stable APIs:
* `std::io` (the name)
* `std::io::prelude` (the name)
* `Read`
* `Read::read`
* `Read::{read_to_end, read_to_string}` after being modified to return a `usize`
for the number of bytes read.
* `Write`
* `Write::write`
* `Write::{write_all, write_fmt}`
* `BufRead`
* `BufRead::{fill_buf, consume}`
* `BufRead::{read_line, read_until}` after being modified to return a `usize`
for the number of bytes read.
* `BufReader`
* `BufReader::{new, with_capacity}`
* `BufReader::{get_ref, get_mut, into_inner}`
* `{Read,BufRead} for BufReader`
* `BufWriter`
* `BufWriter::{new, with_capacity}`
* `BufWriter::{get_ref, get_mut, into_inner}`
* `Write for BufWriter`
* `IntoInnerError`
* `IntoInnerError::{error, into_inner}`
* `{Error,Display} for IntoInnerError`
* `LineWriter`
* `LineWriter::{new, with_capacity}` - `with_capacity` was added
* `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added)
* `Write for LineWriter`
* `BufStream`
* `BufStream::{new, with_capacities}`
* `BufStream::{get_ref, get_mut, into_inner}`
* `{BufRead,Read,Write} for BufStream`
* `stdin`
* `Stdin`
* `Stdin::lock`
* `Stdin::read_line` - added method
* `StdinLock`
* `Read for Stdin`
* `{Read,BufRead} for StdinLock`
* `stdout`
* `Stdout`
* `Stdout::lock`
* `StdoutLock`
* `Write for Stdout`
* `Write for StdoutLock`
* `stderr`
* `Stderr`
* `Stderr::lock`
* `StderrLock`
* `Write for Stderr`
* `Write for StderrLock`
* `io::Result`
* `io::Error`
* `io::Error::last_os_error`
* `{Display, Error} for Error`
Unstable APIs:
(reasons can be found in the commit itself)
* `Write::flush`
* `Seek`
* `ErrorKind`
* `Error::new`
* `Error::from_os_error`
* `Error::kind`
Deprecated APIs
* `Error::description` - available via the `Error` trait
* `Error::detail` - available via the `Display` implementation
* `thread::Builder::{stdout, stderr}`
Changes in functionality:
* `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing
backtraces has migrated to `std::io`.
* The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed
by folding functionality into the corresponding trait.
[breaking-change]
The new `io` module has had some time to bake and this commit stabilizes some of
the utilities associated with it. This commit also deprecates a number of
`std::old_io::util` functions and structures.
These items are now `#[stable]`
* `Cursor`
* `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}`
* Implementations of I/O traits for `Cursor<T>`
* Delegating implementations of I/O traits for references and `Box` pointers
* Implementations of I/O traits for primitives like slices and `Vec<T>`
* `ReadExt::bytes`
* `Bytes` (and impls)
* `ReadExt::chain`
* `Chain` (and impls)
* `ReadExt::take` (and impls)
* `BufReadExt::lines`
* `Lines` (and impls)
* `io::copy`
* `io::{empty, Empty}` (and impls)
* `io::{sink, Sink}` (and impls)
* `io::{repeat, Repeat}` (and impls)
These items remain `#[unstable]`
* Core I/O traits. These may want a little bit more time to bake along with the
commonly used methods like `read_to_end`.
* `BufReadExt::split` - this function may be renamed to not conflict with
`SliceExt::split`.
* `Error` - there are a number of questions about its representation,
`ErrorKind`, and usability.
These items are now `#[deprecated]` in `old_io`
* `LimitReader` - use `take` instead
* `NullWriter` - use `io::sink` instead
* `ZeroReader` - use `io::repeat` instead
* `NullReader` - use `io::empty` instead
* `MultiWriter` - use `broadcast` instead
* `ChainedReader` - use `chain` instead
* `TeeReader` - use `tee` instead
* `copy` - use `io::copy` instead
[breaking-change]
The new `io` module has had some time to bake and this commit stabilizes some of
the utilities associated with it. This commit also deprecates a number of
`std::old_io::util` functions and structures.
These items are now `#[stable]`
* `Cursor`
* `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}`
* Implementations of I/O traits for `Cursor<T>`
* Delegating implementations of I/O traits for references and `Box` pointers
* Implementations of I/O traits for primitives like slices and `Vec<T>`
* `ReadExt::bytes`
* `Bytes` (and impls)
* `ReadExt::chain`
* `Chain` (and impls)
* `ReadExt::take` (and impls)
* `BufReadExt::lines`
* `Lines` (and impls)
* `io::copy`
* `io::{empty, Empty}` (and impls)
* `io::{sink, Sink}` (and impls)
* `io::{repeat, Repeat}` (and impls)
These items remain `#[unstable]`
* Core I/O traits. These may want a little bit more time to bake along with the
commonly used methods like `read_to_end`.
* `BufReadExt::split` - this function may be renamed to not conflict with
`SliceExt::split`.
* `Error` - there are a number of questions about its representation,
`ErrorKind`, and usability.
These items are now `#[deprecated]` in `old_io`
* `LimitReader` - use `take` instead
* `NullWriter` - use `io::sink` instead
* `ZeroReader` - use `io::repeat` instead
* `NullReader` - use `io::empty` instead
* `MultiWriter` - use `broadcast` instead
* `ChainedReader` - use `chain` instead
* `TeeReader` - use `tee` instead
* `copy` - use `io::copy` instead
[breaking-change]
This is an implementation of RFC 899 and adds stdio functionality to the new
`std::io` module. Details of the API can be found on the RFC, but from a high
level:
* `io::{stdin, stdout, stderr}` constructors are now available. There are also
`*_raw` variants for unbuffered and unlocked access.
* All handles are globally shared (excluding raw variants).
* The stderr handle is no longer buffered.
* All handles can be explicitly locked (excluding the raw variants).
The `print!` and `println!` machinery has not yet been hooked up to these
streams just yet. The `std::fmt::output` module has also not yet been
implemented as part of this commit.
* Adds features and allows
* Removes unused muts, unused imports, dead code
* Migrates some deprecated code to new io/env
* Changes std::num::uint/int to be re-exports of std::num::usize/isize
libcollections, liballoc, and libcoretest no longer warn during testing.
libstd warns much less, though there's some dangly bits that weren't obvious fixes. In particular, how to only supress deprecated warnings in specific submodules of std.
New functions, `slice::from_raw_parts` and `slice::from_raw_parts_mut`,
are added to implement the lifetime convention as agreed in rust-lang/rfcs#556.
The functions `slice::from_raw_buf` and `slice::from_raw_mut_buf` are
left deprecated for the time being.
Holding back on changing the signature of `std::ffi::c_str_to_bytes` as consensus in rust-lang/rfcs#592 is building to replace it with a composition of other functions.
Contribution to #21923.
