1
Fork 0
rust/library/std/src/io/mod.rs

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

2833 lines
90 KiB
Rust
Raw Normal View History

//! Traits, helpers, and type definitions for core I/O functionality.
//!
//! The `std::io` module contains a number of common things you'll need
//! when doing input and output. The most core part of this module is
2016-10-11 17:48:14 +02:00
//! the [`Read`] and [`Write`] traits, which provide the
//! most general interface for reading and writing input and output.
//!
//! # Read and Write
//!
2016-10-11 17:48:14 +02:00
//! Because they are traits, [`Read`] and [`Write`] are implemented by a number
2015-10-17 12:34:25 -07:00
//! of other types, and you can implement them for your types too. As such,
//! you'll see a few different types of I/O throughout the documentation in
2016-10-11 17:48:14 +02:00
//! this module: [`File`]s, [`TcpStream`]s, and sometimes even [`Vec<T>`]s. For
//! example, [`Read`] adds a [`read`][`Read::read`] method, which we can use on
2017-08-10 23:11:40 +02:00
//! [`File`]s:
//!
//! ```no_run
//! use std::io;
//! use std::io::prelude::*;
//! use std::fs::File;
//!
//! fn main() -> io::Result<()> {
//! let mut f = File::open("foo.txt")?;
//! let mut buffer = [0; 10];
//!
//! // read up to 10 bytes
//! let n = f.read(&mut buffer)?;
//!
//! println!("The bytes: {:?}", &buffer[..n]);
//! Ok(())
//! }
//! ```
//!
2016-10-11 17:48:14 +02:00
//! [`Read`] and [`Write`] are so important, implementors of the two traits have a
//! nickname: readers and writers. So you'll sometimes see 'a reader' instead
2016-10-11 17:48:14 +02:00
//! of 'a type that implements the [`Read`] trait'. Much easier!
//!
//! ## Seek and BufRead
//!
2016-10-11 17:48:14 +02:00
//! Beyond that, there are two important traits that are provided: [`Seek`]
//! and [`BufRead`]. Both of these build on top of a reader to control
//! how the reading happens. [`Seek`] lets you control where the next byte is
//! coming from:
//!
//! ```no_run
//! use std::io;
//! use std::io::prelude::*;
//! use std::io::SeekFrom;
//! use std::fs::File;
//!
//! fn main() -> io::Result<()> {
//! let mut f = File::open("foo.txt")?;
//! let mut buffer = [0; 10];
//!
//! // skip to the last 10 bytes of the file
//! f.seek(SeekFrom::End(-10))?;
//!
//! // read up to 10 bytes
//! let n = f.read(&mut buffer)?;
//!
//! println!("The bytes: {:?}", &buffer[..n]);
//! Ok(())
//! }
//! ```
//!
2016-10-11 17:48:14 +02:00
//! [`BufRead`] uses an internal buffer to provide a number of other ways to read, but
//! to show it off, we'll need to talk about buffers in general. Keep reading!
//!
//! ## BufReader and BufWriter
//!
//! Byte-based interfaces are unwieldy and can be inefficient, as we'd need to be
//! making near-constant calls to the operating system. To help with this,
2016-10-11 17:48:14 +02:00
//! `std::io` comes with two structs, [`BufReader`] and [`BufWriter`], which wrap
//! readers and writers. The wrapper uses a buffer, reducing the number of
//! calls and providing nicer methods for accessing exactly what you want.
//!
2016-10-11 17:48:14 +02:00
//! For example, [`BufReader`] works with the [`BufRead`] trait to add extra
//! methods to any reader:
//!
//! ```no_run
//! use std::io;
//! use std::io::prelude::*;
//! use std::io::BufReader;
//! use std::fs::File;
//!
//! fn main() -> io::Result<()> {
//! let f = File::open("foo.txt")?;
//! let mut reader = BufReader::new(f);
//! let mut buffer = String::new();
//!
//! // read a line into buffer
//! reader.read_line(&mut buffer)?;
//!
//! println!("{buffer}");
//! Ok(())
//! }
//! ```
//!
2016-10-11 17:48:14 +02:00
//! [`BufWriter`] doesn't add any new ways of writing; it just buffers every call
//! to [`write`][`Write::write`]:
//!
//! ```no_run
//! use std::io;
//! use std::io::prelude::*;
//! use std::io::BufWriter;
//! use std::fs::File;
//!
//! fn main() -> io::Result<()> {
//! let f = File::create("foo.txt")?;
//! {
//! let mut writer = BufWriter::new(f);
//!
//! // write a byte to the buffer
//! writer.write(&[42])?;
//!
//! } // the buffer is flushed once writer goes out of scope
//!
//! Ok(())
//! }
//! ```
//!
//! ## Standard input and output
//!
//! A very common source of input is standard input:
//!
//! ```no_run
//! use std::io;
//!
//! fn main() -> io::Result<()> {
//! let mut input = String::new();
//!
//! io::stdin().read_line(&mut input)?;
//!
//! println!("You typed: {}", input.trim());
//! Ok(())
//! }
//! ```
//!
2017-08-10 23:11:40 +02:00
//! Note that you cannot use the [`?` operator] in functions that do not return
//! a [`Result<T, E>`][`Result`]. Instead, you can call [`.unwrap()`]
2017-08-10 23:11:40 +02:00
//! or `match` on the return value to catch any possible errors:
2017-03-27 16:34:13 -04:00
//!
//! ```no_run
2017-03-28 13:27:46 -04:00
//! use std::io;
//!
//! let mut input = String::new();
2017-03-27 16:34:13 -04:00
//!
//! io::stdin().read_line(&mut input).unwrap();
//! ```
//!
//! And a very common source of output is standard output:
//!
//! ```no_run
//! use std::io;
//! use std::io::prelude::*;
//!
//! fn main() -> io::Result<()> {
//! io::stdout().write(&[42])?;
//! Ok(())
//! }
//! ```
//!
//! Of course, using [`io::stdout`] directly is less common than something like
2016-10-11 17:48:14 +02:00
//! [`println!`].
//!
//! ## Iterator types
//!
//! A large number of the structures provided by `std::io` are for various
2016-10-11 17:48:14 +02:00
//! ways of iterating over I/O. For example, [`Lines`] is used to split over
//! lines:
//!
//! ```no_run
//! use std::io;
//! use std::io::prelude::*;
//! use std::io::BufReader;
//! use std::fs::File;
//!
//! fn main() -> io::Result<()> {
//! let f = File::open("foo.txt")?;
//! let reader = BufReader::new(f);
//!
//! for line in reader.lines() {
//! println!("{}", line?);
//! }
//! Ok(())
//! }
//! ```
//!
//! ## Functions
//!
//! There are a number of [functions][functions-list] that offer access to various
//! features. For example, we can use three of these functions to copy everything
//! from standard input to standard output:
//!
//! ```no_run
//! use std::io;
//!
//! fn main() -> io::Result<()> {
//! io::copy(&mut io::stdin(), &mut io::stdout())?;
//! Ok(())
//! }
//! ```
//!
//! [functions-list]: #functions-1
//!
//! ## io::Result
//!
2016-10-11 17:48:14 +02:00
//! Last, but certainly not least, is [`io::Result`]. This type is used
//! as the return type of many `std::io` functions that can cause an error, and
//! can be returned from your own functions as well. Many of the examples in this
//! module use the [`?` operator]:
//!
//! ```
//! use std::io;
//!
//! fn read_input() -> io::Result<()> {
//! let mut input = String::new();
//!
//! io::stdin().read_line(&mut input)?;
//!
//! println!("You typed: {}", input.trim());
//!
//! Ok(())
//! }
//! ```
//!
2016-10-11 17:48:14 +02:00
//! The return type of `read_input()`, [`io::Result<()>`][`io::Result`], is a very
//! common type for functions which don't have a 'real' return value, but do want to
//! return errors if they happen. In this case, the only purpose of this function is
//! to read the line and print it, so we use `()`.
//!
//! ## Platform-specific behavior
//!
//! Many I/O functions throughout the standard library are documented to indicate
//! what various library or syscalls they are delegated to. This is done to help
//! applications both understand what's happening under the hood as well as investigate
//! any possibly unclear semantics. Note, however, that this is informative, not a binding
//! contract. The implementation of many of these functions are subject to change over
//! time and may call fewer or more syscalls/library functions.
2016-10-11 17:48:14 +02:00
//!
2020-07-18 12:02:03 -07:00
//! [`File`]: crate::fs::File
//! [`TcpStream`]: crate::net::TcpStream
//! [`io::stdout`]: stdout
2020-08-18 19:36:52 +02:00
//! [`io::Result`]: self::Result
2019-01-03 00:20:31 -05:00
//! [`?` operator]: ../../book/appendix-02-operators.html
2020-07-18 12:02:03 -07:00
//! [`Result`]: crate::result::Result
//! [`.unwrap()`]: crate::result::Result::unwrap
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
#![stable(feature = "rust1", since = "1.0.0")]
#[cfg(test)]
mod tests;
2019-02-11 04:23:21 +09:00
use crate::cmp;
use crate::fmt;
use crate::mem::replace;
2019-02-11 04:23:21 +09:00
use crate::ops::{Deref, DerefMut};
use crate::slice;
use crate::str;
2019-02-11 04:23:21 +09:00
use crate::sys;
2021-05-14 03:54:46 +02:00
use crate::sys_common::memchr;
#[stable(feature = "bufwriter_into_parts", since = "1.56.0")]
pub use self::buffered::WriterPanicked;
#[unstable(feature = "internal_output_capture", issue = "none")]
#[doc(no_inline, hidden)]
pub use self::stdio::set_output_capture;
#[unstable(feature = "print_internals", issue = "none")]
pub use self::stdio::{_eprint, _print};
#[stable(feature = "rust1", since = "1.0.0")]
pub use self::{
buffered::{BufReader, BufWriter, IntoInnerError, LineWriter},
copy::copy,
cursor::Cursor,
error::{Error, ErrorKind, Result},
stdio::{stderr, stdin, stdout, Stderr, StderrLock, Stdin, StdinLock, Stdout, StdoutLock},
util::{empty, repeat, sink, Empty, Repeat, Sink},
};
2021-01-17 22:28:18 -08:00
#[unstable(feature = "read_buf", issue = "78485")]
pub use self::readbuf::{BorrowedBuf, BorrowedCursor};
pub(crate) use error::const_io_error;
2021-01-17 22:28:18 -08:00
mod buffered;
2020-08-08 01:30:03 +02:00
pub(crate) mod copy;
mod cursor;
mod error;
mod impls;
pub mod prelude;
2021-01-17 22:28:18 -08:00
mod readbuf;
mod stdio;
mod util;
2019-02-11 04:23:21 +09:00
const DEFAULT_BUF_SIZE: usize = crate::sys_common::io::DEFAULT_BUF_SIZE;
pub(crate) use stdio::cleanup;
2021-04-11 07:05:39 +02:00
struct Guard<'a> {
buf: &'a mut Vec<u8>,
len: usize,
}
impl Drop for Guard<'_> {
fn drop(&mut self) {
unsafe {
self.buf.set_len(self.len);
}
}
}
Optimize File::read_to_end and read_to_string Reading a file into an empty vector or string buffer can incur unnecessary `read` syscalls and memory re-allocations as the buffer "warms up" and grows to its final size. This is perhaps a necessary evil with generic readers, but files can be read in smarter by checking the file size and reserving that much capacity. `std::fs::read` and `read_to_string` already perform this optimization: they open the file, reads its metadata, and call `with_capacity` with the file size. This ensures that the buffer does not need to be resized and an initial string of small `read` syscalls. However, if a user opens the `File` themselves and calls `file.read_to_end` or `file.read_to_string` they do not get this optimization. ```rust let mut buf = Vec::new(); file.read_to_end(&mut buf)?; ``` I searched through this project's codebase and even here are a *lot* of examples of this. They're found all over in unit tests, which isn't a big deal, but there are also several real instances in the compiler and in Cargo. I've documented the ones I found in a comment here: https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999 Most telling, the `Read` trait and the `read_to_end` method both show this exact pattern as examples of how to use readers. What this says to me is that this shouldn't be solved by simply fixing the instances of it in this codebase. If it's here it's certain to be prevalent in the wider Rust ecosystem. To that end, this commit adds specializations of `read_to_end` and `read_to_string` directly on `File`. This way it's no longer a minor footgun to start with an empty buffer when reading a file in. A nice side effect of this change is that code that accesses a `File` as a bare `Read` constraint or via a `dyn Read` trait object will benefit. For example, this code from `compiler/rustc_serialize/src/json.rs`: ```rust pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> { let mut contents = Vec::new(); match rdr.read_to_end(&mut contents) { ``` Related changes: - I also added specializations to `BufReader` to delegate to `self.inner`'s methods. That way it can call `File`'s optimized implementations if the inner reader is a file. - The private `std::io::append_to_string` function is now marked `unsafe`. - `File::read_to_string` being more efficient means that the performance note for `io::read_to_string` can be softened. I've added @camelid's suggested wording from: https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
2021-10-05 19:50:28 -04:00
// Several `read_to_string` and `read_line` methods in the standard library will
// append data into a `String` buffer, but we need to be pretty careful when
// doing this. The implementation will just call `.as_mut_vec()` and then
// delegate to a byte-oriented reading method, but we must ensure that when
// returning we never leave `buf` in a state such that it contains invalid UTF-8
// in its bounds.
//
// To this end, we use an RAII guard (to protect against panics) which updates
// the length of the string when it is dropped. This guard initially truncates
// the string to the prior length and only after we've validated that the
// new contents are valid UTF-8 do we allow it to set a longer length.
//
// The unsafety in this function is twofold:
//
// 1. We're looking at the raw bytes of `buf`, so we take on the burden of UTF-8
// checks.
// 2. We're passing a raw buffer to the function `f`, and it is expected that
// the function only *appends* bytes to the buffer. We'll get undefined
// behavior if existing bytes are overwritten to have non-UTF-8 data.
Optimize File::read_to_end and read_to_string Reading a file into an empty vector or string buffer can incur unnecessary `read` syscalls and memory re-allocations as the buffer "warms up" and grows to its final size. This is perhaps a necessary evil with generic readers, but files can be read in smarter by checking the file size and reserving that much capacity. `std::fs::read` and `read_to_string` already perform this optimization: they open the file, reads its metadata, and call `with_capacity` with the file size. This ensures that the buffer does not need to be resized and an initial string of small `read` syscalls. However, if a user opens the `File` themselves and calls `file.read_to_end` or `file.read_to_string` they do not get this optimization. ```rust let mut buf = Vec::new(); file.read_to_end(&mut buf)?; ``` I searched through this project's codebase and even here are a *lot* of examples of this. They're found all over in unit tests, which isn't a big deal, but there are also several real instances in the compiler and in Cargo. I've documented the ones I found in a comment here: https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999 Most telling, the `Read` trait and the `read_to_end` method both show this exact pattern as examples of how to use readers. What this says to me is that this shouldn't be solved by simply fixing the instances of it in this codebase. If it's here it's certain to be prevalent in the wider Rust ecosystem. To that end, this commit adds specializations of `read_to_end` and `read_to_string` directly on `File`. This way it's no longer a minor footgun to start with an empty buffer when reading a file in. A nice side effect of this change is that code that accesses a `File` as a bare `Read` constraint or via a `dyn Read` trait object will benefit. For example, this code from `compiler/rustc_serialize/src/json.rs`: ```rust pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> { let mut contents = Vec::new(); match rdr.read_to_end(&mut contents) { ``` Related changes: - I also added specializations to `BufReader` to delegate to `self.inner`'s methods. That way it can call `File`'s optimized implementations if the inner reader is a file. - The private `std::io::append_to_string` function is now marked `unsafe`. - `File::read_to_string` being more efficient means that the performance note for `io::read_to_string` can be softened. I've added @camelid's suggested wording from: https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
2021-10-05 19:50:28 -04:00
pub(crate) unsafe fn append_to_string<F>(buf: &mut String, f: F) -> Result<usize>
where
F: FnOnce(&mut Vec<u8>) -> Result<usize>,
{
Optimize File::read_to_end and read_to_string Reading a file into an empty vector or string buffer can incur unnecessary `read` syscalls and memory re-allocations as the buffer "warms up" and grows to its final size. This is perhaps a necessary evil with generic readers, but files can be read in smarter by checking the file size and reserving that much capacity. `std::fs::read` and `read_to_string` already perform this optimization: they open the file, reads its metadata, and call `with_capacity` with the file size. This ensures that the buffer does not need to be resized and an initial string of small `read` syscalls. However, if a user opens the `File` themselves and calls `file.read_to_end` or `file.read_to_string` they do not get this optimization. ```rust let mut buf = Vec::new(); file.read_to_end(&mut buf)?; ``` I searched through this project's codebase and even here are a *lot* of examples of this. They're found all over in unit tests, which isn't a big deal, but there are also several real instances in the compiler and in Cargo. I've documented the ones I found in a comment here: https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999 Most telling, the `Read` trait and the `read_to_end` method both show this exact pattern as examples of how to use readers. What this says to me is that this shouldn't be solved by simply fixing the instances of it in this codebase. If it's here it's certain to be prevalent in the wider Rust ecosystem. To that end, this commit adds specializations of `read_to_end` and `read_to_string` directly on `File`. This way it's no longer a minor footgun to start with an empty buffer when reading a file in. A nice side effect of this change is that code that accesses a `File` as a bare `Read` constraint or via a `dyn Read` trait object will benefit. For example, this code from `compiler/rustc_serialize/src/json.rs`: ```rust pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> { let mut contents = Vec::new(); match rdr.read_to_end(&mut contents) { ``` Related changes: - I also added specializations to `BufReader` to delegate to `self.inner`'s methods. That way it can call `File`'s optimized implementations if the inner reader is a file. - The private `std::io::append_to_string` function is now marked `unsafe`. - `File::read_to_string` being more efficient means that the performance note for `io::read_to_string` can be softened. I've added @camelid's suggested wording from: https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
2021-10-05 19:50:28 -04:00
let mut g = Guard { len: buf.len(), buf: buf.as_mut_vec() };
let ret = f(g.buf);
if str::from_utf8(&g.buf[g.len..]).is_err() {
ret.and_then(|_| {
Err(error::const_io_error!(
ErrorKind::InvalidData,
"stream did not contain valid UTF-8"
))
Optimize File::read_to_end and read_to_string Reading a file into an empty vector or string buffer can incur unnecessary `read` syscalls and memory re-allocations as the buffer "warms up" and grows to its final size. This is perhaps a necessary evil with generic readers, but files can be read in smarter by checking the file size and reserving that much capacity. `std::fs::read` and `read_to_string` already perform this optimization: they open the file, reads its metadata, and call `with_capacity` with the file size. This ensures that the buffer does not need to be resized and an initial string of small `read` syscalls. However, if a user opens the `File` themselves and calls `file.read_to_end` or `file.read_to_string` they do not get this optimization. ```rust let mut buf = Vec::new(); file.read_to_end(&mut buf)?; ``` I searched through this project's codebase and even here are a *lot* of examples of this. They're found all over in unit tests, which isn't a big deal, but there are also several real instances in the compiler and in Cargo. I've documented the ones I found in a comment here: https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999 Most telling, the `Read` trait and the `read_to_end` method both show this exact pattern as examples of how to use readers. What this says to me is that this shouldn't be solved by simply fixing the instances of it in this codebase. If it's here it's certain to be prevalent in the wider Rust ecosystem. To that end, this commit adds specializations of `read_to_end` and `read_to_string` directly on `File`. This way it's no longer a minor footgun to start with an empty buffer when reading a file in. A nice side effect of this change is that code that accesses a `File` as a bare `Read` constraint or via a `dyn Read` trait object will benefit. For example, this code from `compiler/rustc_serialize/src/json.rs`: ```rust pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> { let mut contents = Vec::new(); match rdr.read_to_end(&mut contents) { ``` Related changes: - I also added specializations to `BufReader` to delegate to `self.inner`'s methods. That way it can call `File`'s optimized implementations if the inner reader is a file. - The private `std::io::append_to_string` function is now marked `unsafe`. - `File::read_to_string` being more efficient means that the performance note for `io::read_to_string` can be softened. I've added @camelid's suggested wording from: https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
2021-10-05 19:50:28 -04:00
})
} else {
g.len = g.buf.len();
ret
}
}
// This uses an adaptive system to extend the vector when it fills. We want to
// avoid paying to allocate and zero a huge chunk of memory if the reader only
// has 4 bytes while still making large reads if the reader does have a ton
// of data to return. Simply tacking on an extra DEFAULT_BUF_SIZE space every
// time is 4,500 times (!) slower than a default reservation size of 32 if the
// reader has a very small amount of data to return.
Optimize File::read_to_end and read_to_string Reading a file into an empty vector or string buffer can incur unnecessary `read` syscalls and memory re-allocations as the buffer "warms up" and grows to its final size. This is perhaps a necessary evil with generic readers, but files can be read in smarter by checking the file size and reserving that much capacity. `std::fs::read` and `read_to_string` already perform this optimization: they open the file, reads its metadata, and call `with_capacity` with the file size. This ensures that the buffer does not need to be resized and an initial string of small `read` syscalls. However, if a user opens the `File` themselves and calls `file.read_to_end` or `file.read_to_string` they do not get this optimization. ```rust let mut buf = Vec::new(); file.read_to_end(&mut buf)?; ``` I searched through this project's codebase and even here are a *lot* of examples of this. They're found all over in unit tests, which isn't a big deal, but there are also several real instances in the compiler and in Cargo. I've documented the ones I found in a comment here: https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999 Most telling, the `Read` trait and the `read_to_end` method both show this exact pattern as examples of how to use readers. What this says to me is that this shouldn't be solved by simply fixing the instances of it in this codebase. If it's here it's certain to be prevalent in the wider Rust ecosystem. To that end, this commit adds specializations of `read_to_end` and `read_to_string` directly on `File`. This way it's no longer a minor footgun to start with an empty buffer when reading a file in. A nice side effect of this change is that code that accesses a `File` as a bare `Read` constraint or via a `dyn Read` trait object will benefit. For example, this code from `compiler/rustc_serialize/src/json.rs`: ```rust pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> { let mut contents = Vec::new(); match rdr.read_to_end(&mut contents) { ``` Related changes: - I also added specializations to `BufReader` to delegate to `self.inner`'s methods. That way it can call `File`'s optimized implementations if the inner reader is a file. - The private `std::io::append_to_string` function is now marked `unsafe`. - `File::read_to_string` being more efficient means that the performance note for `io::read_to_string` can be softened. I've added @camelid's suggested wording from: https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
2021-10-05 19:50:28 -04:00
pub(crate) fn default_read_to_end<R: Read + ?Sized>(r: &mut R, buf: &mut Vec<u8>) -> Result<usize> {
2021-10-21 19:35:40 -07:00
let start_len = buf.len();
let start_cap = buf.capacity();
2021-01-17 22:28:18 -08:00
2021-12-14 15:23:34 +01:00
let mut initialized = 0; // Extra initialized bytes from previous loop iteration
loop {
2021-01-17 22:28:18 -08:00
if buf.len() == buf.capacity() {
buf.reserve(32); // buf is full, need more space
}
let mut read_buf: BorrowedBuf<'_> = buf.spare_capacity_mut().into();
2021-10-20 20:57:28 -07:00
2021-12-14 15:23:34 +01:00
// SAFETY: These bytes were initialized but not filled in the previous loop
2021-01-17 22:28:18 -08:00
unsafe {
read_buf.set_init(initialized);
}
let mut cursor = read_buf.unfilled();
match r.read_buf(cursor.reborrow()) {
2021-01-17 22:28:18 -08:00
Ok(()) => {}
Err(e) if e.kind() == ErrorKind::Interrupted => continue,
2021-01-11 07:48:24 -05:00
Err(e) => return Err(e),
}
if cursor.written() == 0 {
2021-10-21 21:18:22 -07:00
return Ok(buf.len() - start_len);
2021-01-17 22:28:18 -08:00
}
// store how much was initialized but not filled
initialized = cursor.init_ref().len();
2021-10-20 20:57:28 -07:00
// SAFETY: BorrowedBuf's invariants mean this much memory is initialized.
2021-01-17 22:28:18 -08:00
unsafe {
let new_len = read_buf.filled().len() + buf.len();
2021-01-17 22:28:18 -08:00
buf.set_len(new_len);
}
2021-01-17 22:28:18 -08:00
2021-10-21 19:35:40 -07:00
if buf.len() == buf.capacity() && buf.capacity() == start_cap {
// The buffer might be an exact fit. Let's read into a probe buffer
// and see if it returns `Ok(0)`. If so, we've avoided an
// unnecessary doubling of the capacity. But if not, append the
// probe buffer to the primary buffer and let its capacity grow.
let mut probe = [0u8; 32];
loop {
match r.read(&mut probe) {
Ok(0) => return Ok(buf.len() - start_len),
Ok(n) => {
buf.extend_from_slice(&probe[..n]);
break;
}
Err(ref e) if e.kind() == ErrorKind::Interrupted => continue,
Err(e) => return Err(e),
}
}
}
}
}
Optimize File::read_to_end and read_to_string Reading a file into an empty vector or string buffer can incur unnecessary `read` syscalls and memory re-allocations as the buffer "warms up" and grows to its final size. This is perhaps a necessary evil with generic readers, but files can be read in smarter by checking the file size and reserving that much capacity. `std::fs::read` and `read_to_string` already perform this optimization: they open the file, reads its metadata, and call `with_capacity` with the file size. This ensures that the buffer does not need to be resized and an initial string of small `read` syscalls. However, if a user opens the `File` themselves and calls `file.read_to_end` or `file.read_to_string` they do not get this optimization. ```rust let mut buf = Vec::new(); file.read_to_end(&mut buf)?; ``` I searched through this project's codebase and even here are a *lot* of examples of this. They're found all over in unit tests, which isn't a big deal, but there are also several real instances in the compiler and in Cargo. I've documented the ones I found in a comment here: https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999 Most telling, the `Read` trait and the `read_to_end` method both show this exact pattern as examples of how to use readers. What this says to me is that this shouldn't be solved by simply fixing the instances of it in this codebase. If it's here it's certain to be prevalent in the wider Rust ecosystem. To that end, this commit adds specializations of `read_to_end` and `read_to_string` directly on `File`. This way it's no longer a minor footgun to start with an empty buffer when reading a file in. A nice side effect of this change is that code that accesses a `File` as a bare `Read` constraint or via a `dyn Read` trait object will benefit. For example, this code from `compiler/rustc_serialize/src/json.rs`: ```rust pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> { let mut contents = Vec::new(); match rdr.read_to_end(&mut contents) { ``` Related changes: - I also added specializations to `BufReader` to delegate to `self.inner`'s methods. That way it can call `File`'s optimized implementations if the inner reader is a file. - The private `std::io::append_to_string` function is now marked `unsafe`. - `File::read_to_string` being more efficient means that the performance note for `io::read_to_string` can be softened. I've added @camelid's suggested wording from: https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
2021-10-05 19:50:28 -04:00
pub(crate) fn default_read_to_string<R: Read + ?Sized>(
r: &mut R,
buf: &mut String,
) -> Result<usize> {
// Note that we do *not* call `r.read_to_end()` here. We are passing
// `&mut Vec<u8>` (the raw contents of `buf`) into the `read_to_end`
// method to fill it up. An arbitrary implementation could overwrite the
// entire contents of the vector, not just append to it (which is what
// we are expecting).
//
// To prevent extraneously checking the UTF-8-ness of the entire buffer
// we pass it to our hardcoded `default_read_to_end` implementation which
// we know is guaranteed to only read data into the end of the buffer.
unsafe { append_to_string(buf, |b| default_read_to_end(r, b)) }
}
pub(crate) fn default_read_vectored<F>(read: F, bufs: &mut [IoSliceMut<'_>]) -> Result<usize>
where
F: FnOnce(&mut [u8]) -> Result<usize>,
{
let buf = bufs.iter_mut().find(|b| !b.is_empty()).map_or(&mut [][..], |b| &mut **b);
read(buf)
}
pub(crate) fn default_write_vectored<F>(write: F, bufs: &[IoSlice<'_>]) -> Result<usize>
where
F: FnOnce(&[u8]) -> Result<usize>,
{
let buf = bufs.iter().find(|b| !b.is_empty()).map_or(&[][..], |b| &**b);
write(buf)
}
pub(crate) fn default_read_exact<R: Read + ?Sized>(this: &mut R, mut buf: &mut [u8]) -> Result<()> {
while !buf.is_empty() {
match this.read(buf) {
Ok(0) => break,
Ok(n) => {
let tmp = buf;
buf = &mut tmp[n..];
}
Err(ref e) if e.kind() == ErrorKind::Interrupted => {}
Err(e) => return Err(e),
}
}
if !buf.is_empty() {
Err(error::const_io_error!(ErrorKind::UnexpectedEof, "failed to fill whole buffer"))
} else {
Ok(())
}
}
pub(crate) fn default_read_buf<F>(read: F, mut cursor: BorrowedCursor<'_>) -> Result<()>
where
F: FnOnce(&mut [u8]) -> Result<usize>,
{
let n = read(cursor.ensure_init().init_mut())?;
unsafe {
// SAFETY: we initialised using `ensure_init` so there is no uninit data to advance to.
cursor.advance(n);
}
Ok(())
}
2015-07-09 16:23:27 -04:00
/// The `Read` trait allows for reading bytes from a source.
///
/// Implementors of the `Read` trait are called 'readers'.
///
2017-08-21 11:45:33 +02:00
/// Readers are defined by one required method, [`read()`]. Each call to [`read()`]
2015-07-09 16:23:27 -04:00
/// will attempt to pull bytes from this source into a provided buffer. A
2017-08-21 11:45:33 +02:00
/// number of other methods are implemented in terms of [`read()`], giving
2015-07-09 16:23:27 -04:00
/// implementors a number of ways to read bytes while only needing to implement
/// a single method.
///
/// Readers are intended to be composable with one another. Many implementors
2017-08-21 11:45:33 +02:00
/// throughout [`std::io`] take and provide types which implement the `Read`
2015-07-09 16:23:27 -04:00
/// trait.
///
2017-08-21 11:45:33 +02:00
/// Please note that each call to [`read()`] may involve a system call, and
/// therefore, using something that implements [`BufRead`], such as
/// [`BufReader`], will be more efficient.
///
2015-07-09 16:23:27 -04:00
/// # Examples
///
2017-08-21 11:45:33 +02:00
/// [`File`]s implement `Read`:
2015-07-09 16:23:27 -04:00
///
/// ```no_run
/// use std::io;
2015-07-10 13:49:12 -04:00
/// use std::io::prelude::*;
2015-07-09 16:23:27 -04:00
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let mut f = File::open("foo.txt")?;
/// let mut buffer = [0; 10];
2015-07-09 16:23:27 -04:00
///
/// // read up to 10 bytes
/// f.read(&mut buffer)?;
2015-07-09 16:23:27 -04:00
///
/// let mut buffer = Vec::new();
/// // read the whole file
/// f.read_to_end(&mut buffer)?;
2015-07-09 16:23:27 -04:00
///
/// // read into a String, so that you don't need to do the conversion.
/// let mut buffer = String::new();
/// f.read_to_string(&mut buffer)?;
2015-07-09 16:23:27 -04:00
///
/// // and more! See the other methods for more details.
/// Ok(())
/// }
2015-07-09 16:23:27 -04:00
/// ```
///
/// Read from [`&str`] because [`&[u8]`][prim@slice] implements `Read`:
///
/// ```no_run
/// # use std::io;
/// use std::io::prelude::*;
///
/// fn main() -> io::Result<()> {
/// let mut b = "This string will be read".as_bytes();
/// let mut buffer = [0; 10];
///
/// // read up to 10 bytes
/// b.read(&mut buffer)?;
///
/// // etc... it works exactly as a File does!
/// Ok(())
/// }
/// ```
///
2020-08-19 16:26:17 +02:00
/// [`read()`]: Read::read
2020-08-22 16:29:55 -04:00
/// [`&str`]: prim@str
2020-07-18 12:02:03 -07:00
/// [`std::io`]: self
/// [`File`]: crate::fs::File
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
#[stable(feature = "rust1", since = "1.0.0")]
2021-05-06 13:36:07 +02:00
#[doc(notable_trait)]
#[cfg_attr(not(test), rustc_diagnostic_item = "IoRead")]
pub trait Read {
/// Pull some bytes from this source into the specified buffer, returning
/// how many bytes were read.
///
/// This function does not provide any guarantees about whether it blocks
/// waiting for data, but if an object needs to block for a read and cannot,
2017-08-21 11:45:33 +02:00
/// it will typically signal this via an [`Err`] return value.
///
/// If the return value of this method is [`Ok(n)`], then implementations must
/// guarantee that `0 <= n <= buf.len()`. A nonzero `n` value indicates
2015-03-25 18:02:11 +01:00
/// that the buffer `buf` has been filled in with `n` bytes of data from this
/// source. If `n` is `0`, then it can indicate one of two scenarios:
///
/// 1. This reader has reached its "end of file" and will likely no longer
/// be able to produce bytes. Note that this does not mean that the
/// reader will *always* no longer be able to produce bytes. As an example,
/// on Linux, this method will call the `recv` syscall for a [`TcpStream`],
/// where returning zero indicates the connection was shut down correctly. While
/// for [`File`], it is possible to reach the end of file and get zero as result,
/// but if more data is appended to the file, future calls to `read` will return
/// more data.
/// 2. The buffer specified was 0 bytes in length.
///
/// It is not an error if the returned value `n` is smaller than the buffer size,
/// even when the reader is not at the end of the stream yet.
/// This may happen for example because fewer bytes are actually available right now
/// (e. g. being close to end-of-file) or because read() was interrupted by a signal.
///
/// As this trait is safe to implement, callers cannot rely on `n <= buf.len()` for safety.
/// Extra care needs to be taken when `unsafe` functions are used to access the read bytes.
/// Callers have to ensure that no unchecked out-of-bounds accesses are possible even if
/// `n > buf.len()`.
///
/// No guarantees are provided about the contents of `buf` when this
/// function is called, implementations cannot rely on any property of the
/// contents of `buf` being true. It is recommended that *implementations*
/// only write data to `buf` instead of reading its contents.
///
/// Correspondingly, however, *callers* of this method must not assume any guarantees
/// about how the implementation uses `buf`. The trait is safe to implement,
2019-07-21 12:47:34 +02:00
/// so it is possible that the code that's supposed to write to the buffer might also read
/// from it. It is your responsibility to make sure that `buf` is initialized
/// before calling `read`. Calling `read` with an uninitialized `buf` (of the kind one
/// obtains via [`MaybeUninit<T>`]) is not safe, and can lead to undefined behavior.
///
2020-07-18 12:02:03 -07:00
/// [`MaybeUninit<T>`]: crate::mem::MaybeUninit
///
/// # Errors
///
/// If this function encounters any form of I/O or other error, an error
/// variant will be returned. If an error is returned then it must be
/// guaranteed that no bytes were read.
2015-07-09 16:23:27 -04:00
///
2017-08-21 11:45:33 +02:00
/// An error of the [`ErrorKind::Interrupted`] kind is non-fatal and the read
/// operation should be retried if there is nothing else to do.
///
2015-07-09 16:23:27 -04:00
/// # Examples
///
2017-08-21 11:45:33 +02:00
/// [`File`]s implement `Read`:
2015-07-09 16:23:27 -04:00
///
2020-07-18 12:02:03 -07:00
/// [`Ok(n)`]: Ok
/// [`File`]: crate::fs::File
/// [`TcpStream`]: crate::net::TcpStream
2015-07-09 16:23:27 -04:00
///
/// ```no_run
2015-07-09 16:23:27 -04:00
/// use std::io;
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let mut f = File::open("foo.txt")?;
/// let mut buffer = [0; 10];
2015-07-09 16:23:27 -04:00
///
/// // read up to 10 bytes
/// let n = f.read(&mut buffer[..])?;
///
/// println!("The bytes: {:?}", &buffer[..n]);
/// Ok(())
/// }
2015-07-09 16:23:27 -04:00
/// ```
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
#[stable(feature = "rust1", since = "1.0.0")]
fn read(&mut self, buf: &mut [u8]) -> Result<usize>;
/// Like `read`, except that it reads into a slice of buffers.
///
/// Data is copied to fill each buffer in order, with the final buffer
2020-04-18 14:30:16 -04:00
/// written to possibly being only partially filled. This method must
/// behave equivalently to a single call to `read` with concatenated
/// buffers.
///
/// The default implementation calls `read` with either the first nonempty
/// buffer provided, or an empty one if none exists.
#[stable(feature = "iovec", since = "1.36.0")]
fn read_vectored(&mut self, bufs: &mut [IoSliceMut<'_>]) -> Result<usize> {
default_read_vectored(|b| self.read(b), bufs)
}
/// Determines if this `Read`er has an efficient `read_vectored`
/// implementation.
///
/// If a `Read`er does not override the default `read_vectored`
/// implementation, code using it may want to avoid the method all together
/// and coalesce writes into a single buffer for higher performance.
///
/// The default implementation returns `false`.
2020-03-11 18:02:52 -07:00
#[unstable(feature = "can_vector", issue = "69941")]
fn is_read_vectored(&self) -> bool {
false
}
/// Read all bytes until EOF in this source, placing them into `buf`.
///
/// All bytes read from this source will be appended to the specified buffer
2017-08-21 11:45:33 +02:00
/// `buf`. This function will continuously call [`read()`] to append more data to
/// `buf` until [`read()`] returns either [`Ok(0)`] or an error of
/// non-[`ErrorKind::Interrupted`] kind.
///
2015-04-13 22:21:51 +03:00
/// If successful, this function will return the total number of bytes read.
///
/// # Errors
///
/// If this function encounters an error of the kind
2017-08-21 11:45:33 +02:00
/// [`ErrorKind::Interrupted`] then the error is ignored and the operation
/// will continue.
///
/// If any other read error is encountered then this function immediately
/// returns. Any bytes which have already been read will be appended to
/// `buf`.
2015-07-09 16:23:27 -04:00
///
/// # Examples
///
2017-08-21 11:45:33 +02:00
/// [`File`]s implement `Read`:
2015-07-09 16:23:27 -04:00
///
2020-08-19 16:26:17 +02:00
/// [`read()`]: Read::read
2020-07-18 12:02:03 -07:00
/// [`Ok(0)`]: Ok
/// [`File`]: crate::fs::File
2015-07-09 16:23:27 -04:00
///
/// ```no_run
2015-07-09 16:23:27 -04:00
/// use std::io;
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let mut f = File::open("foo.txt")?;
/// let mut buffer = Vec::new();
2015-07-09 16:23:27 -04:00
///
/// // read the whole file
/// f.read_to_end(&mut buffer)?;
/// Ok(())
/// }
2015-07-09 16:23:27 -04:00
/// ```
///
/// (See also the [`std::fs::read`] convenience function for reading from a
/// file.)
///
2020-07-18 12:02:03 -07:00
/// [`std::fs::read`]: crate::fs::read
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
#[stable(feature = "rust1", since = "1.0.0")]
fn read_to_end(&mut self, buf: &mut Vec<u8>) -> Result<usize> {
Optimize File::read_to_end and read_to_string Reading a file into an empty vector or string buffer can incur unnecessary `read` syscalls and memory re-allocations as the buffer "warms up" and grows to its final size. This is perhaps a necessary evil with generic readers, but files can be read in smarter by checking the file size and reserving that much capacity. `std::fs::read` and `read_to_string` already perform this optimization: they open the file, reads its metadata, and call `with_capacity` with the file size. This ensures that the buffer does not need to be resized and an initial string of small `read` syscalls. However, if a user opens the `File` themselves and calls `file.read_to_end` or `file.read_to_string` they do not get this optimization. ```rust let mut buf = Vec::new(); file.read_to_end(&mut buf)?; ``` I searched through this project's codebase and even here are a *lot* of examples of this. They're found all over in unit tests, which isn't a big deal, but there are also several real instances in the compiler and in Cargo. I've documented the ones I found in a comment here: https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999 Most telling, the `Read` trait and the `read_to_end` method both show this exact pattern as examples of how to use readers. What this says to me is that this shouldn't be solved by simply fixing the instances of it in this codebase. If it's here it's certain to be prevalent in the wider Rust ecosystem. To that end, this commit adds specializations of `read_to_end` and `read_to_string` directly on `File`. This way it's no longer a minor footgun to start with an empty buffer when reading a file in. A nice side effect of this change is that code that accesses a `File` as a bare `Read` constraint or via a `dyn Read` trait object will benefit. For example, this code from `compiler/rustc_serialize/src/json.rs`: ```rust pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> { let mut contents = Vec::new(); match rdr.read_to_end(&mut contents) { ``` Related changes: - I also added specializations to `BufReader` to delegate to `self.inner`'s methods. That way it can call `File`'s optimized implementations if the inner reader is a file. - The private `std::io::append_to_string` function is now marked `unsafe`. - `File::read_to_string` being more efficient means that the performance note for `io::read_to_string` can be softened. I've added @camelid's suggested wording from: https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
2021-10-05 19:50:28 -04:00
default_read_to_end(self, buf)
}
/// Read all bytes until EOF in this source, appending them to `buf`.
///
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
/// If successful, this function returns the number of bytes which were read
/// and appended to `buf`.
///
/// # Errors
///
/// If the data in this stream is *not* valid UTF-8 then an error is
/// returned and `buf` is unchanged.
///
2020-08-18 19:36:52 +02:00
/// See [`read_to_end`] for other error semantics.
2015-07-09 16:23:27 -04:00
///
2020-08-19 16:26:17 +02:00
/// [`read_to_end`]: Read::read_to_end
2015-07-09 16:23:27 -04:00
///
/// # Examples
///
2020-08-18 19:36:52 +02:00
/// [`File`]s implement `Read`:
2015-07-09 16:23:27 -04:00
///
2020-08-18 19:36:52 +02:00
/// [`File`]: crate::fs::File
2015-07-09 16:23:27 -04:00
///
/// ```no_run
2015-07-09 16:23:27 -04:00
/// use std::io;
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let mut f = File::open("foo.txt")?;
/// let mut buffer = String::new();
2015-07-09 16:23:27 -04:00
///
/// f.read_to_string(&mut buffer)?;
/// Ok(())
/// }
2015-07-09 16:23:27 -04:00
/// ```
///
/// (See also the [`std::fs::read_to_string`] convenience function for
/// reading from a file.)
///
2020-07-18 12:02:03 -07:00
/// [`std::fs::read_to_string`]: crate::fs::read_to_string
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
#[stable(feature = "rust1", since = "1.0.0")]
fn read_to_string(&mut self, buf: &mut String) -> Result<usize> {
Optimize File::read_to_end and read_to_string Reading a file into an empty vector or string buffer can incur unnecessary `read` syscalls and memory re-allocations as the buffer "warms up" and grows to its final size. This is perhaps a necessary evil with generic readers, but files can be read in smarter by checking the file size and reserving that much capacity. `std::fs::read` and `read_to_string` already perform this optimization: they open the file, reads its metadata, and call `with_capacity` with the file size. This ensures that the buffer does not need to be resized and an initial string of small `read` syscalls. However, if a user opens the `File` themselves and calls `file.read_to_end` or `file.read_to_string` they do not get this optimization. ```rust let mut buf = Vec::new(); file.read_to_end(&mut buf)?; ``` I searched through this project's codebase and even here are a *lot* of examples of this. They're found all over in unit tests, which isn't a big deal, but there are also several real instances in the compiler and in Cargo. I've documented the ones I found in a comment here: https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999 Most telling, the `Read` trait and the `read_to_end` method both show this exact pattern as examples of how to use readers. What this says to me is that this shouldn't be solved by simply fixing the instances of it in this codebase. If it's here it's certain to be prevalent in the wider Rust ecosystem. To that end, this commit adds specializations of `read_to_end` and `read_to_string` directly on `File`. This way it's no longer a minor footgun to start with an empty buffer when reading a file in. A nice side effect of this change is that code that accesses a `File` as a bare `Read` constraint or via a `dyn Read` trait object will benefit. For example, this code from `compiler/rustc_serialize/src/json.rs`: ```rust pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> { let mut contents = Vec::new(); match rdr.read_to_end(&mut contents) { ``` Related changes: - I also added specializations to `BufReader` to delegate to `self.inner`'s methods. That way it can call `File`'s optimized implementations if the inner reader is a file. - The private `std::io::append_to_string` function is now marked `unsafe`. - `File::read_to_string` being more efficient means that the performance note for `io::read_to_string` can be softened. I've added @camelid's suggested wording from: https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
2021-10-05 19:50:28 -04:00
default_read_to_string(self, buf)
}
/// Read the exact number of bytes required to fill `buf`.
///
/// This function reads as many bytes as necessary to completely fill the
/// specified buffer `buf`.
///
/// No guarantees are provided about the contents of `buf` when this
/// function is called, implementations cannot rely on any property of the
/// contents of `buf` being true. It is recommended that implementations
/// only write data to `buf` instead of reading its contents. The
/// documentation on [`read`] has a more detailed explanation on this
/// subject.
///
/// # Errors
///
/// If this function encounters an error of the kind
2017-08-21 11:45:33 +02:00
/// [`ErrorKind::Interrupted`] then the error is ignored and the operation
/// will continue.
///
/// If this function encounters an "end of file" before completely filling
2017-08-21 11:45:33 +02:00
/// the buffer, it returns an error of the kind [`ErrorKind::UnexpectedEof`].
/// The contents of `buf` are unspecified in this case.
///
/// If any other read error is encountered then this function immediately
/// returns. The contents of `buf` are unspecified in this case.
///
/// If this function returns an error, it is unspecified how many bytes it
/// has read, but it will never read more than would be necessary to
/// completely fill the buffer.
///
/// # Examples
///
2017-08-21 11:45:33 +02:00
/// [`File`]s implement `Read`:
///
2020-08-19 16:26:17 +02:00
/// [`read`]: Read::read
2020-07-18 12:02:03 -07:00
/// [`File`]: crate::fs::File
///
/// ```no_run
/// use std::io;
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let mut f = File::open("foo.txt")?;
/// let mut buffer = [0; 10];
///
/// // read exactly 10 bytes
/// f.read_exact(&mut buffer)?;
/// Ok(())
/// }
/// ```
std: Stabilize APIs for the 1.6 release This commit is the standard API stabilization commit for the 1.6 release cycle. The list of issues and APIs below have all been through their cycle-long FCP and the libs team decisions are listed below Stabilized APIs * `Read::read_exact` * `ErrorKind::UnexpectedEof` (renamed from `UnexpectedEOF`) * libcore -- this was a bit of a nuanced stabilization, the crate itself is now marked as `#[stable]` and the methods appearing via traits for primitives like `char` and `str` are now also marked as stable. Note that the extension traits themeselves are marked as unstable as they're imported via the prelude. The `try!` macro was also moved from the standard library into libcore to have the same interface. Otherwise the functions all have copied stability from the standard library now. * The `#![no_std]` attribute * `fs::DirBuilder` * `fs::DirBuilder::new` * `fs::DirBuilder::recursive` * `fs::DirBuilder::create` * `os::unix::fs::DirBuilderExt` * `os::unix::fs::DirBuilderExt::mode` * `vec::Drain` * `vec::Vec::drain` * `string::Drain` * `string::String::drain` * `vec_deque::Drain` * `vec_deque::VecDeque::drain` * `collections::hash_map::Drain` * `collections::hash_map::HashMap::drain` * `collections::hash_set::Drain` * `collections::hash_set::HashSet::drain` * `collections::binary_heap::Drain` * `collections::binary_heap::BinaryHeap::drain` * `Vec::extend_from_slice` (renamed from `push_all`) * `Mutex::get_mut` * `Mutex::into_inner` * `RwLock::get_mut` * `RwLock::into_inner` * `Iterator::min_by_key` (renamed from `min_by`) * `Iterator::max_by_key` (renamed from `max_by`) Deprecated APIs * `ErrorKind::UnexpectedEOF` (renamed to `UnexpectedEof`) * `OsString::from_bytes` * `OsStr::to_cstring` * `OsStr::to_bytes` * `fs::walk_dir` and `fs::WalkDir` * `path::Components::peek` * `slice::bytes::MutableByteVector` * `slice::bytes::copy_memory` * `Vec::push_all` (renamed to `extend_from_slice`) * `Duration::span` * `IpAddr` * `SocketAddr::ip` * `Read::tee` * `io::Tee` * `Write::broadcast` * `io::Broadcast` * `Iterator::min_by` (renamed to `min_by_key`) * `Iterator::max_by` (renamed to `max_by_key`) * `net::lookup_addr` New APIs (still unstable) * `<[T]>::sort_by_key` (added to mirror `min_by_key`) Closes #27585 Closes #27704 Closes #27707 Closes #27710 Closes #27711 Closes #27727 Closes #27740 Closes #27744 Closes #27799 Closes #27801 cc #27801 (doesn't close as `Chars` is still unstable) Closes #28968
2015-12-02 17:31:49 -08:00
#[stable(feature = "read_exact", since = "1.6.0")]
fn read_exact(&mut self, buf: &mut [u8]) -> Result<()> {
default_read_exact(self, buf)
}
2021-01-17 22:28:18 -08:00
/// Pull some bytes from this source into the specified buffer.
///
/// This is equivalent to the [`read`](Read::read) method, except that it is passed a [`BorrowedCursor`] rather than `[u8]` to allow use
2021-01-17 22:28:18 -08:00
/// with uninitialized buffers. The new data will be appended to any existing contents of `buf`.
///
/// The default implementation delegates to `read`.
#[unstable(feature = "read_buf", issue = "78485")]
fn read_buf(&mut self, buf: BorrowedCursor<'_>) -> Result<()> {
default_read_buf(|b| self.read(b), buf)
2021-01-17 22:28:18 -08:00
}
/// Read the exact number of bytes required to fill `cursor`.
2021-01-17 22:28:18 -08:00
///
/// This is equivalent to the [`read_exact`](Read::read_exact) method, except that it is passed a [`BorrowedCursor`] rather than `[u8]` to
2021-01-17 22:28:18 -08:00
/// allow use with uninitialized buffers.
#[unstable(feature = "read_buf", issue = "78485")]
fn read_buf_exact(&mut self, mut cursor: BorrowedCursor<'_>) -> Result<()> {
while cursor.capacity() > 0 {
let prev_written = cursor.written();
match self.read_buf(cursor.reborrow()) {
2021-01-17 22:28:18 -08:00
Ok(()) => {}
Err(e) if e.kind() == ErrorKind::Interrupted => continue,
Err(e) => return Err(e),
}
if cursor.written() == prev_written {
2021-01-17 22:28:18 -08:00
return Err(Error::new(ErrorKind::UnexpectedEof, "failed to fill buffer"));
}
}
Ok(())
}
/// Creates a "by reference" adaptor for this instance of `Read`.
///
/// The returned adapter also implements `Read` and will simply borrow this
/// current reader.
2015-07-09 16:23:27 -04:00
///
/// # Examples
///
2020-08-18 19:36:52 +02:00
/// [`File`]s implement `Read`:
2015-07-09 16:23:27 -04:00
///
2020-08-18 19:36:52 +02:00
/// [`File`]: crate::fs::File
2015-07-09 16:23:27 -04:00
///
/// ```no_run
2015-07-09 16:23:27 -04:00
/// use std::io;
/// use std::io::Read;
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let mut f = File::open("foo.txt")?;
/// let mut buffer = Vec::new();
/// let mut other_buffer = Vec::new();
2015-07-09 16:23:27 -04:00
///
/// {
/// let reference = f.by_ref();
2015-07-09 16:23:27 -04:00
///
/// // read at most 5 bytes
/// reference.take(5).read_to_end(&mut buffer)?;
2015-07-09 16:23:27 -04:00
///
/// } // drop our &mut reference so we can use f again
2015-07-09 16:23:27 -04:00
///
/// // original file still usable, read the rest
/// f.read_to_end(&mut other_buffer)?;
/// Ok(())
/// }
2015-07-09 16:23:27 -04:00
/// ```
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
fn by_ref(&mut self) -> &mut Self
where
Self: Sized,
{
self
}
2017-08-21 11:45:33 +02:00
/// Transforms this `Read` instance to an [`Iterator`] over its bytes.
///
Apply 16 commits (squashed) ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::fmt ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::{rc, sync} ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::string ---------- Fix spacing for links inside code blocks in alloc::vec ---------- Fix spacing for links inside code blocks in core::option ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in core::result ---------- Fix spacing for links inside code blocks in core::{iter::{self, iterator}, stream::stream, poll} ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in std::{fs, path} ---------- Fix spacing for links inside code blocks in std::{collections, time} ---------- Fix spacing for links inside code blocks in and make formatting of `&str`-like types consistent in std::ffi::{c_str, os_str} ---------- Fix spacing for links inside code blocks, and improve link tooltips in std::ffi ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in std::{io::{self, buffered::{bufreader, bufwriter}, cursor, util}, net::{self, addr}} ---------- Fix typo in link to `into` for `OsString` docs ---------- Remove tooltips that will probably become redundant in the future ---------- Apply suggestions from code review Replacing `…std/primitive.reference.html` paths with just `reference` Co-authored-by: Joshua Nelson <github@jyn.dev> ---------- Also replace `…std/primitive.reference.html` paths with just `reference` in `core::pin`
2021-08-25 11:45:08 +02:00
/// The returned type implements [`Iterator`] where the [`Item`] is
/// <code>[Result]<[u8], [io::Error]></code>.
/// The yielded item is [`Ok`] if a byte was successfully read and [`Err`]
/// otherwise. EOF is mapped to returning [`None`] from this iterator.
2015-07-09 16:23:27 -04:00
///
/// The default implementation calls `read` for each byte,
/// which can be very inefficient for data that's not in memory,
/// such as [`File`]. Consider using a [`BufReader`] in such cases.
///
2015-07-09 16:23:27 -04:00
/// # Examples
///
2020-08-18 19:36:52 +02:00
/// [`File`]s implement `Read`:
2015-07-09 16:23:27 -04:00
///
Apply 16 commits (squashed) ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::fmt ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::{rc, sync} ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::string ---------- Fix spacing for links inside code blocks in alloc::vec ---------- Fix spacing for links inside code blocks in core::option ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in core::result ---------- Fix spacing for links inside code blocks in core::{iter::{self, iterator}, stream::stream, poll} ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in std::{fs, path} ---------- Fix spacing for links inside code blocks in std::{collections, time} ---------- Fix spacing for links inside code blocks in and make formatting of `&str`-like types consistent in std::ffi::{c_str, os_str} ---------- Fix spacing for links inside code blocks, and improve link tooltips in std::ffi ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in std::{io::{self, buffered::{bufreader, bufwriter}, cursor, util}, net::{self, addr}} ---------- Fix typo in link to `into` for `OsString` docs ---------- Remove tooltips that will probably become redundant in the future ---------- Apply suggestions from code review Replacing `…std/primitive.reference.html` paths with just `reference` Co-authored-by: Joshua Nelson <github@jyn.dev> ---------- Also replace `…std/primitive.reference.html` paths with just `reference` in `core::pin`
2021-08-25 11:45:08 +02:00
/// [`Item`]: Iterator::Item
/// [`File`]: crate::fs::File "fs::File"
/// [Result]: crate::result::Result "Result"
/// [io::Error]: self::Error "io::Error"
2015-07-09 16:23:27 -04:00
///
/// ```no_run
2015-07-09 16:23:27 -04:00
/// use std::io;
/// use std::io::prelude::*;
/// use std::io::BufReader;
2015-07-09 16:23:27 -04:00
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let f = BufReader::new(File::open("foo.txt")?);
2015-07-09 16:23:27 -04:00
///
/// for byte in f.bytes() {
/// println!("{}", byte.unwrap());
/// }
/// Ok(())
2015-07-09 16:23:27 -04:00
/// }
/// ```
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
fn bytes(self) -> Bytes<Self>
where
Self: Sized,
{
Bytes { inner: self }
}
/// Creates an adapter which will chain this stream with another.
///
/// The returned `Read` instance will first read all bytes from this object
/// until EOF is encountered. Afterwards the output is equivalent to the
/// output of `next`.
2015-07-09 16:23:27 -04:00
///
/// # Examples
///
2020-08-18 19:36:52 +02:00
/// [`File`]s implement `Read`:
2015-07-09 16:23:27 -04:00
///
2020-08-18 19:36:52 +02:00
/// [`File`]: crate::fs::File
2015-07-09 16:23:27 -04:00
///
/// ```no_run
2015-07-09 16:23:27 -04:00
/// use std::io;
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
2022-07-25 17:04:51 +08:00
/// let f1 = File::open("foo.txt")?;
/// let f2 = File::open("bar.txt")?;
2015-07-09 16:23:27 -04:00
///
/// let mut handle = f1.chain(f2);
/// let mut buffer = String::new();
2015-07-09 16:23:27 -04:00
///
/// // read the value into a String. We could use any Read method here,
/// // this is just one example.
/// handle.read_to_string(&mut buffer)?;
/// Ok(())
/// }
2015-07-09 16:23:27 -04:00
/// ```
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
fn chain<R: Read>(self, next: R) -> Chain<Self, R>
where
Self: Sized,
{
Chain { first: self, second: next, done_first: false }
}
/// Creates an adapter which will read at most `limit` bytes from it.
///
/// This function returns a new instance of `Read` which will read at most
2017-08-21 11:45:33 +02:00
/// `limit` bytes, after which it will always return EOF ([`Ok(0)`]). Any
/// read errors will not count towards the number of bytes read and future
2017-08-21 11:45:33 +02:00
/// calls to [`read()`] may succeed.
2015-07-09 16:23:27 -04:00
///
/// # Examples
///
2017-08-21 11:45:33 +02:00
/// [`File`]s implement `Read`:
2015-07-09 16:23:27 -04:00
///
2020-07-18 12:02:03 -07:00
/// [`File`]: crate::fs::File
/// [`Ok(0)`]: Ok
/// [`read()`]: Read::read
2015-07-09 16:23:27 -04:00
///
/// ```no_run
2015-07-09 16:23:27 -04:00
/// use std::io;
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
2022-07-25 17:04:51 +08:00
/// let f = File::open("foo.txt")?;
/// let mut buffer = [0; 5];
2015-07-09 16:23:27 -04:00
///
/// // read at most five bytes
/// let mut handle = f.take(5);
2015-07-09 16:23:27 -04:00
///
/// handle.read(&mut buffer)?;
/// Ok(())
/// }
2015-07-09 16:23:27 -04:00
/// ```
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
fn take(self, limit: u64) -> Take<Self>
where
Self: Sized,
{
Take { inner: self, limit }
}
}
/// Read all bytes from a [reader][Read] into a new [`String`].
///
/// This is a convenience function for [`Read::read_to_string`]. Using this
/// function avoids having to create a variable first and provides more type
/// safety since you can only get the buffer out if there were no errors. (If you
/// use [`Read::read_to_string`] you have to remember to check whether the read
2020-12-30 11:44:03 -08:00
/// succeeded because otherwise your buffer will be empty or only partially full.)
///
2021-01-11 19:17:13 -08:00
/// # Performance
///
/// The downside of this function's increased ease of use and type safety is
/// that it gives you less control over performance. For example, you can't
/// pre-allocate memory like you can using [`String::with_capacity`] and
/// [`Read::read_to_string`]. Also, you can't re-use the buffer if an error
/// occurs while reading.
///
/// In many cases, this function's performance will be adequate and the ease of use
/// and type safety tradeoffs will be worth it. However, there are cases where you
/// need more control over performance, and in those cases you should definitely use
/// [`Read::read_to_string`] directly.
///
Optimize File::read_to_end and read_to_string Reading a file into an empty vector or string buffer can incur unnecessary `read` syscalls and memory re-allocations as the buffer "warms up" and grows to its final size. This is perhaps a necessary evil with generic readers, but files can be read in smarter by checking the file size and reserving that much capacity. `std::fs::read` and `read_to_string` already perform this optimization: they open the file, reads its metadata, and call `with_capacity` with the file size. This ensures that the buffer does not need to be resized and an initial string of small `read` syscalls. However, if a user opens the `File` themselves and calls `file.read_to_end` or `file.read_to_string` they do not get this optimization. ```rust let mut buf = Vec::new(); file.read_to_end(&mut buf)?; ``` I searched through this project's codebase and even here are a *lot* of examples of this. They're found all over in unit tests, which isn't a big deal, but there are also several real instances in the compiler and in Cargo. I've documented the ones I found in a comment here: https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999 Most telling, the `Read` trait and the `read_to_end` method both show this exact pattern as examples of how to use readers. What this says to me is that this shouldn't be solved by simply fixing the instances of it in this codebase. If it's here it's certain to be prevalent in the wider Rust ecosystem. To that end, this commit adds specializations of `read_to_end` and `read_to_string` directly on `File`. This way it's no longer a minor footgun to start with an empty buffer when reading a file in. A nice side effect of this change is that code that accesses a `File` as a bare `Read` constraint or via a `dyn Read` trait object will benefit. For example, this code from `compiler/rustc_serialize/src/json.rs`: ```rust pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> { let mut contents = Vec::new(); match rdr.read_to_end(&mut contents) { ``` Related changes: - I also added specializations to `BufReader` to delegate to `self.inner`'s methods. That way it can call `File`'s optimized implementations if the inner reader is a file. - The private `std::io::append_to_string` function is now marked `unsafe`. - `File::read_to_string` being more efficient means that the performance note for `io::read_to_string` can be softened. I've added @camelid's suggested wording from: https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
2021-10-05 19:50:28 -04:00
/// Note that in some special cases, such as when reading files, this function will
/// pre-allocate memory based on the size of the input it is reading. In those
/// cases, the performance should be as good as if you had used
/// [`Read::read_to_string`] with a manually pre-allocated buffer.
///
2020-12-30 11:44:03 -08:00
/// # Errors
///
/// This function forces you to handle errors because the output (the `String`)
/// is wrapped in a [`Result`]. See [`Read::read_to_string`] for the errors
/// that can occur. If any error occurs, you will get an [`Err`], so you
/// don't have to worry about your buffer being empty or partially full.
///
/// # Examples
///
/// ```no_run
/// # use std::io;
/// fn main() -> io::Result<()> {
/// let stdin = io::read_to_string(io::stdin())?;
/// println!("Stdin was:");
/// println!("{stdin}");
/// Ok(())
/// }
/// ```
2022-09-19 14:54:16 +02:00
#[stable(feature = "io_read_to_string", since = "1.65.0")]
pub fn read_to_string<R: Read>(mut reader: R) -> Result<String> {
let mut buf = String::new();
reader.read_to_string(&mut buf)?;
Ok(buf)
}
/// A buffer type used with `Read::read_vectored`.
///
/// It is semantically a wrapper around an `&mut [u8]`, but is guaranteed to be
/// ABI compatible with the `iovec` type on Unix platforms and `WSABUF` on
/// Windows.
#[stable(feature = "iovec", since = "1.36.0")]
#[repr(transparent)]
pub struct IoSliceMut<'a>(sys::io::IoSliceMut<'a>);
#[stable(feature = "iovec-send-sync", since = "1.44.0")]
unsafe impl<'a> Send for IoSliceMut<'a> {}
#[stable(feature = "iovec-send-sync", since = "1.44.0")]
unsafe impl<'a> Sync for IoSliceMut<'a> {}
#[stable(feature = "iovec", since = "1.36.0")]
impl<'a> fmt::Debug for IoSliceMut<'a> {
fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
fmt::Debug::fmt(self.0.as_slice(), fmt)
}
}
impl<'a> IoSliceMut<'a> {
/// Creates a new `IoSliceMut` wrapping a byte slice.
///
/// # Panics
///
/// Panics on Windows if the slice is larger than 4GB.
#[stable(feature = "iovec", since = "1.36.0")]
#[inline]
pub fn new(buf: &'a mut [u8]) -> IoSliceMut<'a> {
IoSliceMut(sys::io::IoSliceMut::new(buf))
}
2019-07-25 22:30:52 +02:00
/// Advance the internal cursor of the slice.
///
/// Also see [`IoSliceMut::advance_slices`] to advance the cursors of
/// multiple buffers.
///
/// # Panics
///
/// Panics when trying to advance beyond the end of the slice.
///
/// # Examples
///
/// ```
/// #![feature(io_slice_advance)]
///
/// use std::io::IoSliceMut;
/// use std::ops::Deref;
///
/// let mut data = [1; 8];
/// let mut buf = IoSliceMut::new(&mut data);
///
2021-06-17 12:13:06 +02:00
/// // Mark 3 bytes as read.
/// buf.advance(3);
/// assert_eq!(buf.deref(), [1; 5].as_ref());
/// ```
#[unstable(feature = "io_slice_advance", issue = "62726")]
#[inline]
pub fn advance(&mut self, n: usize) {
self.0.advance(n)
}
/// Advance a slice of slices.
///
/// Shrinks the slice to remove any `IoSliceMut`s that are fully advanced over.
/// If the cursor ends up in the middle of an `IoSliceMut`, it is modified
/// to start at that cursor.
///
/// For example, if we have a slice of two 8-byte `IoSliceMut`s, and we advance by 10 bytes,
/// the result will only include the second `IoSliceMut`, advanced by 2 bytes.
///
/// # Panics
2019-07-25 22:30:52 +02:00
///
/// Panics when trying to advance beyond the end of the slices.
2019-07-25 22:30:52 +02:00
///
/// # Examples
///
/// ```
/// #![feature(io_slice_advance)]
///
/// use std::io::IoSliceMut;
/// use std::ops::Deref;
///
/// let mut buf1 = [1; 8];
/// let mut buf2 = [2; 16];
/// let mut buf3 = [3; 8];
/// let mut bufs = &mut [
/// IoSliceMut::new(&mut buf1),
/// IoSliceMut::new(&mut buf2),
/// IoSliceMut::new(&mut buf3),
/// ][..];
///
/// // Mark 10 bytes as read.
/// IoSliceMut::advance_slices(&mut bufs, 10);
2019-07-25 22:30:52 +02:00
/// assert_eq!(bufs[0].deref(), [2; 14].as_ref());
/// assert_eq!(bufs[1].deref(), [3; 8].as_ref());
/// ```
#[unstable(feature = "io_slice_advance", issue = "62726")]
#[inline]
pub fn advance_slices(bufs: &mut &mut [IoSliceMut<'a>], n: usize) {
2019-07-25 22:30:52 +02:00
// Number of buffers to remove.
let mut remove = 0;
// Total length of all the to be removed buffers.
let mut accumulated_len = 0;
for buf in bufs.iter() {
if accumulated_len + buf.len() > n {
break;
} else {
accumulated_len += buf.len();
remove += 1;
}
}
*bufs = &mut replace(bufs, &mut [])[remove..];
if bufs.is_empty() {
assert!(n == accumulated_len, "advancing io slices beyond their length");
} else {
bufs[0].advance(n - accumulated_len)
2019-07-25 22:30:52 +02:00
}
}
}
#[stable(feature = "iovec", since = "1.36.0")]
impl<'a> Deref for IoSliceMut<'a> {
type Target = [u8];
#[inline]
fn deref(&self) -> &[u8] {
self.0.as_slice()
}
}
#[stable(feature = "iovec", since = "1.36.0")]
impl<'a> DerefMut for IoSliceMut<'a> {
#[inline]
fn deref_mut(&mut self) -> &mut [u8] {
self.0.as_mut_slice()
}
}
/// A buffer type used with `Write::write_vectored`.
///
/// It is semantically a wrapper around a `&[u8]`, but is guaranteed to be
/// ABI compatible with the `iovec` type on Unix platforms and `WSABUF` on
/// Windows.
#[stable(feature = "iovec", since = "1.36.0")]
2020-02-23 18:18:45 +01:00
#[derive(Copy, Clone)]
#[repr(transparent)]
pub struct IoSlice<'a>(sys::io::IoSlice<'a>);
2020-03-24 00:34:48 +01:00
#[stable(feature = "iovec-send-sync", since = "1.44.0")]
unsafe impl<'a> Send for IoSlice<'a> {}
2020-03-24 00:34:48 +01:00
#[stable(feature = "iovec-send-sync", since = "1.44.0")]
unsafe impl<'a> Sync for IoSlice<'a> {}
#[stable(feature = "iovec", since = "1.36.0")]
impl<'a> fmt::Debug for IoSlice<'a> {
fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
fmt::Debug::fmt(self.0.as_slice(), fmt)
}
}
impl<'a> IoSlice<'a> {
/// Creates a new `IoSlice` wrapping a byte slice.
///
/// # Panics
///
/// Panics on Windows if the slice is larger than 4GB.
#[stable(feature = "iovec", since = "1.36.0")]
#[must_use]
#[inline]
pub fn new(buf: &'a [u8]) -> IoSlice<'a> {
IoSlice(sys::io::IoSlice::new(buf))
}
2019-07-25 22:30:52 +02:00
/// Advance the internal cursor of the slice.
///
/// Also see [`IoSlice::advance_slices`] to advance the cursors of multiple
/// buffers.
///
/// # Panics
///
/// Panics when trying to advance beyond the end of the slice.
///
/// # Examples
///
/// ```
/// #![feature(io_slice_advance)]
///
/// use std::io::IoSlice;
/// use std::ops::Deref;
///
/// let data = [1; 8];
/// let mut buf = IoSlice::new(&data);
///
2021-06-17 12:13:06 +02:00
/// // Mark 3 bytes as read.
/// buf.advance(3);
/// assert_eq!(buf.deref(), [1; 5].as_ref());
/// ```
#[unstable(feature = "io_slice_advance", issue = "62726")]
#[inline]
pub fn advance(&mut self, n: usize) {
self.0.advance(n)
}
/// Advance a slice of slices.
///
/// Shrinks the slice to remove any `IoSlice`s that are fully advanced over.
/// If the cursor ends up in the middle of an `IoSlice`, it is modified
/// to start at that cursor.
///
/// For example, if we have a slice of two 8-byte `IoSlice`s, and we advance by 10 bytes,
/// the result will only include the second `IoSlice`, advanced by 2 bytes.
///
/// # Panics
2019-07-25 22:30:52 +02:00
///
/// Panics when trying to advance beyond the end of the slices.
2019-07-25 22:30:52 +02:00
///
/// # Examples
///
/// ```
/// #![feature(io_slice_advance)]
///
/// use std::io::IoSlice;
/// use std::ops::Deref;
///
/// let buf1 = [1; 8];
/// let buf2 = [2; 16];
/// let buf3 = [3; 8];
2019-07-25 22:30:52 +02:00
/// let mut bufs = &mut [
/// IoSlice::new(&buf1),
/// IoSlice::new(&buf2),
/// IoSlice::new(&buf3),
2019-07-25 22:30:52 +02:00
/// ][..];
///
/// // Mark 10 bytes as written.
/// IoSlice::advance_slices(&mut bufs, 10);
2019-07-25 22:30:52 +02:00
/// assert_eq!(bufs[0].deref(), [2; 14].as_ref());
/// assert_eq!(bufs[1].deref(), [3; 8].as_ref());
#[unstable(feature = "io_slice_advance", issue = "62726")]
#[inline]
pub fn advance_slices(bufs: &mut &mut [IoSlice<'a>], n: usize) {
2019-07-25 22:30:52 +02:00
// Number of buffers to remove.
let mut remove = 0;
// Total length of all the to be removed buffers.
let mut accumulated_len = 0;
for buf in bufs.iter() {
if accumulated_len + buf.len() > n {
break;
} else {
accumulated_len += buf.len();
remove += 1;
}
}
*bufs = &mut replace(bufs, &mut [])[remove..];
if bufs.is_empty() {
assert!(n == accumulated_len, "advancing io slices beyond their length");
} else {
bufs[0].advance(n - accumulated_len)
2019-07-25 22:30:52 +02:00
}
}
}
#[stable(feature = "iovec", since = "1.36.0")]
impl<'a> Deref for IoSlice<'a> {
type Target = [u8];
#[inline]
fn deref(&self) -> &[u8] {
self.0.as_slice()
}
}
/// A trait for objects which are byte-oriented sinks.
///
2015-07-09 17:13:32 -04:00
/// Implementors of the `Write` trait are sometimes called 'writers'.
///
/// Writers are defined by two required methods, [`write`] and [`flush`]:
///
/// * The [`write`] method will attempt to write some data into the object,
2015-07-09 16:23:27 -04:00
/// returning how many bytes were successfully written.
///
/// * The [`flush`] method is useful for adapters and explicit buffers
2015-07-09 16:23:27 -04:00
/// themselves for ensuring that all buffered data has been pushed out to the
/// 'true sink'.
///
/// Writers are intended to be composable with one another. Many implementors
2016-11-04 20:51:57 +01:00
/// throughout [`std::io`] take and provide types which implement the `Write`
2015-07-09 16:23:27 -04:00
/// trait.
///
2020-08-19 16:26:17 +02:00
/// [`write`]: Write::write
/// [`flush`]: Write::flush
2020-08-12 15:30:15 -07:00
/// [`std::io`]: self
2016-11-04 20:51:57 +01:00
///
2015-07-09 16:23:27 -04:00
/// # Examples
///
/// ```no_run
2015-07-09 16:23:27 -04:00
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> std::io::Result<()> {
/// let data = b"some bytes";
///
/// let mut pos = 0;
/// let mut buffer = File::create("foo.txt")?;
2015-07-09 16:23:27 -04:00
///
/// while pos < data.len() {
/// let bytes_written = buffer.write(&data[pos..])?;
/// pos += bytes_written;
/// }
/// Ok(())
/// }
2015-07-09 16:23:27 -04:00
/// ```
///
/// The trait also provides convenience methods like [`write_all`], which calls
/// `write` in a loop until its entire input has been written.
///
2020-08-19 16:26:17 +02:00
/// [`write_all`]: Write::write_all
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
#[stable(feature = "rust1", since = "1.0.0")]
2021-05-06 13:36:07 +02:00
#[doc(notable_trait)]
#[cfg_attr(not(test), rustc_diagnostic_item = "IoWrite")]
pub trait Write {
/// Write a buffer into this writer, returning how many bytes were written.
///
/// This function will attempt to write the entire contents of `buf`, but
/// the entire write might not succeed, or the write may also generate an
/// error. A call to `write` represents *at most one* attempt to write to
/// any wrapped object.
///
/// Calls to `write` are not guaranteed to block waiting for data to be
2015-05-25 01:53:44 +02:00
/// written, and a write which would otherwise block can be indicated through
2018-01-08 14:16:16 +01:00
/// an [`Err`] variant.
///
2018-01-08 14:16:16 +01:00
/// If the return value is [`Ok(n)`] then it must be guaranteed that
/// `n <= buf.len()`. A return value of `0` typically means that the
/// underlying object is no longer able to accept bytes and will likely not
/// be able to in the future as well, or that the buffer provided is empty.
///
/// # Errors
///
/// Each call to `write` may generate an I/O error indicating that the
/// operation could not be completed. If an error is returned then no bytes
/// in the buffer were written to this writer.
///
/// It is **not** considered an error if the entire buffer could not be
/// written to this writer.
2015-07-09 17:13:32 -04:00
///
2018-01-08 14:16:16 +01:00
/// An error of the [`ErrorKind::Interrupted`] kind is non-fatal and the
/// write operation should be retried if there is nothing else to do.
///
2015-07-09 17:13:32 -04:00
/// # Examples
///
/// ```no_run
2015-07-09 17:13:32 -04:00
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> std::io::Result<()> {
/// let mut buffer = File::create("foo.txt")?;
2015-07-09 17:13:32 -04:00
///
/// // Writes some prefix of the byte string, not necessarily all of it.
/// buffer.write(b"some bytes")?;
/// Ok(())
/// }
2015-07-09 17:13:32 -04:00
/// ```
2020-08-18 19:36:52 +02:00
///
/// [`Ok(n)`]: Ok
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
#[stable(feature = "rust1", since = "1.0.0")]
fn write(&mut self, buf: &[u8]) -> Result<usize>;
2020-08-18 19:36:52 +02:00
/// Like [`write`], except that it writes from a slice of buffers.
///
/// Data is copied from each buffer in order, with the final buffer
/// read from possibly being only partially consumed. This method must
2020-08-18 19:36:52 +02:00
/// behave as a call to [`write`] with the buffers concatenated would.
///
2020-08-18 19:36:52 +02:00
/// The default implementation calls [`write`] with either the first nonempty
/// buffer provided, or an empty one if none exists.
2020-08-18 19:36:52 +02:00
///
/// # Examples
///
/// ```no_run
/// use std::io::IoSlice;
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> std::io::Result<()> {
/// let data1 = [1; 8];
/// let data2 = [15; 8];
/// let io_slice1 = IoSlice::new(&data1);
/// let io_slice2 = IoSlice::new(&data2);
///
/// let mut buffer = File::create("foo.txt")?;
///
/// // Writes some prefix of the byte string, not necessarily all of it.
/// buffer.write_vectored(&[io_slice1, io_slice2])?;
/// Ok(())
/// }
/// ```
///
2020-08-19 16:26:17 +02:00
/// [`write`]: Write::write
#[stable(feature = "iovec", since = "1.36.0")]
fn write_vectored(&mut self, bufs: &[IoSlice<'_>]) -> Result<usize> {
default_write_vectored(|b| self.write(b), bufs)
}
/// Determines if this `Write`r has an efficient [`write_vectored`]
/// implementation.
///
/// If a `Write`r does not override the default [`write_vectored`]
/// implementation, code using it may want to avoid the method all together
/// and coalesce writes into a single buffer for higher performance.
///
/// The default implementation returns `false`.
2020-08-18 19:36:52 +02:00
///
2020-08-19 16:26:17 +02:00
/// [`write_vectored`]: Write::write_vectored
2020-03-11 18:02:52 -07:00
#[unstable(feature = "can_vector", issue = "69941")]
fn is_write_vectored(&self) -> bool {
false
}
/// Flush this output stream, ensuring that all intermediately buffered
/// contents reach their destination.
///
/// # Errors
///
/// It is considered an error if not all bytes could be written due to
/// I/O errors or EOF being reached.
2015-07-09 17:13:32 -04:00
///
/// # Examples
///
/// ```no_run
2015-07-09 17:13:32 -04:00
/// use std::io::prelude::*;
/// use std::io::BufWriter;
/// use std::fs::File;
///
/// fn main() -> std::io::Result<()> {
/// let mut buffer = BufWriter::new(File::create("foo.txt")?);
2015-07-09 17:13:32 -04:00
///
/// buffer.write_all(b"some bytes")?;
/// buffer.flush()?;
/// Ok(())
/// }
2015-07-09 17:13:32 -04:00
/// ```
#[stable(feature = "rust1", since = "1.0.0")]
fn flush(&mut self) -> Result<()>;
/// Attempts to write an entire buffer into this writer.
///
2018-01-08 14:16:16 +01:00
/// This method will continuously call [`write`] until there is no more data
/// to be written or an error of non-[`ErrorKind::Interrupted`] kind is
/// returned. This method will not return until the entire buffer has been
/// successfully written or such an error occurs. The first error that is
2018-01-08 14:16:16 +01:00
/// not of [`ErrorKind::Interrupted`] kind generated from this method will be
/// returned.
///
/// If the buffer contains no data, this will never call [`write`].
///
/// # Errors
///
/// This function will return the first error of
2018-01-08 14:16:16 +01:00
/// non-[`ErrorKind::Interrupted`] kind that [`write`] returns.
///
2020-08-19 16:26:17 +02:00
/// [`write`]: Write::write
2015-07-09 17:13:32 -04:00
///
/// # Examples
///
/// ```no_run
2015-07-09 17:13:32 -04:00
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> std::io::Result<()> {
/// let mut buffer = File::create("foo.txt")?;
2015-07-09 17:13:32 -04:00
///
/// buffer.write_all(b"some bytes")?;
/// Ok(())
/// }
2015-07-09 17:13:32 -04:00
/// ```
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
#[stable(feature = "rust1", since = "1.0.0")]
fn write_all(&mut self, mut buf: &[u8]) -> Result<()> {
while !buf.is_empty() {
match self.write(buf) {
Ok(0) => {
return Err(error::const_io_error!(
ErrorKind::WriteZero,
"failed to write whole buffer",
));
}
Ok(n) => buf = &buf[n..],
Err(ref e) if e.kind() == ErrorKind::Interrupted => {}
Err(e) => return Err(e),
}
}
Ok(())
}
/// Attempts to write multiple buffers into this writer.
///
/// This method will continuously call [`write_vectored`] until there is no
/// more data to be written or an error of non-[`ErrorKind::Interrupted`]
/// kind is returned. This method will not return until all buffers have
/// been successfully written or such an error occurs. The first error that
/// is not of [`ErrorKind::Interrupted`] kind generated from this method
/// will be returned.
///
/// If the buffer contains no data, this will never call [`write_vectored`].
///
/// # Notes
///
2020-08-18 19:36:52 +02:00
/// Unlike [`write_vectored`], this takes a *mutable* reference to
/// a slice of [`IoSlice`]s, not an immutable one. That's because we need to
/// modify the slice to keep track of the bytes already written.
///
/// Once this function returns, the contents of `bufs` are unspecified, as
2020-08-18 19:36:52 +02:00
/// this depends on how many calls to [`write_vectored`] were necessary. It is
/// best to understand this function as taking ownership of `bufs` and to
/// not use `bufs` afterwards. The underlying buffers, to which the
2020-08-18 19:36:52 +02:00
/// [`IoSlice`]s point (but not the [`IoSlice`]s themselves), are unchanged and
/// can be reused.
///
2020-08-19 16:26:17 +02:00
/// [`write_vectored`]: Write::write_vectored
///
/// # Examples
///
/// ```
/// #![feature(write_all_vectored)]
/// # fn main() -> std::io::Result<()> {
///
/// use std::io::{Write, IoSlice};
///
/// let mut writer = Vec::new();
/// let bufs = &mut [
/// IoSlice::new(&[1]),
/// IoSlice::new(&[2, 3]),
/// IoSlice::new(&[4, 5, 6]),
/// ];
///
/// writer.write_all_vectored(bufs)?;
/// // Note: the contents of `bufs` is now undefined, see the Notes section.
///
/// assert_eq!(writer, &[1, 2, 3, 4, 5, 6]);
/// # Ok(()) }
/// ```
#[unstable(feature = "write_all_vectored", issue = "70436")]
fn write_all_vectored(&mut self, mut bufs: &mut [IoSlice<'_>]) -> Result<()> {
// Guarantee that bufs is empty if it contains no data,
// to avoid calling write_vectored if there is no data to be written.
IoSlice::advance_slices(&mut bufs, 0);
while !bufs.is_empty() {
match self.write_vectored(bufs) {
Ok(0) => {
return Err(error::const_io_error!(
ErrorKind::WriteZero,
"failed to write whole buffer",
));
}
Ok(n) => IoSlice::advance_slices(&mut bufs, n),
Err(ref e) if e.kind() == ErrorKind::Interrupted => {}
Err(e) => return Err(e),
}
}
Ok(())
}
/// Writes a formatted string into this writer, returning any error
/// encountered.
///
2015-07-09 17:13:32 -04:00
/// This method is primarily used to interface with the
/// [`format_args!()`] macro, and it is rare that this should
/// explicitly be called. The [`write!()`] macro should be favored to
2015-07-09 17:13:32 -04:00
/// invoke this method instead.
///
2020-08-18 19:36:52 +02:00
/// This function internally uses the [`write_all`] method on
2015-07-09 17:13:32 -04:00
/// this trait and hence will continuously write data so long as no errors
/// are received. This also means that partial writes are not indicated in
/// this signature.
///
2020-08-19 16:26:17 +02:00
/// [`write_all`]: Write::write_all
///
/// # Errors
///
/// This function will return any I/O error reported while formatting.
2015-07-09 17:13:32 -04:00
///
/// # Examples
///
/// ```no_run
2015-07-09 17:13:32 -04:00
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> std::io::Result<()> {
/// let mut buffer = File::create("foo.txt")?;
2015-07-09 17:13:32 -04:00
///
/// // this call
/// write!(buffer, "{:.*}", 2, 1.234567)?;
/// // turns into this:
/// buffer.write_fmt(format_args!("{:.*}", 2, 1.234567))?;
/// Ok(())
/// }
2015-07-09 17:13:32 -04:00
/// ```
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
#[stable(feature = "rust1", since = "1.0.0")]
fn write_fmt(&mut self, fmt: fmt::Arguments<'_>) -> Result<()> {
// Create a shim which translates a Write to a fmt::Write and saves
// off I/O errors. instead of discarding them
struct Adapter<'a, T: ?Sized + 'a> {
inner: &'a mut T,
error: Result<()>,
}
impl<T: Write + ?Sized> fmt::Write for Adapter<'_, T> {
fn write_str(&mut self, s: &str) -> fmt::Result {
match self.inner.write_all(s.as_bytes()) {
Ok(()) => Ok(()),
Err(e) => {
self.error = Err(e);
Err(fmt::Error)
}
}
}
}
let mut output = Adapter { inner: self, error: Ok(()) };
match fmt::write(&mut output, fmt) {
Ok(()) => Ok(()),
Err(..) => {
// check if the error came from the underlying `Write` or not
if output.error.is_err() {
output.error
} else {
Err(error::const_io_error!(ErrorKind::Uncategorized, "formatter error"))
}
}
}
}
/// Creates a "by reference" adapter for this instance of `Write`.
///
/// The returned adapter also implements `Write` and will simply borrow this
/// current writer.
2015-07-09 17:13:32 -04:00
///
/// # Examples
///
/// ```no_run
2015-07-09 17:13:32 -04:00
/// use std::io::Write;
/// use std::fs::File;
///
/// fn main() -> std::io::Result<()> {
/// let mut buffer = File::create("foo.txt")?;
2015-07-09 17:13:32 -04:00
///
/// let reference = buffer.by_ref();
2015-07-09 17:13:32 -04:00
///
/// // we can use reference just like our original buffer
/// reference.write_all(b"some bytes")?;
/// Ok(())
/// }
2015-07-09 17:13:32 -04:00
/// ```
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
fn by_ref(&mut self) -> &mut Self
where
Self: Sized,
{
self
}
}
2015-07-09 16:34:32 -04:00
/// The `Seek` trait provides a cursor which can be moved within a stream of
/// bytes.
///
/// The stream typically has a fixed size, allowing seeking relative to either
/// end or the current offset.
2015-07-09 16:34:32 -04:00
///
/// # Examples
///
2020-08-18 19:36:52 +02:00
/// [`File`]s implement `Seek`:
2015-07-09 16:34:32 -04:00
///
2020-08-18 19:36:52 +02:00
/// [`File`]: crate::fs::File
2015-07-09 16:34:32 -04:00
///
/// ```no_run
2015-07-09 16:34:32 -04:00
/// use std::io;
2015-07-09 17:13:32 -04:00
/// use std::io::prelude::*;
2015-07-09 16:34:32 -04:00
/// use std::fs::File;
/// use std::io::SeekFrom;
///
/// fn main() -> io::Result<()> {
/// let mut f = File::open("foo.txt")?;
2015-07-09 16:34:32 -04:00
///
/// // move the cursor 42 bytes from the start of the file
/// f.seek(SeekFrom::Start(42))?;
/// Ok(())
/// }
2015-07-09 16:34:32 -04:00
/// ```
#[stable(feature = "rust1", since = "1.0.0")]
pub trait Seek {
2015-07-09 16:34:32 -04:00
/// Seek to an offset, in bytes, in a stream.
///
/// A seek beyond the end of a stream is allowed, but behavior is defined
/// by the implementation.
///
/// If the seek operation completed successfully,
/// this method returns the new position from the start of the stream.
2016-07-30 13:37:52 +02:00
/// That position can be used later with [`SeekFrom::Start`].
///
/// # Errors
///
2021-05-18 17:05:40 +01:00
/// Seeking can fail, for example because it might involve flushing a buffer.
///
2015-07-09 16:34:32 -04:00
/// Seeking to a negative offset is considered an error.
#[stable(feature = "rust1", since = "1.0.0")]
fn seek(&mut self, pos: SeekFrom) -> Result<u64>;
/// Rewind to the beginning of a stream.
///
/// This is a convenience method, equivalent to `seek(SeekFrom::Start(0))`.
///
/// # Errors
///
2021-05-18 17:05:40 +01:00
/// Rewinding can fail, for example because it might involve flushing a buffer.
///
/// # Example
///
/// ```no_run
/// use std::io::{Read, Seek, Write};
/// use std::fs::OpenOptions;
///
/// let mut f = OpenOptions::new()
/// .write(true)
/// .read(true)
/// .create(true)
/// .open("foo.txt").unwrap();
///
/// let hello = "Hello!\n";
/// write!(f, "{hello}").unwrap();
/// f.rewind().unwrap();
///
/// let mut buf = String::new();
/// f.read_to_string(&mut buf).unwrap();
/// assert_eq!(&buf, hello);
/// ```
2021-07-01 15:07:09 -07:00
#[stable(feature = "seek_rewind", since = "1.55.0")]
fn rewind(&mut self) -> Result<()> {
self.seek(SeekFrom::Start(0))?;
Ok(())
}
/// Returns the length of this stream (in bytes).
///
/// This method is implemented using up to three seek operations. If this
/// method returns successfully, the seek position is unchanged (i.e. the
/// position before calling this method is the same as afterwards).
/// However, if this method returns an error, the seek position is
/// unspecified.
///
/// If you need to obtain the length of *many* streams and you don't care
/// about the seek position afterwards, you can reduce the number of seek
/// operations by simply calling `seek(SeekFrom::End(0))` and using its
/// return value (it is also the stream length).
///
/// Note that length of a stream can change over time (for example, when
/// data is appended to a file). So calling this method multiple times does
/// not necessarily return the same length each time.
///
/// # Example
///
/// ```no_run
/// #![feature(seek_stream_len)]
/// use std::{
/// io::{self, Seek},
/// fs::File,
/// };
///
/// fn main() -> io::Result<()> {
/// let mut f = File::open("foo.txt")?;
///
/// let len = f.stream_len()?;
/// println!("The file is currently {len} bytes long");
/// Ok(())
/// }
/// ```
#[unstable(feature = "seek_stream_len", issue = "59359")]
fn stream_len(&mut self) -> Result<u64> {
let old_pos = self.stream_position()?;
let len = self.seek(SeekFrom::End(0))?;
// Avoid seeking a third time when we were already at the end of the
// stream. The branch is usually way cheaper than a seek operation.
if old_pos != len {
self.seek(SeekFrom::Start(old_pos))?;
}
Ok(len)
}
/// Returns the current seek position from the start of the stream.
///
/// This is equivalent to `self.seek(SeekFrom::Current(0))`.
///
/// # Example
///
/// ```no_run
/// use std::{
/// io::{self, BufRead, BufReader, Seek},
/// fs::File,
/// };
///
/// fn main() -> io::Result<()> {
/// let mut f = BufReader::new(File::open("foo.txt")?);
///
/// let before = f.stream_position()?;
/// f.read_line(&mut String::new())?;
/// let after = f.stream_position()?;
///
/// println!("The first line was {} bytes long", after - before);
/// Ok(())
/// }
/// ```
#[stable(feature = "seek_convenience", since = "1.51.0")]
fn stream_position(&mut self) -> Result<u64> {
self.seek(SeekFrom::Current(0))
}
}
/// Enumeration of possible methods to seek within an I/O object.
2016-07-30 13:37:52 +02:00
///
/// It is used by the [`Seek`] trait.
#[derive(Copy, PartialEq, Eq, Clone, Debug)]
#[stable(feature = "rust1", since = "1.0.0")]
pub enum SeekFrom {
2019-02-09 22:16:58 +00:00
/// Sets the offset to the provided number of bytes.
#[stable(feature = "rust1", since = "1.0.0")]
2016-02-19 16:08:36 -08:00
Start(#[stable(feature = "rust1", since = "1.0.0")] u64),
2019-02-09 22:16:58 +00:00
/// Sets the offset to the size of this object plus the specified number of
/// bytes.
///
2015-12-30 19:54:06 +02:00
/// It is possible to seek beyond the end of an object, but it's an error to
/// seek before byte 0.
#[stable(feature = "rust1", since = "1.0.0")]
2016-02-19 16:08:36 -08:00
End(#[stable(feature = "rust1", since = "1.0.0")] i64),
2019-02-09 22:16:58 +00:00
/// Sets the offset to the current position plus the specified number of
/// bytes.
///
2015-12-30 19:54:06 +02:00
/// It is possible to seek beyond the end of an object, but it's an error to
/// seek before byte 0.
#[stable(feature = "rust1", since = "1.0.0")]
2016-02-19 16:08:36 -08:00
Current(#[stable(feature = "rust1", since = "1.0.0")] i64),
}
fn read_until<R: BufRead + ?Sized>(r: &mut R, delim: u8, buf: &mut Vec<u8>) -> Result<usize> {
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
let mut read = 0;
loop {
let (done, used) = {
let available = match r.fill_buf() {
Ok(n) => n,
Err(ref e) if e.kind() == ErrorKind::Interrupted => continue,
Err(e) => return Err(e),
};
match memchr::memchr(delim, available) {
Some(i) => {
buf.extend_from_slice(&available[..=i]);
(true, i + 1)
}
None => {
std: Stabilize APIs for the 1.6 release This commit is the standard API stabilization commit for the 1.6 release cycle. The list of issues and APIs below have all been through their cycle-long FCP and the libs team decisions are listed below Stabilized APIs * `Read::read_exact` * `ErrorKind::UnexpectedEof` (renamed from `UnexpectedEOF`) * libcore -- this was a bit of a nuanced stabilization, the crate itself is now marked as `#[stable]` and the methods appearing via traits for primitives like `char` and `str` are now also marked as stable. Note that the extension traits themeselves are marked as unstable as they're imported via the prelude. The `try!` macro was also moved from the standard library into libcore to have the same interface. Otherwise the functions all have copied stability from the standard library now. * The `#![no_std]` attribute * `fs::DirBuilder` * `fs::DirBuilder::new` * `fs::DirBuilder::recursive` * `fs::DirBuilder::create` * `os::unix::fs::DirBuilderExt` * `os::unix::fs::DirBuilderExt::mode` * `vec::Drain` * `vec::Vec::drain` * `string::Drain` * `string::String::drain` * `vec_deque::Drain` * `vec_deque::VecDeque::drain` * `collections::hash_map::Drain` * `collections::hash_map::HashMap::drain` * `collections::hash_set::Drain` * `collections::hash_set::HashSet::drain` * `collections::binary_heap::Drain` * `collections::binary_heap::BinaryHeap::drain` * `Vec::extend_from_slice` (renamed from `push_all`) * `Mutex::get_mut` * `Mutex::into_inner` * `RwLock::get_mut` * `RwLock::into_inner` * `Iterator::min_by_key` (renamed from `min_by`) * `Iterator::max_by_key` (renamed from `max_by`) Deprecated APIs * `ErrorKind::UnexpectedEOF` (renamed to `UnexpectedEof`) * `OsString::from_bytes` * `OsStr::to_cstring` * `OsStr::to_bytes` * `fs::walk_dir` and `fs::WalkDir` * `path::Components::peek` * `slice::bytes::MutableByteVector` * `slice::bytes::copy_memory` * `Vec::push_all` (renamed to `extend_from_slice`) * `Duration::span` * `IpAddr` * `SocketAddr::ip` * `Read::tee` * `io::Tee` * `Write::broadcast` * `io::Broadcast` * `Iterator::min_by` (renamed to `min_by_key`) * `Iterator::max_by` (renamed to `max_by_key`) * `net::lookup_addr` New APIs (still unstable) * `<[T]>::sort_by_key` (added to mirror `min_by_key`) Closes #27585 Closes #27704 Closes #27707 Closes #27710 Closes #27711 Closes #27727 Closes #27740 Closes #27744 Closes #27799 Closes #27801 cc #27801 (doesn't close as `Chars` is still unstable) Closes #28968
2015-12-02 17:31:49 -08:00
buf.extend_from_slice(available);
(false, available.len())
}
}
};
r.consume(used);
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
read += used;
if done || used == 0 {
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
return Ok(read);
}
}
}
2015-07-09 14:04:30 -04:00
/// A `BufRead` is a type of `Read`er which has an internal buffer, allowing it
/// to perform extra ways of reading.
///
2015-07-09 16:23:27 -04:00
/// For example, reading line-by-line is inefficient without using a buffer, so
/// if you want to read by line, you'll need `BufRead`, which includes a
/// [`read_line`] method as well as a [`lines`] iterator.
2015-07-09 14:04:30 -04:00
///
/// # Examples
///
/// A locked standard input implements `BufRead`:
///
/// ```no_run
2015-07-09 14:04:30 -04:00
/// use std::io;
2015-07-10 13:49:12 -04:00
/// use std::io::prelude::*;
2015-07-09 14:04:30 -04:00
///
/// let stdin = io::stdin();
/// for line in stdin.lock().lines() {
/// println!("{}", line.unwrap());
/// }
/// ```
///
2016-10-18 20:36:44 +02:00
/// If you have something that implements [`Read`], you can use the [`BufReader`
/// type][`BufReader`] to turn it into a `BufRead`.
2015-07-09 14:04:30 -04:00
///
2016-10-18 20:36:44 +02:00
/// For example, [`File`] implements [`Read`], but not `BufRead`.
/// [`BufReader`] to the rescue!
2015-07-09 14:04:30 -04:00
///
2020-07-18 12:02:03 -07:00
/// [`File`]: crate::fs::File
2020-08-19 16:26:17 +02:00
/// [`read_line`]: BufRead::read_line
/// [`lines`]: BufRead::lines
2015-07-09 14:04:30 -04:00
///
/// ```no_run
2015-07-10 13:49:12 -04:00
/// use std::io::{self, BufReader};
/// use std::io::prelude::*;
2015-07-09 14:04:30 -04:00
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let f = File::open("foo.txt")?;
/// let f = BufReader::new(f);
2015-07-09 14:04:30 -04:00
///
/// for line in f.lines() {
/// println!("{}", line.unwrap());
/// }
2015-07-09 14:04:30 -04:00
///
/// Ok(())
/// }
2015-07-09 14:04:30 -04:00
/// ```
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
#[stable(feature = "rust1", since = "1.0.0")]
pub trait BufRead: Read {
/// Returns the contents of the internal buffer, filling it with more data
/// from the inner reader if it is empty.
///
2015-07-09 14:04:30 -04:00
/// This function is a lower-level call. It needs to be paired with the
/// [`consume`] method to function properly. When calling this
2015-07-09 14:04:30 -04:00
/// method, none of the contents will be "read" in the sense that later
/// calling `read` may return the same contents. As such, [`consume`] must
2016-10-18 20:36:44 +02:00
/// be called with the number of bytes that are consumed from this buffer to
2015-07-09 14:04:30 -04:00
/// ensure that the bytes are never returned twice.
///
2020-08-19 16:26:17 +02:00
/// [`consume`]: BufRead::consume
///
/// An empty buffer returned indicates that the stream has reached EOF.
///
/// # Errors
///
/// This function will return an I/O error if the underlying reader was
/// read, but returned an error.
2015-07-09 14:04:30 -04:00
///
/// # Examples
///
/// A locked standard input implements `BufRead`:
///
/// ```no_run
2015-07-09 14:04:30 -04:00
/// use std::io;
/// use std::io::prelude::*;
///
/// let stdin = io::stdin();
/// let mut stdin = stdin.lock();
///
2019-05-18 12:38:06 -06:00
/// let buffer = stdin.fill_buf().unwrap();
2015-07-09 14:04:30 -04:00
///
2019-05-18 12:38:06 -06:00
/// // work with buffer
/// println!("{buffer:?}");
2015-07-09 14:04:30 -04:00
///
/// // ensure the bytes we worked with aren't returned again later
2019-05-18 12:38:06 -06:00
/// let length = buffer.len();
2015-07-09 14:04:30 -04:00
/// stdin.consume(length);
/// ```
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
#[stable(feature = "rust1", since = "1.0.0")]
fn fill_buf(&mut self) -> Result<&[u8]>;
/// Tells this buffer that `amt` bytes have been consumed from the buffer,
/// so they should no longer be returned in calls to `read`.
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
///
2015-07-09 14:04:30 -04:00
/// This function is a lower-level call. It needs to be paired with the
/// [`fill_buf`] method to function properly. This function does
2015-07-09 14:04:30 -04:00
/// not perform any I/O, it simply informs this object that some amount of
/// its buffer, returned from [`fill_buf`], has been consumed and should
2016-10-18 20:36:44 +02:00
/// no longer be returned. As such, this function may do odd things if
/// [`fill_buf`] isn't called before calling it.
///
2015-07-09 14:04:30 -04:00
/// The `amt` must be `<=` the number of bytes in the buffer returned by
/// [`fill_buf`].
///
2015-07-09 14:04:30 -04:00
/// # Examples
///
/// Since `consume()` is meant to be used with [`fill_buf`],
2015-07-09 14:04:30 -04:00
/// that method's example includes an example of `consume()`.
2016-10-18 20:36:44 +02:00
///
2020-08-19 16:26:17 +02:00
/// [`fill_buf`]: BufRead::fill_buf
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
#[stable(feature = "rust1", since = "1.0.0")]
fn consume(&mut self, amt: usize);
2021-05-29 17:47:51 -04:00
/// Check if the underlying `Read` has any data left to be read.
///
/// This function may fill the buffer to check for data,
/// so this functions returns `Result<bool>`, not `bool`.
///
/// Default implementation calls `fill_buf` and checks that
/// returned slice is empty (which means that there is no data left,
/// since EOF is reached).
///
/// Examples
///
/// ```
/// #![feature(buf_read_has_data_left)]
/// use std::io;
/// use std::io::prelude::*;
///
/// let stdin = io::stdin();
/// let mut stdin = stdin.lock();
///
/// while stdin.has_data_left().unwrap() {
/// let mut line = String::new();
/// stdin.read_line(&mut line).unwrap();
/// // work with line
/// println!("{line:?}");
2021-05-29 17:47:51 -04:00
/// }
/// ```
2021-06-17 23:17:16 -04:00
#[unstable(feature = "buf_read_has_data_left", reason = "recently added", issue = "86423")]
2021-05-29 17:47:51 -04:00
fn has_data_left(&mut self) -> Result<bool> {
self.fill_buf().map(|b| !b.is_empty())
}
/// Read all bytes into `buf` until the delimiter `byte` or EOF is reached.
///
2015-07-09 14:04:30 -04:00
/// This function will read bytes from the underlying stream until the
/// delimiter or EOF is found. Once found, all bytes up to, and including,
/// the delimiter (if found) will be appended to `buf`.
///
/// If successful, this function will return the total number of bytes read.
///
/// This function is blocking and should be used carefully: it is possible for
/// an attacker to continuously send bytes without ever sending the delimiter
/// or EOF.
///
/// # Errors
///
2016-10-18 20:36:44 +02:00
/// This function will ignore all instances of [`ErrorKind::Interrupted`] and
/// will otherwise return any errors returned by [`fill_buf`].
///
/// If an I/O error is encountered then all bytes read so far will be
/// present in `buf` and its length will have been adjusted appropriately.
2015-07-09 14:04:30 -04:00
///
2020-08-19 16:26:17 +02:00
/// [`fill_buf`]: BufRead::fill_buf
2016-10-18 20:36:44 +02:00
///
/// # Examples
2015-07-09 14:04:30 -04:00
///
/// [`std::io::Cursor`][`Cursor`] is a type that implements `BufRead`. In
/// this example, we use [`Cursor`] to read all the bytes in a byte slice
/// in hyphen delimited segments:
2015-07-09 14:04:30 -04:00
///
/// ```
/// use std::io::{self, BufRead};
///
/// let mut cursor = io::Cursor::new(b"lorem-ipsum");
/// let mut buf = vec![];
///
/// // cursor is at 'l'
/// let num_bytes = cursor.read_until(b'-', &mut buf)
/// .expect("reading from cursor won't fail");
/// assert_eq!(num_bytes, 6);
/// assert_eq!(buf, b"lorem-");
/// buf.clear();
///
/// // cursor is at 'i'
/// let num_bytes = cursor.read_until(b'-', &mut buf)
/// .expect("reading from cursor won't fail");
/// assert_eq!(num_bytes, 5);
/// assert_eq!(buf, b"ipsum");
/// buf.clear();
///
/// // cursor is at EOF
/// let num_bytes = cursor.read_until(b'-', &mut buf)
/// .expect("reading from cursor won't fail");
/// assert_eq!(num_bytes, 0);
/// assert_eq!(buf, b"");
2015-07-09 14:04:30 -04:00
/// ```
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
#[stable(feature = "rust1", since = "1.0.0")]
fn read_until(&mut self, byte: u8, buf: &mut Vec<u8>) -> Result<usize> {
read_until(self, byte, buf)
}
2020-08-18 19:36:52 +02:00
/// Read all bytes until a newline (the `0xA` byte) is reached, and append
2022-03-16 03:04:40 +01:00
/// them to the provided buffer. You do not need to clear the buffer before
/// appending.
///
2015-07-09 14:04:30 -04:00
/// This function will read bytes from the underlying stream until the
2020-08-18 19:36:52 +02:00
/// newline delimiter (the `0xA` byte) or EOF is found. Once found, all bytes
2015-07-09 14:04:30 -04:00
/// up to, and including, the delimiter (if found) will be appended to
/// `buf`.
///
/// If successful, this function will return the total number of bytes read.
///
2020-08-18 19:36:52 +02:00
/// If this function returns [`Ok(0)`], the stream has reached EOF.
///
/// This function is blocking and should be used carefully: it is possible for
/// an attacker to continuously send bytes without ever sending a newline
/// or EOF.
///
2020-08-18 19:36:52 +02:00
/// [`Ok(0)`]: Ok
///
/// # Errors
///
/// This function has the same error semantics as [`read_until`] and will
2016-10-18 20:36:44 +02:00
/// also return an error if the read bytes are not valid UTF-8. If an I/O
/// error is encountered then `buf` may contain some bytes already read in
/// the event that all data read so far was valid UTF-8.
2015-07-09 14:04:30 -04:00
///
2020-08-19 16:26:17 +02:00
/// [`read_until`]: BufRead::read_until
2018-02-19 17:19:30 +01:00
///
2015-07-09 14:04:30 -04:00
/// # Examples
///
/// [`std::io::Cursor`][`Cursor`] is a type that implements `BufRead`. In
/// this example, we use [`Cursor`] to read all the lines in a byte slice:
2015-07-09 14:04:30 -04:00
///
/// ```
/// use std::io::{self, BufRead};
///
/// let mut cursor = io::Cursor::new(b"foo\nbar");
/// let mut buf = String::new();
///
/// // cursor is at 'f'
/// let num_bytes = cursor.read_line(&mut buf)
/// .expect("reading from cursor won't fail");
/// assert_eq!(num_bytes, 4);
/// assert_eq!(buf, "foo\n");
/// buf.clear();
///
/// // cursor is at 'b'
/// let num_bytes = cursor.read_line(&mut buf)
/// .expect("reading from cursor won't fail");
/// assert_eq!(num_bytes, 3);
/// assert_eq!(buf, "bar");
/// buf.clear();
///
/// // cursor is at EOF
/// let num_bytes = cursor.read_line(&mut buf)
/// .expect("reading from cursor won't fail");
/// assert_eq!(num_bytes, 0);
/// assert_eq!(buf, "");
2015-07-09 14:04:30 -04:00
/// ```
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
#[stable(feature = "rust1", since = "1.0.0")]
fn read_line(&mut self, buf: &mut String) -> Result<usize> {
// Note that we are not calling the `.read_until` method here, but
// rather our hardcoded implementation. For more details as to why, see
// the comments in `read_to_end`.
Optimize File::read_to_end and read_to_string Reading a file into an empty vector or string buffer can incur unnecessary `read` syscalls and memory re-allocations as the buffer "warms up" and grows to its final size. This is perhaps a necessary evil with generic readers, but files can be read in smarter by checking the file size and reserving that much capacity. `std::fs::read` and `read_to_string` already perform this optimization: they open the file, reads its metadata, and call `with_capacity` with the file size. This ensures that the buffer does not need to be resized and an initial string of small `read` syscalls. However, if a user opens the `File` themselves and calls `file.read_to_end` or `file.read_to_string` they do not get this optimization. ```rust let mut buf = Vec::new(); file.read_to_end(&mut buf)?; ``` I searched through this project's codebase and even here are a *lot* of examples of this. They're found all over in unit tests, which isn't a big deal, but there are also several real instances in the compiler and in Cargo. I've documented the ones I found in a comment here: https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999 Most telling, the `Read` trait and the `read_to_end` method both show this exact pattern as examples of how to use readers. What this says to me is that this shouldn't be solved by simply fixing the instances of it in this codebase. If it's here it's certain to be prevalent in the wider Rust ecosystem. To that end, this commit adds specializations of `read_to_end` and `read_to_string` directly on `File`. This way it's no longer a minor footgun to start with an empty buffer when reading a file in. A nice side effect of this change is that code that accesses a `File` as a bare `Read` constraint or via a `dyn Read` trait object will benefit. For example, this code from `compiler/rustc_serialize/src/json.rs`: ```rust pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> { let mut contents = Vec::new(); match rdr.read_to_end(&mut contents) { ``` Related changes: - I also added specializations to `BufReader` to delegate to `self.inner`'s methods. That way it can call `File`'s optimized implementations if the inner reader is a file. - The private `std::io::append_to_string` function is now marked `unsafe`. - `File::read_to_string` being more efficient means that the performance note for `io::read_to_string` can be softened. I've added @camelid's suggested wording from: https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
2021-10-05 19:50:28 -04:00
unsafe { append_to_string(buf, |b| read_until(self, b'\n', b)) }
}
/// Returns an iterator over the contents of this reader split on the byte
/// `byte`.
///
/// The iterator returned from this function will return instances of
Apply 16 commits (squashed) ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::fmt ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::{rc, sync} ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::string ---------- Fix spacing for links inside code blocks in alloc::vec ---------- Fix spacing for links inside code blocks in core::option ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in core::result ---------- Fix spacing for links inside code blocks in core::{iter::{self, iterator}, stream::stream, poll} ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in std::{fs, path} ---------- Fix spacing for links inside code blocks in std::{collections, time} ---------- Fix spacing for links inside code blocks in and make formatting of `&str`-like types consistent in std::ffi::{c_str, os_str} ---------- Fix spacing for links inside code blocks, and improve link tooltips in std::ffi ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in std::{io::{self, buffered::{bufreader, bufwriter}, cursor, util}, net::{self, addr}} ---------- Fix typo in link to `into` for `OsString` docs ---------- Remove tooltips that will probably become redundant in the future ---------- Apply suggestions from code review Replacing `…std/primitive.reference.html` paths with just `reference` Co-authored-by: Joshua Nelson <github@jyn.dev> ---------- Also replace `…std/primitive.reference.html` paths with just `reference` in `core::pin`
2021-08-25 11:45:08 +02:00
/// <code>[io::Result]<[Vec]\<u8>></code>. Each vector returned will *not* have
2016-10-18 20:36:44 +02:00
/// the delimiter byte at the end.
///
/// This function will yield errors whenever [`read_until`] would have
2016-10-18 20:36:44 +02:00
/// also yielded an error.
2015-07-09 14:04:30 -04:00
///
Apply 16 commits (squashed) ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::fmt ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::{rc, sync} ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::string ---------- Fix spacing for links inside code blocks in alloc::vec ---------- Fix spacing for links inside code blocks in core::option ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in core::result ---------- Fix spacing for links inside code blocks in core::{iter::{self, iterator}, stream::stream, poll} ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in std::{fs, path} ---------- Fix spacing for links inside code blocks in std::{collections, time} ---------- Fix spacing for links inside code blocks in and make formatting of `&str`-like types consistent in std::ffi::{c_str, os_str} ---------- Fix spacing for links inside code blocks, and improve link tooltips in std::ffi ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in std::{io::{self, buffered::{bufreader, bufwriter}, cursor, util}, net::{self, addr}} ---------- Fix typo in link to `into` for `OsString` docs ---------- Remove tooltips that will probably become redundant in the future ---------- Apply suggestions from code review Replacing `…std/primitive.reference.html` paths with just `reference` Co-authored-by: Joshua Nelson <github@jyn.dev> ---------- Also replace `…std/primitive.reference.html` paths with just `reference` in `core::pin`
2021-08-25 11:45:08 +02:00
/// [io::Result]: self::Result "io::Result"
2020-08-19 16:26:17 +02:00
/// [`read_until`]: BufRead::read_until
2016-10-18 20:36:44 +02:00
///
/// # Examples
///
/// [`std::io::Cursor`][`Cursor`] is a type that implements `BufRead`. In
/// this example, we use [`Cursor`] to iterate over all hyphen delimited
/// segments in a byte slice
///
2015-07-09 14:04:30 -04:00
/// ```
/// use std::io::{self, BufRead};
2015-07-09 14:04:30 -04:00
///
/// let cursor = io::Cursor::new(b"lorem-ipsum-dolor");
2015-07-09 14:04:30 -04:00
///
/// let mut split_iter = cursor.split(b'-').map(|l| l.unwrap());
/// assert_eq!(split_iter.next(), Some(b"lorem".to_vec()));
/// assert_eq!(split_iter.next(), Some(b"ipsum".to_vec()));
/// assert_eq!(split_iter.next(), Some(b"dolor".to_vec()));
/// assert_eq!(split_iter.next(), None);
2015-07-09 14:04:30 -04:00
/// ```
#[stable(feature = "rust1", since = "1.0.0")]
fn split(self, byte: u8) -> Split<Self>
where
Self: Sized,
{
Split { buf: self, delim: byte }
}
/// Returns an iterator over the lines of this reader.
///
/// The iterator returned from this function will yield instances of
Apply 16 commits (squashed) ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::fmt ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::{rc, sync} ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::string ---------- Fix spacing for links inside code blocks in alloc::vec ---------- Fix spacing for links inside code blocks in core::option ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in core::result ---------- Fix spacing for links inside code blocks in core::{iter::{self, iterator}, stream::stream, poll} ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in std::{fs, path} ---------- Fix spacing for links inside code blocks in std::{collections, time} ---------- Fix spacing for links inside code blocks in and make formatting of `&str`-like types consistent in std::ffi::{c_str, os_str} ---------- Fix spacing for links inside code blocks, and improve link tooltips in std::ffi ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in std::{io::{self, buffered::{bufreader, bufwriter}, cursor, util}, net::{self, addr}} ---------- Fix typo in link to `into` for `OsString` docs ---------- Remove tooltips that will probably become redundant in the future ---------- Apply suggestions from code review Replacing `…std/primitive.reference.html` paths with just `reference` Co-authored-by: Joshua Nelson <github@jyn.dev> ---------- Also replace `…std/primitive.reference.html` paths with just `reference` in `core::pin`
2021-08-25 11:45:08 +02:00
/// <code>[io::Result]<[String]></code>. Each string returned will *not* have a newline
2020-08-19 16:26:17 +02:00
/// byte (the `0xA` byte) or `CRLF` (`0xD`, `0xA` bytes) at the end.
2015-07-09 14:04:30 -04:00
///
Apply 16 commits (squashed) ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::fmt ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::{rc, sync} ---------- Fix spacing for links inside code blocks, and improve link tooltips in alloc::string ---------- Fix spacing for links inside code blocks in alloc::vec ---------- Fix spacing for links inside code blocks in core::option ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in core::result ---------- Fix spacing for links inside code blocks in core::{iter::{self, iterator}, stream::stream, poll} ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in std::{fs, path} ---------- Fix spacing for links inside code blocks in std::{collections, time} ---------- Fix spacing for links inside code blocks in and make formatting of `&str`-like types consistent in std::ffi::{c_str, os_str} ---------- Fix spacing for links inside code blocks, and improve link tooltips in std::ffi ---------- Fix spacing for links inside code blocks, and improve a few link tooltips in std::{io::{self, buffered::{bufreader, bufwriter}, cursor, util}, net::{self, addr}} ---------- Fix typo in link to `into` for `OsString` docs ---------- Remove tooltips that will probably become redundant in the future ---------- Apply suggestions from code review Replacing `…std/primitive.reference.html` paths with just `reference` Co-authored-by: Joshua Nelson <github@jyn.dev> ---------- Also replace `…std/primitive.reference.html` paths with just `reference` in `core::pin`
2021-08-25 11:45:08 +02:00
/// [io::Result]: self::Result "io::Result"
2016-10-18 20:36:44 +02:00
///
2015-07-09 14:04:30 -04:00
/// # Examples
///
/// [`std::io::Cursor`][`Cursor`] is a type that implements `BufRead`. In
/// this example, we use [`Cursor`] to iterate over all the lines in a byte
/// slice.
///
2015-07-09 14:04:30 -04:00
/// ```
/// use std::io::{self, BufRead};
2015-07-09 14:04:30 -04:00
///
/// let cursor = io::Cursor::new(b"lorem\nipsum\r\ndolor");
2015-07-09 14:04:30 -04:00
///
/// let mut lines_iter = cursor.lines().map(|l| l.unwrap());
/// assert_eq!(lines_iter.next(), Some(String::from("lorem")));
/// assert_eq!(lines_iter.next(), Some(String::from("ipsum")));
/// assert_eq!(lines_iter.next(), Some(String::from("dolor")));
/// assert_eq!(lines_iter.next(), None);
2015-07-09 14:04:30 -04:00
/// ```
///
/// # Errors
///
/// Each line of the iterator has the same error semantics as [`BufRead::read_line`].
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
fn lines(self) -> Lines<Self>
where
Self: Sized,
{
Lines { buf: self }
}
}
/// Adapter to chain together two readers.
///
/// This struct is generally created by calling [`chain`] on a reader.
/// Please see the documentation of [`chain`] for more details.
///
/// [`chain`]: Read::chain
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
#[derive(Debug)]
pub struct Chain<T, U> {
first: T,
second: U,
done_first: bool,
}
impl<T, U> Chain<T, U> {
/// Consumes the `Chain`, returning the wrapped readers.
///
/// # Examples
///
/// ```no_run
/// use std::io;
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let mut foo_file = File::open("foo.txt")?;
/// let mut bar_file = File::open("bar.txt")?;
///
/// let chain = foo_file.chain(bar_file);
/// let (foo_file, bar_file) = chain.into_inner();
/// Ok(())
/// }
/// ```
#[stable(feature = "more_io_inner_methods", since = "1.20.0")]
pub fn into_inner(self) -> (T, U) {
(self.first, self.second)
}
/// Gets references to the underlying readers in this `Chain`.
///
/// # Examples
///
/// ```no_run
/// use std::io;
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let mut foo_file = File::open("foo.txt")?;
/// let mut bar_file = File::open("bar.txt")?;
///
/// let chain = foo_file.chain(bar_file);
/// let (foo_file, bar_file) = chain.get_ref();
/// Ok(())
/// }
/// ```
#[stable(feature = "more_io_inner_methods", since = "1.20.0")]
pub fn get_ref(&self) -> (&T, &U) {
(&self.first, &self.second)
}
/// Gets mutable references to the underlying readers in this `Chain`.
///
/// Care should be taken to avoid modifying the internal I/O state of the
/// underlying readers as doing so may corrupt the internal state of this
/// `Chain`.
///
/// # Examples
///
/// ```no_run
/// use std::io;
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let mut foo_file = File::open("foo.txt")?;
/// let mut bar_file = File::open("bar.txt")?;
///
/// let mut chain = foo_file.chain(bar_file);
/// let (foo_file, bar_file) = chain.get_mut();
/// Ok(())
/// }
/// ```
#[stable(feature = "more_io_inner_methods", since = "1.20.0")]
pub fn get_mut(&mut self) -> (&mut T, &mut U) {
(&mut self.first, &mut self.second)
}
}
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
impl<T: Read, U: Read> Read for Chain<T, U> {
fn read(&mut self, buf: &mut [u8]) -> Result<usize> {
if !self.done_first {
match self.first.read(buf)? {
2019-07-18 14:22:22 +02:00
0 if !buf.is_empty() => self.done_first = true,
n => return Ok(n),
}
}
self.second.read(buf)
}
fn read_vectored(&mut self, bufs: &mut [IoSliceMut<'_>]) -> Result<usize> {
if !self.done_first {
match self.first.read_vectored(bufs)? {
0 if bufs.iter().any(|b| !b.is_empty()) => self.done_first = true,
n => return Ok(n),
}
}
self.second.read_vectored(bufs)
}
}
2016-03-28 01:28:03 +02:00
#[stable(feature = "chain_bufread", since = "1.9.0")]
impl<T: BufRead, U: BufRead> BufRead for Chain<T, U> {
fn fill_buf(&mut self) -> Result<&[u8]> {
if !self.done_first {
match self.first.fill_buf()? {
buf if buf.is_empty() => {
self.done_first = true;
}
2016-03-28 01:28:03 +02:00
buf => return Ok(buf),
}
}
self.second.fill_buf()
}
fn consume(&mut self, amt: usize) {
if !self.done_first { self.first.consume(amt) } else { self.second.consume(amt) }
2016-03-28 01:28:03 +02:00
}
}
impl<T, U> SizeHint for Chain<T, U> {
#[inline]
fn lower_bound(&self) -> usize {
SizeHint::lower_bound(&self.first) + SizeHint::lower_bound(&self.second)
}
#[inline]
2021-01-16 17:48:22 -05:00
fn upper_bound(&self) -> Option<usize> {
match (SizeHint::upper_bound(&self.first), SizeHint::upper_bound(&self.second)) {
(Some(first), Some(second)) => first.checked_add(second),
_ => None,
}
}
}
/// Reader adapter which limits the bytes read from an underlying reader.
///
/// This struct is generally created by calling [`take`] on a reader.
/// Please see the documentation of [`take`] for more details.
///
/// [`take`]: Read::take
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
#[derive(Debug)]
pub struct Take<T> {
inner: T,
limit: u64,
}
impl<T> Take<T> {
/// Returns the number of bytes that can be read before this instance will
/// return EOF.
///
/// # Note
///
2016-11-04 20:51:57 +01:00
/// This instance may reach `EOF` after reading fewer bytes than indicated by
/// this method if the underlying [`Read`] instance reaches EOF.
///
2016-07-30 13:30:41 +02:00
/// # Examples
///
/// ```no_run
2016-07-30 13:30:41 +02:00
/// use std::io;
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let f = File::open("foo.txt")?;
2016-07-30 13:30:41 +02:00
///
/// // read at most five bytes
/// let handle = f.take(5);
2016-07-30 13:30:41 +02:00
///
/// println!("limit: {}", handle.limit());
/// Ok(())
/// }
2016-07-30 13:30:41 +02:00
/// ```
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
pub fn limit(&self) -> u64 {
self.limit
}
/// Sets the number of bytes that can be read before this instance will
/// return EOF. This is the same as constructing a new `Take` instance, so
/// the amount of bytes read and the previous limit value don't matter when
/// calling this method.
///
/// # Examples
///
/// ```no_run
/// use std::io;
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let f = File::open("foo.txt")?;
///
/// // read at most five bytes
/// let mut handle = f.take(5);
/// handle.set_limit(10);
///
/// assert_eq!(handle.limit(), 10);
/// Ok(())
/// }
/// ```
2018-04-05 00:30:49 -06:00
#[stable(feature = "take_set_limit", since = "1.27.0")]
pub fn set_limit(&mut self, limit: u64) {
self.limit = limit;
}
/// Consumes the `Take`, returning the wrapped reader.
///
/// # Examples
///
/// ```no_run
/// use std::io;
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let mut file = File::open("foo.txt")?;
///
/// let mut buffer = [0; 5];
/// let mut handle = file.take(5);
/// handle.read(&mut buffer)?;
///
/// let file = handle.into_inner();
/// Ok(())
/// }
/// ```
2016-12-14 12:39:16 -08:00
#[stable(feature = "io_take_into_inner", since = "1.15.0")]
pub fn into_inner(self) -> T {
self.inner
}
/// Gets a reference to the underlying reader.
///
/// # Examples
///
/// ```no_run
/// use std::io;
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let mut file = File::open("foo.txt")?;
///
/// let mut buffer = [0; 5];
/// let mut handle = file.take(5);
/// handle.read(&mut buffer)?;
///
/// let file = handle.get_ref();
/// Ok(())
/// }
/// ```
#[stable(feature = "more_io_inner_methods", since = "1.20.0")]
pub fn get_ref(&self) -> &T {
&self.inner
}
/// Gets a mutable reference to the underlying reader.
///
/// Care should be taken to avoid modifying the internal I/O state of the
/// underlying reader as doing so may corrupt the internal limit of this
/// `Take`.
///
/// # Examples
///
/// ```no_run
/// use std::io;
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// fn main() -> io::Result<()> {
/// let mut file = File::open("foo.txt")?;
///
/// let mut buffer = [0; 5];
/// let mut handle = file.take(5);
/// handle.read(&mut buffer)?;
///
/// let file = handle.get_mut();
/// Ok(())
/// }
/// ```
#[stable(feature = "more_io_inner_methods", since = "1.20.0")]
pub fn get_mut(&mut self) -> &mut T {
&mut self.inner
}
}
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
impl<T: Read> Read for Take<T> {
fn read(&mut self, buf: &mut [u8]) -> Result<usize> {
// Don't call into inner reader at all at EOF because it may still block
if self.limit == 0 {
return Ok(0);
}
let max = cmp::min(buf.len() as u64, self.limit) as usize;
let n = self.inner.read(&mut buf[..max])?;
assert!(n as u64 <= self.limit, "number of read bytes exceeds limit");
self.limit -= n as u64;
Ok(n)
}
fn read_buf(&mut self, mut buf: BorrowedCursor<'_>) -> Result<()> {
2021-01-17 22:28:18 -08:00
// Don't call into inner reader at all at EOF because it may still block
if self.limit == 0 {
return Ok(());
}
if self.limit <= buf.capacity() as u64 {
2021-10-20 20:57:28 -07:00
// if we just use an as cast to convert, limit may wrap around on a 32 bit target
let limit = cmp::min(self.limit, usize::MAX as u64) as usize;
let extra_init = cmp::min(limit as usize, buf.init_ref().len());
2021-10-20 20:57:28 -07:00
// SAFETY: no uninit data is written to ibuf
let ibuf = unsafe { &mut buf.as_mut()[..limit] };
2021-01-17 22:28:18 -08:00
let mut sliced_buf: BorrowedBuf<'_> = ibuf.into();
2021-01-17 22:28:18 -08:00
2021-10-20 20:57:28 -07:00
// SAFETY: extra_init bytes of ibuf are known to be initialized
2021-01-17 22:28:18 -08:00
unsafe {
sliced_buf.set_init(extra_init);
2021-01-17 22:28:18 -08:00
}
let mut cursor = sliced_buf.unfilled();
self.inner.read_buf(cursor.reborrow())?;
2021-01-17 22:28:18 -08:00
let new_init = cursor.init_ref().len();
let filled = sliced_buf.len();
2021-01-17 22:28:18 -08:00
// cursor / sliced_buf / ibuf must drop here
2021-01-17 22:28:18 -08:00
unsafe {
// SAFETY: filled bytes have been filled and therefore initialized
buf.advance(filled);
// SAFETY: new_init bytes of buf's unfilled buffer have been initialized
buf.set_init(new_init);
2021-01-17 22:28:18 -08:00
}
self.limit -= filled as u64;
} else {
let written = buf.written();
self.inner.read_buf(buf.reborrow())?;
self.limit -= (buf.written() - written) as u64;
2021-01-17 22:28:18 -08:00
}
Ok(())
}
}
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
2015-02-21 14:59:29 -08:00
impl<T: BufRead> BufRead for Take<T> {
fn fill_buf(&mut self) -> Result<&[u8]> {
// Don't call into inner reader at all at EOF because it may still block
if self.limit == 0 {
return Ok(&[]);
}
let buf = self.inner.fill_buf()?;
2015-02-21 14:59:29 -08:00
let cap = cmp::min(buf.len() as u64, self.limit) as usize;
Ok(&buf[..cap])
}
fn consume(&mut self, amt: usize) {
// Don't let callers reset the limit by passing an overlarge value
let amt = cmp::min(amt as u64, self.limit) as usize;
self.limit -= amt as u64;
self.inner.consume(amt);
}
}
impl<T> SizeHint for Take<T> {
#[inline]
fn lower_bound(&self) -> usize {
cmp::min(SizeHint::lower_bound(&self.inner) as u64, self.limit) as usize
}
#[inline]
fn upper_bound(&self) -> Option<usize> {
match SizeHint::upper_bound(&self.inner) {
Some(upper_bound) => Some(cmp::min(upper_bound as u64, self.limit) as usize),
None => self.limit.try_into().ok(),
}
}
}
/// An iterator over `u8` values of a reader.
///
/// This struct is generally created by calling [`bytes`] on a reader.
/// Please see the documentation of [`bytes`] for more details.
///
/// [`bytes`]: Read::bytes
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
#[derive(Debug)]
pub struct Bytes<R> {
inner: R,
}
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
impl<R: Read> Iterator for Bytes<R> {
type Item = Result<u8>;
fn next(&mut self) -> Option<Result<u8>> {
2018-11-29 20:53:13 -05:00
let mut byte = 0;
loop {
2018-11-29 20:53:13 -05:00
return match self.inner.read(slice::from_mut(&mut byte)) {
Ok(0) => None,
2018-11-29 20:53:13 -05:00
Ok(..) => Some(Ok(byte)),
2018-12-17 17:43:52 -08:00
Err(ref e) if e.kind() == ErrorKind::Interrupted => continue,
Err(e) => Some(Err(e)),
};
}
}
2021-01-15 14:20:51 -05:00
2021-01-15 20:45:43 -05:00
fn size_hint(&self) -> (usize, Option<usize>) {
SizeHint::size_hint(&self.inner)
2021-01-15 14:20:51 -05:00
}
}
trait SizeHint {
2021-01-15 19:43:52 -05:00
fn lower_bound(&self) -> usize;
2021-01-15 19:29:49 -05:00
2021-01-15 20:45:43 -05:00
fn upper_bound(&self) -> Option<usize>;
2021-01-15 14:20:51 -05:00
fn size_hint(&self) -> (usize, Option<usize>) {
(self.lower_bound(), self.upper_bound())
}
}
2021-01-15 18:45:00 -05:00
impl<T> SizeHint for T {
#[inline]
2021-01-15 18:45:00 -05:00
default fn lower_bound(&self) -> usize {
0
}
2021-01-15 20:45:43 -05:00
#[inline]
2021-01-15 20:45:43 -05:00
default fn upper_bound(&self) -> Option<usize> {
None
}
}
impl<T> SizeHint for &mut T {
#[inline]
fn lower_bound(&self) -> usize {
SizeHint::lower_bound(*self)
}
#[inline]
fn upper_bound(&self) -> Option<usize> {
SizeHint::upper_bound(*self)
}
}
impl<T> SizeHint for Box<T> {
#[inline]
fn lower_bound(&self) -> usize {
SizeHint::lower_bound(&**self)
}
#[inline]
fn upper_bound(&self) -> Option<usize> {
SizeHint::upper_bound(&**self)
}
}
impl SizeHint for &[u8] {
#[inline]
fn lower_bound(&self) -> usize {
self.len()
}
#[inline]
fn upper_bound(&self) -> Option<usize> {
Some(self.len())
}
}
/// An iterator over the contents of an instance of `BufRead` split on a
/// particular byte.
///
2019-09-16 20:02:36 +08:00
/// This struct is generally created by calling [`split`] on a `BufRead`.
/// Please see the documentation of [`split`] for more details.
///
/// [`split`]: BufRead::split
#[stable(feature = "rust1", since = "1.0.0")]
#[derive(Debug)]
pub struct Split<B> {
buf: B,
delim: u8,
}
#[stable(feature = "rust1", since = "1.0.0")]
impl<B: BufRead> Iterator for Split<B> {
type Item = Result<Vec<u8>>;
fn next(&mut self) -> Option<Result<Vec<u8>>> {
let mut buf = Vec::new();
match self.buf.read_until(self.delim, &mut buf) {
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
Ok(0) => None,
Ok(_n) => {
if buf[buf.len() - 1] == self.delim {
buf.pop();
}
Some(Ok(buf))
}
Err(e) => Some(Err(e)),
}
}
}
/// An iterator over the lines of an instance of `BufRead`.
///
2019-09-16 20:02:36 +08:00
/// This struct is generally created by calling [`lines`] on a `BufRead`.
/// Please see the documentation of [`lines`] for more details.
///
/// [`lines`]: BufRead::lines
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
#[derive(Debug)]
pub struct Lines<B> {
buf: B,
}
std: Stabilize portions of the `io` module The new `io` module has had some time to bake and this commit stabilizes some of the utilities associated with it. This commit also deprecates a number of `std::old_io::util` functions and structures. These items are now `#[stable]` * `Cursor` * `Cursor::{new, into_inner, get_ref, get_mut, position, set_position}` * Implementations of I/O traits for `Cursor<T>` * Delegating implementations of I/O traits for references and `Box` pointers * Implementations of I/O traits for primitives like slices and `Vec<T>` * `ReadExt::bytes` * `Bytes` (and impls) * `ReadExt::chain` * `Chain` (and impls) * `ReadExt::take` (and impls) * `BufReadExt::lines` * `Lines` (and impls) * `io::copy` * `io::{empty, Empty}` (and impls) * `io::{sink, Sink}` (and impls) * `io::{repeat, Repeat}` (and impls) These items remain `#[unstable]` * Core I/O traits. These may want a little bit more time to bake along with the commonly used methods like `read_to_end`. * `BufReadExt::split` - this function may be renamed to not conflict with `SliceExt::split`. * `Error` - there are a number of questions about its representation, `ErrorKind`, and usability. These items are now `#[deprecated]` in `old_io` * `LimitReader` - use `take` instead * `NullWriter` - use `io::sink` instead * `ZeroReader` - use `io::repeat` instead * `NullReader` - use `io::empty` instead * `MultiWriter` - use `broadcast` instead * `ChainedReader` - use `chain` instead * `TeeReader` - use `tee` instead * `copy` - use `io::copy` instead [breaking-change]
2015-03-03 14:49:03 -08:00
#[stable(feature = "rust1", since = "1.0.0")]
impl<B: BufRead> Iterator for Lines<B> {
type Item = Result<String>;
fn next(&mut self) -> Option<Result<String>> {
let mut buf = String::new();
match self.buf.read_line(&mut buf) {
std: Stabilize the `io` module The new `std::io` module has had some time to bake now, and this commit stabilizes its functionality. There are still portions of the module which remain unstable, and below contains a summart of the actions taken. This commit also deprecates the entire contents of the `old_io` module in a blanket fashion. All APIs should now have a reasonable replacement in the new I/O modules. Stable APIs: * `std::io` (the name) * `std::io::prelude` (the name) * `Read` * `Read::read` * `Read::{read_to_end, read_to_string}` after being modified to return a `usize` for the number of bytes read. * `Write` * `Write::write` * `Write::{write_all, write_fmt}` * `BufRead` * `BufRead::{fill_buf, consume}` * `BufRead::{read_line, read_until}` after being modified to return a `usize` for the number of bytes read. * `BufReader` * `BufReader::{new, with_capacity}` * `BufReader::{get_ref, get_mut, into_inner}` * `{Read,BufRead} for BufReader` * `BufWriter` * `BufWriter::{new, with_capacity}` * `BufWriter::{get_ref, get_mut, into_inner}` * `Write for BufWriter` * `IntoInnerError` * `IntoInnerError::{error, into_inner}` * `{Error,Display} for IntoInnerError` * `LineWriter` * `LineWriter::{new, with_capacity}` - `with_capacity` was added * `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added) * `Write for LineWriter` * `BufStream` * `BufStream::{new, with_capacities}` * `BufStream::{get_ref, get_mut, into_inner}` * `{BufRead,Read,Write} for BufStream` * `stdin` * `Stdin` * `Stdin::lock` * `Stdin::read_line` - added method * `StdinLock` * `Read for Stdin` * `{Read,BufRead} for StdinLock` * `stdout` * `Stdout` * `Stdout::lock` * `StdoutLock` * `Write for Stdout` * `Write for StdoutLock` * `stderr` * `Stderr` * `Stderr::lock` * `StderrLock` * `Write for Stderr` * `Write for StderrLock` * `io::Result` * `io::Error` * `io::Error::last_os_error` * `{Display, Error} for Error` Unstable APIs: (reasons can be found in the commit itself) * `Write::flush` * `Seek` * `ErrorKind` * `Error::new` * `Error::from_os_error` * `Error::kind` Deprecated APIs * `Error::description` - available via the `Error` trait * `Error::detail` - available via the `Display` implementation * `thread::Builder::{stdout, stderr}` Changes in functionality: * `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing backtraces has migrated to `std::io`. * The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed by folding functionality into the corresponding trait. [breaking-change]
2015-03-11 14:16:46 -07:00
Ok(0) => None,
Ok(_n) => {
if buf.ends_with('\n') {
buf.pop();
if buf.ends_with('\r') {
buf.pop();
}
}
Some(Ok(buf))
}
Err(e) => Some(Err(e)),
}
}
}