1
Fork 0
Commit graph

196 commits

Author SHA1 Message Date
bors
bea5bebf3d Auto merge of #105917 - a1phyr:read_chain_more_impls, r=workingjubilee
Specialize some methods of `io::Chain`

This PR specializes the implementation of some methods of `io::Chain`, which could bring performance improvements when using it.
2024-02-19 04:43:54 +00:00
Benoît du Garreau
0a42a540c6 Make io::BorrowedCursor::advance safe
This also keeps the old `advance` method under `advance_unchecked` name.

This makes pattern like `std::io::default_read_buf` safe to write.
2024-02-07 16:46:28 +01:00
Matthias Krüger
d8e9ddc843
Rollup merge of #120607 - conradludgate:fix-120603, r=dtolnay
fix #120603 by adding a check in default_read_buf

Fixes #120603 by checking the returned read n is in-bounds of the cursor.

Interestingly, I noticed that `BorrowedBuf` side-steps this issue by using checked accesses. Maybe this can be switched to unchecked to mirror what BufReader does bf3c6c5bed/library/core/src/io/borrowed_buf.rs (L95)
2024-02-05 06:37:16 +01:00
Conrad Ludgate
a27e45a71b fix #120603 by adding a check in default_read_buf 2024-02-03 11:30:26 +00:00
Benoît du Garreau
a158fb33ef Improve io::Read::read_buf_exact error case
- Use `const_io_error` instead of `Error::new`
- Use the same message as `read_exact`
2024-01-31 11:06:19 +01:00
Kornel
03545161e6 Handle out of memory errors in io:Read::read_to_end() 2024-01-29 23:53:09 +00:00
Matthias Krüger
97ef5a3b53
Rollup merge of #118222 - the8472:copy-use-vec-write, r=m-ou-se
unify read_to_end and io::copy impls for reading into a Vec

This ports over the initial probe (to avoid allocation) and the dynamic read sizing from the io::copy specialization to the `default_read_to_end` implementation which already had its own optimizations for different cases.

I think it should be a best-of-both now.

suggested by `@a1phyr` in https://github.com/rust-lang/rust/pull/117576#issuecomment-1803408492
2023-11-28 16:09:54 +01:00
The 8472
bc7dd5fa6d unify read_to_end and io::copy impls for reading into a Vec 2023-11-26 18:13:36 +01:00
Matthias Krüger
b8657093d6
Rollup merge of #116807 - seanlinsley:patch-2, r=thomcc
Improve rewind documentation

The persistent use of an internal cursor for readers is expected for buffer data types that aren't read all at once, but for files it leads to the confusing situation where calling `read_to_end` on the same file handle multiple times only returns the contents of the file for the first call. This PR adds a note to the documentation clarifying that in that case, `rewind()` must first be called.

I'm unsure if this is the right location for the docs update. Maybe it should also be duplicated on `File`?
2023-11-24 08:23:51 +01:00
bors
e68f935117 Auto merge of #98943 - WilliamVenner:feat/bufread_skip_until, r=dtolnay
Add `BufRead::skip_until`

Alternative version of `BufRead::read_until` that simply discards data, rather than copying it into a buffer.

Useful for situations like skipping irrelevant data in a binary file format that is NUL-terminated.

<details>
<summary>Benchmark</summary>

```
running 2 tests
test bench_read_until ... bench:         123 ns/iter (+/- 6)
test bench_skip_until ... bench:          66 ns/iter (+/- 3)
```

```rs
#![feature(test)]
extern crate test;
use test::Bencher;

use std::io::{ErrorKind, BufRead};

fn skip_until<R: BufRead + ?Sized>(r: &mut R, delim: u8) -> Result<usize, std::io::Error> {
    let mut read = 0;
    loop {
        let (done, used) = {
            let available = match r.fill_buf() {
                Ok(n) => n,
                Err(ref e) if e.kind() == ErrorKind::Interrupted => continue,
                Err(e) => return Err(e),
            };
            match memchr::memchr(delim, available) {
                Some(i) => (true, i + 1),
                None => (false, available.len()),
            }
        };
        r.consume(used);
        read += used;
        if done || used == 0 {
            return Ok(read);
        }
    }
}

const STR: &[u8] = b"Ferris\0Hello, world!\0";

#[bench]
fn bench_skip_until(b: &mut Bencher) {
    b.iter(|| {
        let mut io = std::io::Cursor::new(test::black_box(STR));
        skip_until(&mut io, b'\0').unwrap();
        let mut hello = Vec::with_capacity(b"Hello, world!\0".len());
        let num_bytes = io.read_until(b'\0', &mut hello).unwrap();
        assert_eq!(num_bytes, b"Hello, world!\0".len());
        assert_eq!(hello, b"Hello, world!\0");
    });
}

#[bench]
fn bench_read_until(b: &mut Bencher) {
    b.iter(|| {
        let mut io = std::io::Cursor::new(test::black_box(STR));
        io.read_until(b'\0', &mut Vec::new()).unwrap();
        let mut hello = Vec::with_capacity(b"Hello, world!\0".len());
        let num_bytes = io.read_until(b'\0', &mut hello).unwrap();
        assert_eq!(num_bytes, b"Hello, world!\0".len());
        assert_eq!(hello, b"Hello, world!\0");
    });
}
```
</details>
2023-11-23 22:28:14 +00:00
Takayuki Maeda
baf3059f4e
Rollup merge of #116750 - fintelia:seek_seek_relative, r=Mark-Simulacrum
Add Seek::seek_relative

The `BufReader` struct has a `seek_relative` method because its `Seek::seek` implementation involved dumping the internal buffer (https://github.com/rust-lang/rust/issues/31100).

Unfortunately, there isn't really a good way to take advantage of that method in generic code. This PR adds the same method to the main `Seek` trait with the straightforward default method, and an override for `BufReader` that calls its implementation.

_Also discussed in [this](https://internals.rust-lang.org/t/add-seek-seek-relative/19546) internals.rust-lang.org thread._
2023-11-19 04:14:40 +09:00
John Millikin
341c85648c Move BorrowedBuf and BorrowedCursor from std:io to core::io
Assigned new feature name `core_io_borrowed_buf` to distinguish from the
`Read::read_buf` functionality in `std::io`.
2023-11-09 07:10:11 +09:00
Jonathan Behrens
d9f7c9db02 Improve documentation 2023-11-04 15:45:55 -07:00
Jonathan Behrens
bc058b6f45 Add tracking issue 2023-10-29 19:11:18 -07:00
Jacob Pratt
72d5f4b1dc
Hide internal methods from documentation 2023-10-27 04:30:49 -04:00
David Tolnay
67ea7986c7
Fix invalid stability attribute features in standard library 2023-10-23 13:03:10 -07:00
Nicholas Nethercote
181ce63183 Specialize Bytes<R>::next when R is a BufReader.
This reduces the runtime for a simple program using `Bytes::next` to
iterate through a file from 220ms to 70ms on my Linux box.
2023-10-20 08:52:56 +11:00
Sean Linsley
1fb00335df
Improve rewind documentation 2023-10-16 12:41:43 -05:00
Nicholas Nethercote
4d6810844e Inline Bytes::next and Bytes::size_hint.
This greatly increases its speed.
2023-10-16 08:08:32 +11:00
Jonathan Behrens
d6337e6568 Add Seek::seek_relative 2023-10-14 16:20:32 -07:00
Wilfred Hughes
dca90f7ec1 Remove unnecessary tmp variable in default_read_exact
This variable seems to serve no purpose, and it's a little confusing
when reading std source code, so remove it.
2023-10-07 01:16:45 -04:00
Jason Newcomb
d464b72970 Add more diagnostic items for clippy 2023-10-05 18:21:47 -04:00
Eduardo Sánchez Muñoz
93863383c8 Avoid overflow in IoSlice::advance_slices 2023-09-23 16:27:57 +02:00
bors
5a4e47ebed Auto merge of #114780 - RalfJung:io-safety, r=Amanieu
add more explicit I/O safety documentation

Fixes https://github.com/rust-lang/unsafe-code-guidelines/issues/434
Cc https://github.com/rust-lang/rust/issues/114167
Cc `@Manishearth` `@sunfishcode` `@joshtriplett`
2023-09-22 08:47:51 +00:00
Ben Kimock
642251b71f Use std::io::Error::is_interrupted everywhere 2023-09-03 09:21:10 -04:00
Ralf Jung
1290cd432d further expand on Arc and Box analogy 2023-08-29 21:08:46 +02:00
Ben Kimock
1abaf40ec8 Add a new helper to avoid calling io::Error::kind 2023-08-25 21:26:06 -04:00
Ralf Jung
6d65379e7a clarify what you cannot do 2023-08-22 09:00:07 +02:00
Ralf Jung
2cb9d3def2
typos and wording
Co-authored-by: Dan Gohman <dev@sunfishcode.online>
2023-08-22 08:57:38 +02:00
Josh Triplett
5210f482d7 Partially revert #107200
`Ok(0)` is indeed something the caller may interpret as an error, but
that's the *correct* thing to return if the writer can't accept any more
bytes.
2023-08-16 09:00:16 -07:00
Ralf Jung
e9eca7cda4 reference-counting analogy 2023-08-14 16:53:08 +02:00
Ralf Jung
85e6e82f93 reword the paragraph on file description ownership 2023-08-14 13:55:16 +02:00
Ralf Jung
03c28d5626 don't link to RFCs, they are not up-to-date docs 2023-08-14 13:28:51 +02:00
Ralf Jung
55f18beddd wording; and explain some of the possible consequences of violating io-safety 2023-08-14 08:59:37 +02:00
Ralf Jung
334a54cd83 typos 2023-08-14 08:50:32 +02:00
Ralf Jung
a473e95786 add more explicit I/O safety documentation 2023-08-13 15:07:57 +02:00
Benoît du Garreau
ebc5970329 Add tests and comments about read_to_string and read_line specializations 2023-07-26 23:31:03 +02:00
Benoît du Garreau
cba6e102ec io::Chain: specialize some BufRead methods 2023-07-26 23:28:52 +02:00
Benoît du Garreau
cada71e3f4 io::Chain: specialize some Read methods 2023-07-26 23:28:47 +02:00
Matthias Krüger
876f00a655
Rollup merge of #107200 - mina86:c, r=Amanieu
io: soften ‘at most one write attempt’ requirement in io::Write::write

At the moment, documentation of std::io::Write::write indicates that
call to it ‘represents at most one attempt to write to any wrapped
object’.  It seems that such wording was put there to contrast it with
pre-1.0 interface which attempted to write all the data (it has since
been changed in [RFC 517]).

However, the requirement puts unnecessary constraints and may
complicate adaptors which perform non-trivial transformations on the
data.  For example, they may maintain an internal buffer which needs
to be written out before the write method accepts more data.  It might
be natural to code the method such that it flushes the buffer and then
grabs another chunk of user data.  With the current wording in the
documentation, the adaptor would be forced to return Ok(0).

This commit softens the wording such that implementations can choose
code structure which makes most sense for their particular use case.

While at it, elaborate on the meaning of `Ok(0)` return pointing out
that the write_all methods interprets it as an error.

[RFC 517]: https://rust-lang.github.io/rfcs/0517-io-os-reform.html
2023-06-18 08:06:41 +02:00
Michael Goulet
c55af41e7a
Rollup merge of #111074 - WaffleLapkin:🌟unsizes_your_buf_reader🌟, r=Amanieu
Relax implicit `T: Sized` bounds on `BufReader<T>`, `BufWriter<T>` and `LineWriter<T>`

TL;DR:
```diff,rust
-pub struct BufReader<R> { /* ... */ }
+pub struct BufReader<R: ?Sized> { /* ... */ }

-pub struct BufWriter<W: Write> { /* ... */ }
+pub struct BufWriter<W: ?Sized + Write> { /* ... */ }

-pub struct LineWriter<W: Write> { /* ... */ }
+pub struct LineWriter<W: ?Sized + Write> { /* ... */ }
```

This allows using `&mut BufReader<dyn Read>`, for example.

**This is an insta-stable change**.
2023-06-16 12:53:21 -07:00
Ziru Niu
dd56f930cc Clarify safety concern of io::Read::read is only relevant in unsafe code 2023-05-25 18:38:38 +08:00
William Venner
1d7d7654e6 Add tracking issue ID for bufread_skip_until feature gate 2023-05-18 19:08:41 +01:00
Maybe Waffle
e77b14e2ac Relax implicit R: Sized bound on BufReader<R> 2023-05-01 23:56:23 +00:00
Pietro Albini
4e04da6183 replace version placeholders 2023-04-28 08:47:55 -07:00
bors
9de7d9169c Auto merge of #110655 - ChrisDenton:read-to-end, r=joshtriplett
Limit read size in `File::read_to_end` loop

Fixes #110650.

Windows file reads have perf overhead that's proportional to the buffer size. When we have a reasonable expectation that we know the file size, we can set a reasonable upper bound for the size of the buffer in one read call.
2023-04-23 06:58:28 +00:00
Chris Denton
f74fe8bf4c
Limit read size in File::read_to_end loop
This works around performance issues on  Windows by limiting reads the size of reads when the expected size is known.
2023-04-21 20:54:12 +01:00
Scott McMurray
8055bb87c5 More mem::take in library
A bunch of places were using `replace(…, &mut [])`, but that can just be `take`.
2023-04-20 19:54:46 -07:00
Josh Triplett
afd45c2e10 Stabilize IsTerminal
closes: https://github.com/rust-lang/rust/issues/98070
2023-04-10 17:24:23 +09:00
Tomasz Miąsko
c21f1d09de read_buf_exact: on error, all read bytes are appended to the buffer
Guarantee that when `read_buf_exact` returns, all bytes read will be
appended to the buffer. Including the case when the operations fails.

The motivating use case are operations on a non-blocking reader. When
`read_buf_exact` fails with `ErrorKind::WouldBlock` error, the operation
can be resumed at a later time.
2023-03-11 17:04:41 +01:00