Let io::copy reuse BufWriter buffers
This optimization will allow users to implicitly set the buffer size for io::copy by wrapping the writer into a `BufWriter` if the default block size is insufficient, which should fix#49921
Due to min_specialization limitations this approach only works with `BufWriter` but not for `BufReader<R>` since `R` is unconstrained and thus the necessary specialization on `R: Read` is not always applicable. Once specialization becomes more powerful this optimization could be extended to look at the reader and writer side and use whichever buffer is larger.
Implement Rust 2021 panic
This implements the Rust 2021 versions of `panic!()`. See https://github.com/rust-lang/rust/issues/80162 and https://github.com/rust-lang/rfcs/pull/3007.
It does so by replacing `{std, core}::panic!()` by a bulitin macro that expands to either `$crate::panic::panic_2015!(..)` or `$crate::panic::panic_2021!(..)` depending on the edition of the caller.
This does not yet make std's panic an alias for core's panic on Rust 2021 as the RFC proposes. That will be a separate change: c5273bdfb2 That change is blocked on figuring out what to do with https://github.com/rust-lang/rust/issues/80846 first.
Stabilize raw ref macros
This stabilizes `raw_ref_macros` (https://github.com/rust-lang/rust/issues/73394), which is possible now that https://github.com/rust-lang/rust/issues/74355 is fixed.
However, as I already said in https://github.com/rust-lang/rust/issues/73394#issuecomment-751342185, I am not particularly happy with the current names of the macros. So I propose we also change them, which means I am proposing to stabilize the following in `core::ptr`:
```rust
pub macro const_addr_of($e:expr) {
&raw const $e
}
pub macro mut_addr_of($e:expr) {
&raw mut $e
}
```
The macro name change means we need another round of FCP. Cc `````@rust-lang/libs`````
Fixes#73394
Add `core::stream::Stream`
[[Tracking issue: #79024](https://github.com/rust-lang/rust/issues/79024)]
This patch adds the `core::stream` submodule and implements `core::stream::Stream` in accordance with [RFC2996](https://github.com/rust-lang/rfcs/pull/2996). The RFC hasn't been merged yet, but as requested by the libs team in https://github.com/rust-lang/rfcs/pull/2996#issuecomment-725696389 I'm filing this PR to get the ball rolling.
## Documentatation
The docs in this PR have been adapted from [`std::iter`](https://doc.rust-lang.org/std/iter/index.html), [`async_std::stream`](https://docs.rs/async-std/1.7.0/async_std/stream/index.html), and [`futures::stream::Stream`](https://docs.rs/futures/0.3.8/futures/stream/trait.Stream.html). Once this PR lands my plan is to follow this up with PRs to add helper methods such as `stream::repeat` which can be used to document more of the concepts that are currently missing. That will allow us to cover concepts such as "infinite streams" and "laziness" in more depth.
## Feature gate
The feature gate for `Stream` is `stream_trait`. This matches the `#[lang = "future_trait"]` attribute name. The intention is that only the APIs defined in RFC2996 will use this feature gate, with future additions such as `stream::repeat` using their own feature gates. This is so we can ensure a smooth path towards stabilizing the `Stream` trait without needing to stabilize all the APIs in `core::stream` at once. But also don't start expanding the API until _after_ stabilization, as was the case with `std::future`.
__edit:__ the feature gate has been changed to `async_stream` to match the feature gate proposed in the RFC.
## Conclusion
This PR introduces `core::stream::{Stream, Next}` and re-exports it from `std` as `std::stream::{Stream, Next}`. Landing `Stream` in the stdlib has been a mult-year process; and it's incredibly exciting for this to finally happen!
---
r? `````@KodrAus`````
cc/ `````@rust-lang/wg-async-foundations````` `````@rust-lang/libs`````
Make std::future a re-export of core::future
After 1a764a7ef5, there are no `std::future`-specific items (except for `cfg(bootstrap)` items removed in 93eed402ad). So, instead of defining `std` own module, we can re-export the `core::future` directly.
Stabilize slice::strip_prefix and slice::strip_suffix
These two methods are useful. The corresponding methods on `str` are already stable.
I believe that stablising these now would not get in the way of, in the future, extending these to take a richer pattern API a la `str`'s patterns.
Tracking PR: #73413. I also have an outstanding PR to improve the docs for these two functions and the corresponding ones on `str`: #75078
I have tried to follow the [instructions in the dev guide](https://rustc-dev-guide.rust-lang.org/stabilization_guide.html#stabilization-pr). The part to do with `compiler/rustc_feature` did not seem applicable. I assume that's because these are just library features, so there is no corresponding machinery in rustc.
Fix intra-doc links for non-path primitives
This does *not* currently work for associated items that are
auto-implemented by the compiler (e.g. `never::eq`), because they aren't
present in the source code. I plan to fix this in a follow-up PR.
Fixes https://github.com/rust-lang/rust/issues/63351 using the approach mentioned in https://github.com/rust-lang/rust/issues/63351#issuecomment-683352130.
r? `@Manishearth`
cc `@petrochenkov` - this makes `rustc_resolve::Res` public, is that ok? I'd just add an identical type alias in rustdoc if not, which seems a waste.
We hope later to extend `core::str::Pattern` to slices too, perhaps as
part of stabilising that. We want to minimise the amount of type
inference breakage when we do that, so we don't want to stabilise
strip_prefix and strip_suffix taking a simple `&[T]`.
@KodrAus suggested the approach of introducing a new perma-unstable
trait, which reduces this future inference break risk.
I found it necessary to make two impls of this trait, as the unsize
coercion don't apply when hunting for trait implementations.
Since SlicePattern's only method returns a reference, and the whole
trait is just a wrapper for slices, I made the trait type be the
non-reference type [T] or [T;N] rather than the reference. Otherwise
the trait would have a lifetime parameter.
I marked both the no-op conversion functions `#[inline]`. I'm not
sure if that is necessary but it seemed at the very least harmless.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
This caught several bugs where people expected `slice` to link to the
primitive, but it linked to the module instead.
This also uses `cfg_attr(bootstrap)` since the ambiguity only occurs
when compiling with stage 1.
Dogfood `str_split_once()`
Part of https://github.com/rust-lang/rust/issues/74773.
Beyond increased clarity, this fixes some instances of a common confusion with how `splitn(2)` behaves: the first element will always be `Some()`, regardless of the delimiter, and even if the value is empty.
Given this code:
```rust
fn main() {
let val = "...";
let mut iter = val.splitn(2, '=');
println!("Input: {:?}, first: {:?}, second: {:?}", val, iter.next(), iter.next());
}
```
We get:
```
Input: "no_delimiter", first: Some("no_delimiter"), second: None
Input: "k=v", first: Some("k"), second: Some("v")
Input: "=", first: Some(""), second: Some("")
```
Using `str_split_once()` makes more clear what happens when the delimiter is not found.
The code in io::stdio before this change misused the ReentrantMutexes,
by calling init() on them and moving them afterwards. Now that
ReentrantMutex requires Pin for init(), this mistake is no longer easy
to make.
unix: Extend UnixStream and UnixDatagram to send and receive file descriptors
Add the functions `recv_vectored_fds` and `send_vectored_fds` to `UnixDatagram` and `UnixStream`. With this functions `UnixDatagram` and `UnixStream` can send and receive file descriptors, by using `recvmsg` and `sendmsg` system call.
They were originally called "opt-in, built-in traits" (OIBITs), but
people realized that the name was too confusing and a mouthful, and so
they were renamed to just "auto traits". The feature flag's name wasn't
updated, though, so that's what this PR does.
There are some other spots in the compiler that still refer to OIBITs,
but I don't think changing those now is worth it since they are internal
and not particularly relevant to this PR.
Also see <https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/opt-in.2C.20built-in.20traits.20(auto.20traits).20feature.20name>.
Stabilize clamp
Tracking issue: https://github.com/rust-lang/rust/issues/44095
Clamp has been merged and unstable for about a year and a half now. How do we feel about stabilizing this?
Make the libstd build script smaller
Of all sysroot crates currently only compiler_builtins, miniz_oxide and std require a build script. compiler_builtins uses to conditionally enable certain features and possibly compile a C version ([source](63ccaf11f0/build.rs)), miniz_oxide only uses it to detect if liballoc is supported as the MSRV is 1.34.0 instead of the 1.36.0 which stabilized liballoc ([source](28514ec09f/miniz_oxide/build.rs)). std now only uses it to enable `freebsd12` when the `RUST_STD_FREEBSD_12_ABI` env var is set, to determine if `restricted-std` should be set, to set the `STD_ENV_ARCH` env var identical to `CARGO_CFG_TARGET_ARCH`, and to unconditionally enable `backtrace_in_libstd`.
If all build scripts were to be removed, it would be possible for rustc to completely compile it's own sysroot. It currently requires a rustc version that already has an available libstd to compile the build scripts. If rustc can completely compile it's own sysroot, rustbuild could be simplified to not forcefully use the bootstrap compiler for build scripts.
`@rustbot` modify labels: +T-compiler +libs-impl
Simplify output capturing
This is a sequence of incremental improvements to the unstable/internal `set_panic` and `set_print` mechanism used by the `test` crate:
1. Remove the `LocalOutput` trait and use `Arc<Mutex<dyn Write>>` instead of `Box<dyn LocalOutput>`. In practice, all implementations of `LocalOutput` were just `Arc<Mutex<..>>`. This simplifies some logic and removes all custom `Sink` implementations such as `library/test/src/helpers/sink.rs`. Also removes a layer of indirection, as the outermost `Box` is now gone. It also means that locking now happens per `write_fmt`, not per individual `write` within. (So `"{} {}\n"` now results in one `lock()`, not four or more.)
2. Since in all cases the `dyn Write`s were just `Vec<u8>`s, replace the type with `Arc<Mutex<Vec<u8>>>`. This simplifies things more, as error handling and flushing can be removed now. This also removes the hack needed in the default panic handler to make this work with `::realstd`, as (unlike `Write`) `Vec<u8>` is from `alloc`, not `std`.
3. Replace the `RefCell`s by regular `Cell`s. The `RefCell`s were mostly used as `mem::replace(&mut *cell.borrow_mut(), something)`, which is just `Cell::replace`. This removes an unecessary bookkeeping and makes the code a bit easier to read.
4. Merge `set_panic` and `set_print` into a single `set_output_capture`. Neither the test crate nor rustc (the only users of this feature) have a use for using these separately. Merging them simplifies things even more. This uses a new function name and feature name, to make it clearer this is internal and not supposed to be used by other crates.
Might be easier to review per commit.
specialize io::copy to use copy_file_range, splice or sendfile
Fixes#74426.
Also covers #60689 but only as an optimization instead of an official API.
The specialization only covers std-owned structs so it should avoid the problems with #71091
Currently linux-only but it should be generalizable to other unix systems that have sendfile/sosplice and similar.
There is a bit of optimization potential around the syscall count. Right now it may end up doing more syscalls than the naive copy loop when doing short (<8KiB) copies between file descriptors.
The test case executes the following:
```
[pid 103776] statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=17, ...}) = 0
[pid 103776] write(4, "wxyz", 4) = 4
[pid 103776] write(4, "iklmn", 5) = 5
[pid 103776] copy_file_range(3, NULL, 4, NULL, 5, 0) = 5
```
0-1 `stat` calls to identify the source file type. 0 if the type can be inferred from the struct from which the FD was extracted
𝖬 `write` to drain the `BufReader`/`BufWriter` wrappers. only happen when buffers are present. 𝖬 ≾ number of wrappers present. If there is a write buffer it may absorb the read buffer contents first so only result in a single write. Vectored writes would also be an option but that would require more invasive changes to `BufWriter`.
𝖭 `copy_file_range`/`splice`/`sendfile` until file size, EOF or the byte limit from `Take` is reached. This should generally be *much* more efficient than the read-write loop and also have other benefits such as DMA offload or extent sharing.
## Benchmarks
```
OLD
test io::tests::bench_file_to_file_copy ... bench: 21,002 ns/iter (+/- 750) = 6240 MB/s [ext4]
test io::tests::bench_file_to_file_copy ... bench: 35,704 ns/iter (+/- 1,108) = 3671 MB/s [btrfs]
test io::tests::bench_file_to_socket_copy ... bench: 57,002 ns/iter (+/- 4,205) = 2299 MB/s
test io::tests::bench_socket_pipe_socket_copy ... bench: 142,640 ns/iter (+/- 77,851) = 918 MB/s
NEW
test io::tests::bench_file_to_file_copy ... bench: 14,745 ns/iter (+/- 519) = 8889 MB/s [ext4]
test io::tests::bench_file_to_file_copy ... bench: 6,128 ns/iter (+/- 227) = 21389 MB/s [btrfs]
test io::tests::bench_file_to_socket_copy ... bench: 13,767 ns/iter (+/- 3,767) = 9520 MB/s
test io::tests::bench_socket_pipe_socket_copy ... bench: 26,471 ns/iter (+/- 6,412) = 4951 MB/s
```
Duration::zero() -> Duration::ZERO
In review for #72790, whether or not a constant or a function should be favored for `#![feature(duration_zero)]` was seen as an open question. In https://github.com/rust-lang/rust/issues/73544#issuecomment-691701670 an invitation was opened to either stabilize the methods or propose a switch to the constant value, supplemented with reasoning. Followup comments suggested community preference leans towards the const ZERO, which would be reason enough.
ZERO also "makes sense" beside existing associated consts for Duration. It is ever so slightly awkward to have a series of constants specifying 1 of various units but leave 0 as a method, especially when they are side-by-side in code. It seems unintuitive for the one non-dynamic value (that isn't from Default) to be not-a-const, which could hurt discoverability of the associated constants overall. Elsewhere in `std`, methods for obtaining a constant value were even deprecated, as seen with [std::u32::min_value](https://doc.rust-lang.org/std/primitive.u32.html#method.min_value).
Most importantly, ZERO costs less to use. A match supports a const pattern, but const fn can only be used if evaluated through a const context such as an inline `const { const_fn() }` or a `const NAME: T = const_fn()` declaration elsewhere. Likewise, while https://github.com/rust-lang/rust/issues/73544#issuecomment-691949373 notes `Duration::zero()` can optimize to a constant value, "can" is not "will". Only const contexts have a strong promise of such. Even without that in mind, the comment in question still leans in favor of the constant for simplicity. As it costs less for a developer to use, may cost less to optimize, and seems to have more of a community consensus for it, the associated const seems best.
r? ```@LukasKalbertodt```
It was only ever used with Vec<u8> anyway. This simplifies some things.
- It no longer needs to be flushed, because that's a no-op anyway for
a Vec<u8>.
- Writing to a Vec<u8> never fails.
- No #[cfg(test)] code is needed anymore to use `realstd` instead of
`std`, because Vec comes from alloc, not std (like Write).
Partially fix#55002, deprecate in another release
Co-authored-by: Ashley Mannix <kodraus@hey.com>
Update stable version for stabilize_spin_loop
Co-authored-by: Joshua Nelson <joshua@yottadb.com>
Use better example for spinlock
As suggested by KodrAus
Remove renamed_spin_loop already available in master
Fix spin loop example
replace `#[allow_internal_unstable]` with `#[rustc_allow_const_fn_unstable]` for `const fn`s
`#[allow_internal_unstable]` is currently used to side-step feature gate and stability checks.
While it was originally only meant to be used only on macros, its use was expanded to `const fn`s.
This pr adds stricter checks for the usage of `#[allow_internal_unstable]` (only on macros) and introduces the `#[rustc_allow_const_fn_unstable]` attribute for usage on `const fn`s.
This pr does not change any of the functionality associated with the use of `#[allow_internal_unstable]` on macros or the usage of `#[rustc_allow_const_fn_unstable]` (instead of `#[allow_internal_unstable]`) on `const fn`s (see https://github.com/rust-lang/rust/issues/69399#issuecomment-712911540).
Note: The check for `#[rustc_allow_const_fn_unstable]` currently only validates that the attribute is used on a function, because I don't know how I would check if the function is a `const fn` at the place of the check. I therefore openend this as a 'draft pull request'.
Closesrust-lang/rust#69399
r? @oli-obk
This expands time's test suite to use more and in more places the
range of methods and constants added to Duration in recent
proposals for the sake of testing more API surface area and
improving legibility.
