Fix quadratic behavior of repeated vectored writes
Some implementations of `Write::write_vectored` in the standard library (`BufWriter`, `LineWriter`, `Stdout`, `Stderr`) check all buffers to calculate the total length. This is O(n) over the number of buffers.
It's common that only a limited number of buffers is written at a time (e.g. 1024 for `writev(2)`). `write_vectored_all` will then call `write_vectored` repeatedly, leading to a runtime of O(n²) over the number of buffers.
This fix is to only calculate as much as needed if it's needed.
Here's a test program:
```rust
#![feature(write_all_vectored)]
use std::fs::File;
use std::io::{BufWriter, IoSlice, Write};
use std::time::Instant;
fn main() {
let buf = vec![b'\0'; 100_000_000];
let mut slices: Vec<IoSlice<'_>> = buf.chunks(100).map(IoSlice::new).collect();
let mut writer = BufWriter::new(File::create("/dev/null").unwrap());
let start = Instant::now();
write_smart(&slices, &mut writer);
println!("write_smart(): {:?}", start.elapsed());
let start = Instant::now();
writer.write_all_vectored(&mut slices).unwrap();
println!("write_all_vectored(): {:?}", start.elapsed());
}
fn write_smart(mut slices: &[IoSlice<'_>], writer: &mut impl Write) {
while !slices.is_empty() {
// Only try to write as many slices as can be written
let res = writer
.write_vectored(slices.get(..1024).unwrap_or(slices))
.unwrap();
slices = &slices[(res / 100)..];
}
}
```
Before this change:
```
write_smart(): 6.666952ms
write_all_vectored(): 498.437092ms
```
After this change:
```
write_smart(): 6.377158ms
write_all_vectored(): 6.923412ms
```
`LineWriter` (and by extension `Stdout`) isn't fully repaired by this because it looks for newlines. I could open an issue for that after this is merged, I think it's fixable but not trivially.
Improve std::fs::read_to_string example
Resolves [#118621](https://github.com/rust-lang/rust/issues/118621)
For the original code to succeed it requires address.txt to contain a socketaddress, however it is much easier to follow if this is just any strong - eg address could be a street address or just text.
Also changed the variable name from "foo" to something more meaningful as cargo clippy warns you against using foo as a placeholder.
```
$ cat main.rs
use std::fs;
use std::error::Error;
fn main() -> Result<(), Box<dyn Error>> {
let addr: String = fs::read_to_string("address.txt")?.parse()?;
println!("{}", addr);
Ok(())
}
$ cat address.txt
123 rusty lane
san francisco 94999
$ cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.00s
Running `/home/haydon/workspace/rust-test-pr/tester/target/debug/tester`
123 rusty lane
san francisco 94999
```
Make `std::os::unix::ucred` module private
Tracking issue: #42839
Currently, this unstable module exists: [`std::os::unix::ucred`](https://doc.rust-lang.org/stable/std/os/unix/ucred/index.html).
All it does is provide `UCred` (which is also available from `std::os::unix::net`), `impl_*` (which is probably a mishap and should be private) and `peer_cred` (which is undocumented but has a documented counterpart at `std::os::unix::net::UnixStream::peer_cred`).
This PR makes the entire `ucred` module private and moves it into `net`, because that's where it is used.
I hope it's fine to simply remove it without a deprecation phase. Otherwise, I can add back a deprecated reexport module `std::os::unix::ucred`.
`@rustbot` label: -T-libs +T-libs-api
Rollup of 9 pull requests
Successful merges:
- #121958 (Fix redundant import errors for preload extern crate)
- #121976 (Add an option to have an external download/bootstrap cache)
- #122022 (loongarch: add frecipe and relax target feature)
- #122026 (Do not try to format removed files)
- #122027 (Uplift some feeding out of `associated_type_for_impl_trait_in_impl` and into queries)
- #122063 (Make the lowering of `thir::ExprKind::If` easier to follow)
- #122074 (Add missing PartialOrd trait implementation doc for array)
- #122082 (remove outdated fixme comment)
- #122091 (Note why we're using a new thread in `test_get_os_named_thread`)
r? `@ghost`
`@rustbot` modify labels: rollup
This test won't work on windows 7, as the Thread::set_name function is
not implemented there (win7 does not provide a documented mechanism to
set thread names).
Some implementations of `Write::write_vectored` in the standard
library (`BufWriter`, `LineWriter`, `Stdout`, `Stderr`) check all
buffers to calculate the total length. This is O(n) over the number of
buffers.
It's common that only a limited number of buffers is written at a
time (e.g. 1024 for `writev(2)`). `write_vectored_all` will then call
`write_vectored` repeatedly, leading to a runtime of O(n²) over the
number of buffers.
The fix is to only calculate as much as needed if it's needed.
Cleanup windows `abort_internal`
As the comments on the functions say, we define abort in both in panic_abort and in libstd. This PR makes the two implementation (mostly) the same.
Additionally it:
* uses `options(noreturn)` on the asm instead of using `core::intrinsics::unreachable`.
* removed unnecessary allow lints
* added `FAST_FAIL_FATAL_APP_EXIT` to our generated Windows API bindings instead of defining it manually (std only)
Use the OS thread name by default if `THREAD_INFO` has not been initialized
Currently if `THREAD_INFO` hasn't been initialized then the name will be set to `None`. This PR changes it to use the OS thread name by default. This mostly affects foreign threads at the moment but we could expand this to make more use of the OS thread name in the future.
Note: I've only implemented `Thread::get_name` for windows, linux and macos (and macos adjacent) targets. The rest just return `None`.
fix typo in documentation for std::fs::Permissions
Please check and re-check this PR carefully to see if I got this right.
But by my logic, if the `read_only` function returns `true`, I would not expect be able to write to the file (it being read only); so this text is meant to clarify that `read_only` being `false` doesn't mean *you* can actually write to the file, just that "in general" someone is able to.
Delete architecture-specific memchr code in std::sys
Currently all architecture-specific memchr code is only used in `std::io`. Most of the actual `memchr` capacity exposed to the user through the slice API is instead implemented in `core::slice::memchr`.
Hence this commit deletes `memchr` from `std::sys[_common]` and replace calls to it by calls to `core::slice::memchr` functions. This deletes `(r)memchr` from the list of symbols linked to libc.
The interest of putting architecture specific code back in core is linked to the discussion to be had in #113654
Use volatile access instead of `#[used]` for `on_tls_callback`
The first commit adds a volatile load of `p_thread_callback` when registering a dtor so that the compiler knows if the callback is used or not. I don't believe the added volatile instruction is otherwise significant in the context. In my testing using the volatile load allowed the compiler to correctly reason about whether `on_tls_callback` is used or not, allowing it to be omitted entirely in some cases. Admittedly it usually is used due to `Thread` but that can be avoided (e.g. in DLLs or with custom entry points that avoid the offending APIs). Ideally this would be something the compiler could help a bit more with so we didn't have to use tricks like `#[used]` or volatile. But alas.
I also used this as an opportunity to clean up the `unused` lints which I don't think serve a purpose any more.
The second commit removes the volatile load of `_tls_used` with `#cfg[target_thread_local]` because `#[thread_local]` already implies it. And if it ever didn't then `#[thread_local]` would be broken when there aren't any dtors.
add platform-specific function to get the error number for HermitOS
Extending `std` to get the last error number for HermitOS.
HermitOS is a tier 3 platform and this PR changes only files, wich are related to the tier 3 platform.
Implement unwind safety for Condvar on all platforms
Closes#118009
This commit adds unwind safety consistency to Condvar. Previously, only select platforms implemented unwind safety through auto traits. Known by this committer: On Linux, `Condvar` implemented `UnwindSafe` but on Mac and Windows, it did not. This change changes the implementation from auto to explicit.
In #118009, it was suggested that the platform differences were a bug and that a simple PR could address this. In trying to determine the best information to put in the `#[stable]` attribute, it [was suggested](https://github.com/rust-lang/rust/issues/121690#issuecomment-1968298470) I copy the stability information from the previous unwind safety implementations.
Increase visibility of `join_path` and `split_paths`
Add some crosslinking among `std::env` pages to make it easier to discover `join_paths` and `split_paths`. Also add aliases to help anyone searching for `PATH`.
Closes#118009
This commit adds unwind safety to Condvar. Previously, only select
platforms implemented unwind safety through auto traits. Known by this
committer: Linux was unwind safe, but Mac and Windows are not before
this change.
Extending `std` to get the last error number for HermitOS.
HermitOS is a tier 3 platform and this PR changes only files,
wich are related to the tier 3 platform.
Add a new `wasm32-wasi-preview2` target
This is the initial implementation of the MCP https://github.com/rust-lang/compiler-team/issues/694 creating a new tier 3 target `wasm32-wasi-preview2`. That MCP has been seconded and will most likely be approved in a little over a week from now. For more information on the need for this target, please read the [MCP](https://github.com/rust-lang/compiler-team/issues/694).
There is one aspect of this PR that will become insta-stable once these changes reach a stable compiler:
* A new `target_family` named `wasi` is introduced. This target family incorporates all wasi targets including `wasm32-wasi` and its derivative `wasm32-wasi-preview1-threads`. The difference between `target_family = wasi` and `target_os = wasi` will become much clearer when `wasm32-wasi` is renamed to `wasm32-wasi-preview1` and the `target_os` becomes `wasm32-wasi-preview1`. You can read about this target rename in [this MCP](https://github.com/rust-lang/compiler-team/issues/695) which has also been seconded and will hopefully be officially approved soon.
Additional technical details include:
* Both `std::sys::wasi_preview2` and `std::os::wasi_preview2` have been created and mostly use `#[path]` annotations on their submodules to reach into the existing `wasi` (soon to be `wasi_preview1`) modules. Over time the differences between `wasi_preview1` and `wasi_preview2` will grow and most like all `#[path]` based module aliases will fall away.
* Building `wasi-preview2` relies on a [`wasi-sdk`](https://github.com/WebAssembly/wasi-sdk) in the same way that `wasi-preview1` does (one must include a `wasi-root` path in the `Config.toml` pointing to sysroot included in the wasi-sdk). The target should build against [wasi-sdk v21](https://github.com/WebAssembly/wasi-sdk/releases/tag/wasi-sdk-21) without modifications. However, the wasi-sdk itself is growing [preview2 support](https://github.com/WebAssembly/wasi-sdk/pull/370) so this might shift rapidly. We will be following along quickly to make sure that building the target remains possible as the wasi-sdk changes.
* This requires a [patch to libc](https://github.com/rylev/rust-libc/tree/wasm32-wasi-preview2) that we'll need to land in conjunction with this change. Until that patch lands the target won't actually build.
os::net: expanding TcpStreamExt for Linux with `tcp_deferaccept`.
allows for socket to process only when there is data to process, the option sets a number of seconds until the data is ready.
