Fix quadratic behavior of repeated vectored writes
Some implementations of `Write::write_vectored` in the standard library (`BufWriter`, `LineWriter`, `Stdout`, `Stderr`) check all buffers to calculate the total length. This is O(n) over the number of buffers.
It's common that only a limited number of buffers is written at a time (e.g. 1024 for `writev(2)`). `write_vectored_all` will then call `write_vectored` repeatedly, leading to a runtime of O(n²) over the number of buffers.
This fix is to only calculate as much as needed if it's needed.
Here's a test program:
```rust
#![feature(write_all_vectored)]
use std::fs::File;
use std::io::{BufWriter, IoSlice, Write};
use std::time::Instant;
fn main() {
let buf = vec![b'\0'; 100_000_000];
let mut slices: Vec<IoSlice<'_>> = buf.chunks(100).map(IoSlice::new).collect();
let mut writer = BufWriter::new(File::create("/dev/null").unwrap());
let start = Instant::now();
write_smart(&slices, &mut writer);
println!("write_smart(): {:?}", start.elapsed());
let start = Instant::now();
writer.write_all_vectored(&mut slices).unwrap();
println!("write_all_vectored(): {:?}", start.elapsed());
}
fn write_smart(mut slices: &[IoSlice<'_>], writer: &mut impl Write) {
while !slices.is_empty() {
// Only try to write as many slices as can be written
let res = writer
.write_vectored(slices.get(..1024).unwrap_or(slices))
.unwrap();
slices = &slices[(res / 100)..];
}
}
```
Before this change:
```
write_smart(): 6.666952ms
write_all_vectored(): 498.437092ms
```
After this change:
```
write_smart(): 6.377158ms
write_all_vectored(): 6.923412ms
```
`LineWriter` (and by extension `Stdout`) isn't fully repaired by this because it looks for newlines. I could open an issue for that after this is merged, I think it's fixable but not trivially.
Improve std::fs::read_to_string example
Resolves [#118621](https://github.com/rust-lang/rust/issues/118621)
For the original code to succeed it requires address.txt to contain a socketaddress, however it is much easier to follow if this is just any strong - eg address could be a street address or just text.
Also changed the variable name from "foo" to something more meaningful as cargo clippy warns you against using foo as a placeholder.
```
$ cat main.rs
use std::fs;
use std::error::Error;
fn main() -> Result<(), Box<dyn Error>> {
let addr: String = fs::read_to_string("address.txt")?.parse()?;
println!("{}", addr);
Ok(())
}
$ cat address.txt
123 rusty lane
san francisco 94999
$ cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.00s
Running `/home/haydon/workspace/rust-test-pr/tester/target/debug/tester`
123 rusty lane
san francisco 94999
```
Make `std::os::unix::ucred` module private
Tracking issue: #42839
Currently, this unstable module exists: [`std::os::unix::ucred`](https://doc.rust-lang.org/stable/std/os/unix/ucred/index.html).
All it does is provide `UCred` (which is also available from `std::os::unix::net`), `impl_*` (which is probably a mishap and should be private) and `peer_cred` (which is undocumented but has a documented counterpart at `std::os::unix::net::UnixStream::peer_cred`).
This PR makes the entire `ucred` module private and moves it into `net`, because that's where it is used.
I hope it's fine to simply remove it without a deprecation phase. Otherwise, I can add back a deprecated reexport module `std::os::unix::ucred`.
`@rustbot` label: -T-libs +T-libs-api
Rollup of 9 pull requests
Successful merges:
- #121958 (Fix redundant import errors for preload extern crate)
- #121976 (Add an option to have an external download/bootstrap cache)
- #122022 (loongarch: add frecipe and relax target feature)
- #122026 (Do not try to format removed files)
- #122027 (Uplift some feeding out of `associated_type_for_impl_trait_in_impl` and into queries)
- #122063 (Make the lowering of `thir::ExprKind::If` easier to follow)
- #122074 (Add missing PartialOrd trait implementation doc for array)
- #122082 (remove outdated fixme comment)
- #122091 (Note why we're using a new thread in `test_get_os_named_thread`)
r? `@ghost`
`@rustbot` modify labels: rollup
On modern Linux with Intel AMX and 1KiB matrices,
Arm SVE with potentially 2KiB vectors,
and RISCV Vectors with up to 16KiB vectors,
we must handle dynamic signal stack sizes.
We can do so unconditionally by using getauxval,
but assuming it may return 0 as an answer,
thus falling back to the old constant if needed.
This test won't work on windows 7, as the Thread::set_name function is
not implemented there (win7 does not provide a documented mechanism to
set thread names).
Some implementations of `Write::write_vectored` in the standard
library (`BufWriter`, `LineWriter`, `Stdout`, `Stderr`) check all
buffers to calculate the total length. This is O(n) over the number of
buffers.
It's common that only a limited number of buffers is written at a
time (e.g. 1024 for `writev(2)`). `write_vectored_all` will then call
`write_vectored` repeatedly, leading to a runtime of O(n²) over the
number of buffers.
The fix is to only calculate as much as needed if it's needed.
Cleanup windows `abort_internal`
As the comments on the functions say, we define abort in both in panic_abort and in libstd. This PR makes the two implementation (mostly) the same.
Additionally it:
* uses `options(noreturn)` on the asm instead of using `core::intrinsics::unreachable`.
* removed unnecessary allow lints
* added `FAST_FAIL_FATAL_APP_EXIT` to our generated Windows API bindings instead of defining it manually (std only)
Use the OS thread name by default if `THREAD_INFO` has not been initialized
Currently if `THREAD_INFO` hasn't been initialized then the name will be set to `None`. This PR changes it to use the OS thread name by default. This mostly affects foreign threads at the moment but we could expand this to make more use of the OS thread name in the future.
Note: I've only implemented `Thread::get_name` for windows, linux and macos (and macos adjacent) targets. The rest just return `None`.
fix typo in documentation for std::fs::Permissions
Please check and re-check this PR carefully to see if I got this right.
But by my logic, if the `read_only` function returns `true`, I would not expect be able to write to the file (it being read only); so this text is meant to clarify that `read_only` being `false` doesn't mean *you* can actually write to the file, just that "in general" someone is able to.
Delete architecture-specific memchr code in std::sys
Currently all architecture-specific memchr code is only used in `std::io`. Most of the actual `memchr` capacity exposed to the user through the slice API is instead implemented in `core::slice::memchr`.
Hence this commit deletes `memchr` from `std::sys[_common]` and replace calls to it by calls to `core::slice::memchr` functions. This deletes `(r)memchr` from the list of symbols linked to libc.
The interest of putting architecture specific code back in core is linked to the discussion to be had in #113654
Use volatile access instead of `#[used]` for `on_tls_callback`
The first commit adds a volatile load of `p_thread_callback` when registering a dtor so that the compiler knows if the callback is used or not. I don't believe the added volatile instruction is otherwise significant in the context. In my testing using the volatile load allowed the compiler to correctly reason about whether `on_tls_callback` is used or not, allowing it to be omitted entirely in some cases. Admittedly it usually is used due to `Thread` but that can be avoided (e.g. in DLLs or with custom entry points that avoid the offending APIs). Ideally this would be something the compiler could help a bit more with so we didn't have to use tricks like `#[used]` or volatile. But alas.
I also used this as an opportunity to clean up the `unused` lints which I don't think serve a purpose any more.
The second commit removes the volatile load of `_tls_used` with `#cfg[target_thread_local]` because `#[thread_local]` already implies it. And if it ever didn't then `#[thread_local]` would be broken when there aren't any dtors.
add platform-specific function to get the error number for HermitOS
Extending `std` to get the last error number for HermitOS.
HermitOS is a tier 3 platform and this PR changes only files, wich are related to the tier 3 platform.
Implement unwind safety for Condvar on all platforms
Closes#118009
This commit adds unwind safety consistency to Condvar. Previously, only select platforms implemented unwind safety through auto traits. Known by this committer: On Linux, `Condvar` implemented `UnwindSafe` but on Mac and Windows, it did not. This change changes the implementation from auto to explicit.
In #118009, it was suggested that the platform differences were a bug and that a simple PR could address this. In trying to determine the best information to put in the `#[stable]` attribute, it [was suggested](https://github.com/rust-lang/rust/issues/121690#issuecomment-1968298470) I copy the stability information from the previous unwind safety implementations.
Increase visibility of `join_path` and `split_paths`
Add some crosslinking among `std::env` pages to make it easier to discover `join_paths` and `split_paths`. Also add aliases to help anyone searching for `PATH`.
Closes#118009
This commit adds unwind safety to Condvar. Previously, only select
platforms implemented unwind safety through auto traits. Known by this
committer: Linux was unwind safe, but Mac and Windows are not before
this change.
Extending `std` to get the last error number for HermitOS.
HermitOS is a tier 3 platform and this PR changes only files,
wich are related to the tier 3 platform.
