- the old interface between HermitCore and the Rust Standard Library
based on a small C library (newlib)
- remove this interface and call directly the unikernel
- remove the dependency to the HermitCore linker
- use rust-lld as linker
This is duplicated in a few locations throughout the sysroot to work
around issues with not exporting a macro in libstd but still wanting it
available to sysroot crates to define blocks. Nowadays though we can
simply depend on the `cfg-if` crate on crates.io, allowing us to use it
from there!
This commit removes all in-tree support for generating backtraces in
favor of depending on the `backtrace` crate on crates.io. This resolves
a very longstanding piece of duplication where the standard library has
long contained the ability to generate a backtrace on panics, but the
code was later extracted and duplicated on crates.io with the
`backtrace` crate. Since that fork each implementation has seen various
improvements one way or another, but typically `backtrace`-the-crate has
lagged behind libstd in one way or another.
The goal here is to remove this duplication of a fairly critical piece
of code and ensure that there's only one source of truth for generating
backtraces between the standard library and the crate on crates.io.
Recently I've been working to bring the `backtrace` crate on crates.io
up to speed with the support in the standard library which includes:
* Support for `StackWalkEx` on MSVC to recover inline frames with
debuginfo.
* Using `libbacktrace` by default on MinGW targets.
* Supporting `libbacktrace` on OSX as an option.
* Ensuring all the requisite support in `backtrace`-the-crate compiles
with `#![no_std]`.
* Updating the `libbacktrace` implementation in `backtrace`-the-crate to
initialize the global state with the correct filename where necessary.
After reviewing the code in libstd the `backtrace` crate should be at
exact feature parity with libstd today. The backtraces generated should
have the same symbols and same number of frames in general, and there's
not known divergence from libstd currently.
Note that one major difference between libstd's backtrace support and
the `backtrace` crate is that on OSX the crates.io crate enables the
`coresymbolication` feature by default. This feature, however, uses
private internal APIs that aren't published for OSX. While they provide
more accurate backtraces this isn't appropriate for libstd distributed
as a binary, so libstd's dependency on the `backtrace` crate explicitly
disables this feature and forces OSX to use `libbacktrace` as a
symbolication strategy.
The long-term goal of this refactoring is to eventually move us towards
a world where we can drop `libbacktrace` entirely and simply use Gimli
and the surrounding crates for backtrace support. That's still aways off
but hopefully will much more easily enabled by having the source of
truth for backtraces live in crates.io!
Procedurally if we go forward with this I'd like to transfer the
`backtrace-rs` crate to the rust-lang GitHub organization as well, but I
figured I'd hold off on that until we get closer to merging.
wasi: Implement more of the standard library
This commit fills out more of the `wasm32-unknown-wasi` target's standard library, notably the `std::fs` module and all of its internals. A few tweaks were made along the way to non-`fs` modules, but the last commit contains the bulk of the work which is to wire up all APIs to their equivalent on WASI targets instead of unconditionally returning "unsupported". After this some basic filesystem operations and such should all be working in WASI!
This commit fills out the `std::fs` module and implementation for WASI.
Not all APIs are implemented, such as permissions-related ones and
`canonicalize`, but all others APIs have been implemented and very
lightly tested so far. We'll eventually want to run a more exhaustive
test suite!
For now the highlights of this commit are:
* The `std::fs::File` type is now backed by `WasiFd`, a raw WASI file
descriptor.
* All APIs in `std::fs` (except permissions/canonicalize) have
implementations for the WASI target.
* A suite of unstable extension traits were added to
`std::os::wasi::fs`. These traits expose the raw filesystem
functionality of WASI, namely `*at` syscalls (opening a file relative
to an already opened one, for example). Additionally metadata only
available on wasi is exposed through these traits.
Perhaps one of the most notable parts is the implementation of
path-taking APIs. WASI actually has no fundamental API that just takes a
path, but rather everything is relative to a previously opened file
descriptor. To allow existing APIs to work (that only take a path) WASI
has a few syscalls to learn about "pre opened" file descriptors by the
runtime. We use these to build a map of existing directory names to file
descriptors, and then when using a path we try to anchor it at an
already-opened file.
This support is very rudimentary though and is intended to be shared
with C since it's likely to be so tricky. For now though the C library
doesn't expose quite an API for us to use, so we implement it for now
and will swap it out as soon as one is available.
This commit deletes the `alloc_system` crate from the standard
distribution. This unstable crate is no longer needed in the modern
stable global allocator world, but rather its functionality is folded
directly into the standard library. The standard library was already the
only stable location to access this crate, and as a result this should
not affect any stable code.
This commit applies a few code size optimizations for the wasm target to
the standard library, namely around panics. We notably know that in most
configurations it's impossible for us to print anything in
wasm32-unknown-unknown so we can skip larger portions of panicking that
are otherwise simply informative. This allows us to get quite a nice
size reduction.
Finally we can also tweak where the allocation happens for the
`Box<Any>` that we panic with. By only allocating once unwinding starts
we can reduce the size of a panicking wasm module from 44k to 350 bytes.
As discussed in #47268, libstd isn't ready to have certain functionality
disabled yet. Follow wasm's approach of adding no-op modules for all of
the features that we can't implement.
I've placed all of those shims in a shims/ subdirectory, so we (the
CloudABI folks) can experiment with removing them more easily. It also
ensures that the code that does work doesn't get polluted with lots of
useless boilerplate code.
Though CloudABI is strongly inspired by POSIX, its absence of features
that don't work well with capability-based sandboxing makes it different
enough that adding bits to sys/unix will make things a mess. This change
therefore adds CloudABI specific platform code under sys/cloudabi and
borrows parts from sys/unix that can be used without changes.
One of the goals of this implementation is to build as much as possible
directly on top of CloudABI's system call layer, as opposed to using the
C library. This is preferred, as the system call layer is supposed to be
stable, whereas the C library ABI technically is not. An advantage of
this approach is that it allows us to implement certain interfaces, such
as mutexes and condition variables more optimally. They can be lighter
than the ones provided by pthreads.
This change disables some modules that cannot realistically be
implemented right now. For example, libstd's pathname abstraction is not
designed with POSIX *at() (e.g., openat()) in mind. The *at() functions
are the only set of file system APIs available on CloudABI. There is no
global file system namespace, nor a process working directory.
Discussions on how to port these modules over are outside the scope of
this change.
Apart from this change, there are still some other minor fixups that
need to be made to platform independent code to make things build. These
will be sent out separately, so they can be reviewed more thoroughly.
This commit adds a new target to the compiler: wasm32-unknown-unknown. This
target is a reimagining of what it looks like to generate WebAssembly code from
Rust. Instead of using Emscripten which can bring with it a weighty runtime this
instead is a target which uses only the LLVM backend for WebAssembly and a
"custom linker" for now which will hopefully one day be direct calls to lld.
Notable features of this target include:
* There is zero runtime footprint. The target assumes nothing exists other than
the wasm32 instruction set.
* There is zero toolchain footprint beyond adding the target. No custom linker
is needed, rustc contains everything.
* Very small wasm modules can be generated directly from Rust code using this
target.
* Most of the standard library is stubbed out to return an error, but anything
related to allocation works (aka `HashMap`, `Vec`, etc).
* Naturally, any `#[no_std]` crate should be 100% compatible with this new
target.
This target is currently somewhat janky due to how linking works. The "linking"
is currently unconditional whole program LTO (aka LLVM is being used as a
linker). Naturally that means compiling programs is pretty slow! Eventually
though this target should have a linker.
This target is also intended to be quite experimental. I'm hoping that this can
act as a catalyst for further experimentation in Rust with WebAssembly. Breaking
changes are very likely to land to this target, so it's not recommended to rely
on it in any critical capacity yet. We'll let you know when it's "production
ready".
---
Currently testing-wise this target is looking pretty good but isn't complete.
I've got almost the entire `run-pass` test suite working with this target (lots
of tests ignored, but many passing as well). The `core` test suite is still
getting LLVM bugs fixed to get that working and will take some time. Relatively
simple programs all seem to work though!
---
It's worth nothing that you may not immediately see the "smallest possible wasm
module" for the input you feed to rustc. For various reasons it's very difficult
to get rid of the final "bloat" in vanilla rustc (again, a real linker should
fix all this). For now what you'll have to do is:
cargo install --git https://github.com/alexcrichton/wasm-gc
wasm-gc foo.wasm bar.wasm
And then `bar.wasm` should be the smallest we can get it!
---
In any case for now I'd love feedback on this, particularly on the various
integration points if you've got better ideas of how to approach them!
Add the libstd-modifications needed for the L4Re target
This commit adds the needed modifications to compile the std crate for the L4 Runtime environment (L4Re).
A target for the L4Re was introduced in commit: c151220a84
In many aspects implementations for linux also apply for the L4Re microkernel.
Some uncommon characteristics had to be resolved:
* L4Re has no network funktionality
* L4Re has a maximum stacksize of 1Mb for threads
* L4Re has no uid or gid
Co-authored-by: Sebastian Humenda <sebastian.humenda@tu-dresden.de>
As suggested in the discussion of PR #43972, std should provide a uniform API to
all platforms. Since there's no networking on L4Re, this now is a module in
`sys::net` providing types and functions/methods returning an error for each
action.
This commit adds the needed modifications to compile the std crate
for the L4 Runtime environment (L4Re).
A target for the L4Re was introduced in commit:
c151220a84
In many aspects implementations for linux also apply for the L4Re
microkernel.
Two uncommon characteristics had to be resolved:
* L4Re has no network funktionality
* L4Re has a maximum stacksize of 1Mb for threads
Co-authored-by: Sebastian Humenda <sebastian.humenda@tu-dresden.de>
