Previously, we would silently remove any `None`-delimiters when
capturing a `TokenStream`, 'flattenting' them to their inner tokens.
This was not normally visible, since we usually have
`TokenKind::Interpolated` (which gets converted to a `None`-delimited
group during macro invocation) instead of an actual `None`-delimited
group.
However, there are a couple of cases where this becomes visible to
proc-macros:
1. A cross-crate `macro_rules!` macro has a `None`-delimited group
stored in its body (as a result of being produced by another
`macro_rules!` macro). The cross-crate `macro_rules!` invocation
can then expand to an attribute macro invocation, which needs
to be able to see the `None`-delimited group.
2. A proc-macro can invoke an attribute proc-macro with its re-collected
input. If there are any nonterminals present in the input, they will
get re-collected to `None`-delimited groups, which will then get
captured as part of the attribute macro invocation.
Both of these cases are incredibly obscure, so there hopefully won't be
any breakage. This change will allow more agressive 'flattenting' of
nonterminals in #82608 without losing `None`-delimited groups.
Fixes#74800
The definition of `is_x86_feature_detected!` (and similar macros)
depends on the platform - it is produced by a `cfg_if!` invocation on
x86, and a plain `#[cfg]` on other platforms. Since it is part of the
prelude, we will end up importing different hygiene information
depending on the platform. This previously required us to avoid printing raw
`SyntaxContext` ids in any tests that uses the standard library, since
the captured output will be platform-dependent.
Previously, we replaced all `SyntaxContext` ids with "#CTXT", and the
raw `Span` lo/hi bytes with "LO..HI".
This commit adds `#![no_std]` and `extern crate std` to all proc-macro
tests that print spans. This suppresses the prelude import, while
still using lang items from `std` (which gives us a buildable binary).
With this apporach, we will only load hygiene information for things
which we explicitly import. This lets us re-add
`-Z unpretty=expanded,hygiene`, since its output can now be made stable
across all platforms.
Additionally, we use `-Z span-debug` in more places, which lets us avoid
the "LO..HI" normalization hack.
A raw SyntaxContext id is implicitly dependent on the target platform,
since libstd and libcore have platform-dependent #[cfg]s which affect
which macros are invoked. As a result, we must strip out any
SyntaxContext ids from test output to ensure that the captured stdout is
not platform-dependent.
