1
Fork 0

Overhaul MacArgs::Eq.

The value in `MacArgs::Eq` is currently represented as a `Token`.
Because of `TokenKind::Interpolated`, `Token` can be either a token or
an arbitrary AST fragment. In practice, a `MacArgs::Eq` starts out as a
literal or macro call AST fragment, and then is later lowered to a
literal token. But this is very non-obvious. `Token` is a much more
general type than what is needed.

This commit restricts things, by introducing a new type `MacArgsEqKind`
that is either an AST expression (pre-lowering) or an AST literal
(post-lowering). The downside is that the code is a bit more verbose in
a few places. The benefit is that makes it much clearer what the
possibilities are (though also shorter in some other places). Also, it
removes one use of `TokenKind::Interpolated`, taking us a step closer to
removing that variant, which will let us make `Token` impl `Copy` and
remove many "handle Interpolated" code paths in the parser.

Things to note:
- Error messages have improved. Messages like this:
  ```
  unexpected token: `"bug" + "found"`
  ```
  now say "unexpected expression", which makes more sense. Although
  arbitrary expressions can exist within tokens thanks to
  `TokenKind::Interpolated`, that's not obvious to anyone who doesn't
  know compiler internals.
- In `parse_mac_args_common`, we no longer need to collect tokens for
  the value expression.
This commit is contained in:
Nicholas Nethercote 2022-04-29 06:52:01 +10:00
parent ae5f67f9e8
commit 99f5945f85
21 changed files with 174 additions and 88 deletions

View file

@ -38,7 +38,6 @@
#![recursion_limit = "256"]
#![allow(rustc::potential_query_instability)]
use rustc_ast::token::{self, Token, TokenKind};
use rustc_ast::tokenstream::{CanSynthesizeMissingTokens, TokenStream};
use rustc_ast::visit;
use rustc_ast::{self as ast, *};
@ -874,23 +873,24 @@ impl<'a, 'hir> LoweringContext<'a, 'hir> {
)
}
// This is an inert key-value attribute - it will never be visible to macros
// after it gets lowered to HIR. Therefore, we can synthesize tokens with fake
// spans to handle nonterminals in `#[doc]` (e.g. `#[doc = $e]`).
MacArgs::Eq(eq_span, ref token) => {
// In valid code the value is always representable as a single literal token.
// Otherwise, a dummy token suffices because the error is handled elsewhere.
let token = if let token::Interpolated(nt) = &token.kind
&& let token::NtExpr(expr) = &**nt
{
if let ExprKind::Lit(Lit { token, span, .. }) = expr.kind {
Token::new(TokenKind::Literal(token), span)
} else {
Token::dummy()
}
// after it gets lowered to HIR. Therefore, we can extract literals to handle
// nonterminals in `#[doc]` (e.g. `#[doc = $e]`).
MacArgs::Eq(eq_span, MacArgsEq::Ast(ref expr)) => {
// In valid code the value always ends up as a single literal. Otherwise, a dummy
// literal suffices because the error is handled elsewhere.
let lit = if let ExprKind::Lit(lit) = &expr.kind {
lit.clone()
} else {
unreachable!()
Lit {
token: token::Lit::new(token::LitKind::Err, kw::Empty, None),
kind: LitKind::Err(kw::Empty),
span: DUMMY_SP,
}
};
MacArgs::Eq(eq_span, token)
MacArgs::Eq(eq_span, MacArgsEq::Hir(lit))
}
MacArgs::Eq(_, MacArgsEq::Hir(ref lit)) => {
unreachable!("in literal form when lowering mac args eq: {:?}", lit)
}
}
}