1
Fork 0

Overhaul token collection.

This commit does the following.

- Renames `collect_tokens_trailing_token` as `collect_tokens`, because
  (a) it's annoying long, and (b) the `_trailing_token` bit is less
  accurate now that its types have changed.

- In `collect_tokens`, adds a `Option<CollectPos>` argument and a
  `UsePreAttrPos` in the return type of `f`. These are used in
  `parse_expr_force_collect` (for vanilla expressions) and in
  `parse_stmt_without_recovery` (for two different cases of expression
  statements). Together these ensure are enough to fix all the problems
  with token collection and assoc expressions. The changes to the
  `stringify.rs` test demonstrate some of these.

- Adds a new test. The code in this test was causing an assertion
  failure prior to this commit, due to an invalid `NodeRange`.

The extra complexity is annoying, but necessary to fix the existing
problems.
This commit is contained in:
Nicholas Nethercote 2024-08-06 17:16:40 +10:00
parent fe460ac28b
commit 9d31f86f0d
12 changed files with 414 additions and 292 deletions

View file

@ -913,7 +913,7 @@ impl<'a> Parser<'a> {
let snapshot = self.create_snapshot_for_diagnostic();
let attrs = self.parse_outer_attributes()?;
match self.parse_expr_res(Restrictions::CONST_EXPR, attrs) {
Ok(expr) => {
Ok((expr, _)) => {
return Ok(Some(self.dummy_const_arg_needs_braces(
self.dcx().struct_span_err(expr.span, "invalid const generic expression"),
expr.span,