1
Fork 0

Rollup merge of #57004 - nnethercote:TS-change-Stream, r=petrochenkov

Make `TokenStream` less recursive.

`TokenStream` is currently recursive in *two* ways:

- the `TokenTree` variant contains a `ThinTokenStream`, which can
  contain a `TokenStream`;

- the `TokenStream` variant contains a `Vec<TokenStream>`.

The latter is not necessary and causes significant complexity. This
commit replaces it with the simpler `Vec<(TokenTree, IsJoint)>`.

This reduces complexity significantly. In particular, `StreamCursor` is
eliminated, and `Cursor` becomes much simpler, consisting now of just a
`TokenStream` and an index.

The commit also removes the `Extend` impl for `TokenStream`, because it
is only used in tests. (The commit also removes those tests.)

Overall, the commit reduces the number of lines of code by almost 200.
This commit is contained in:
Mazdak Farrokhzad 2019-01-13 17:21:39 +01:00 committed by GitHub
commit b1200a29b0
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
7 changed files with 148 additions and 341 deletions

View file

@ -2914,7 +2914,7 @@ impl<'a> Parser<'a> {
TokenTree::Delimited(
frame.span,
frame.delim,
frame.tree_cursor.original_stream().into(),
frame.tree_cursor.stream.into(),
)
},
token::CloseDelim(_) | token::Eof => unreachable!(),