rustc: Fix joint-ness of stringified token-streams
This commit fixes `StringReader`'s parsing of tokens which have been stringified through procedural macros. Whether or not a token tree is joint is defined by span information, but when working with procedural macros these spans are often dummy and/or overridden which means that they end up considering all operators joint if they can! The fix here is to track the raw source span as opposed to the overridden span. With this information we can more accurately classify `Punct` structs as either joint or not. Closes #50700
This commit is contained in:
parent
ba64edb3ed
commit
0ee031ab96
5 changed files with 117 additions and 17 deletions
|
@ -581,6 +581,8 @@ impl Token {
|
|||
if tokens.probably_equal_for_proc_macro(&tokens_for_real) {
|
||||
return tokens
|
||||
}
|
||||
info!("cached tokens found, but they're not \"probably equal\", \
|
||||
going with stringified version");
|
||||
}
|
||||
return tokens_for_real
|
||||
}
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue