Move jointness censoring to proc_macro

Proc-macro API currently exposes jointness in `Punct` tokens. That is,
`+` in `+one` is **non** joint.

Our lexer produces jointness info for all tokens, so we need to censor
it *somewhere*

Previously we did this in a lexer, but it makes more sense to do this
in a proc-macro server.
This commit is contained in:
Aleksey Kladov 2020-09-03 15:22:07 +02:00
parent 08deb863bd
commit 850c3219fb
2 changed files with 12 additions and 8 deletions

View file

@ -262,10 +262,7 @@ impl<'a> TokenTreesReader<'a> {
}
_ => {
let tt = TokenTree::Token(self.token.take());
let mut is_joint = self.bump();
if !self.token.is_op() {
is_joint = NonJoint;
}
let is_joint = self.bump();
Ok((tt, is_joint))
}
}