syntax: don't parse numeric literals in the lexer

This removes a bunch of token types. Tokens now store the original, unaltered
numeric literal (that is still checked for correctness), which is parsed into
an actual number later, as needed, when creating the AST.

This can change how syntax extensions work, but otherwise poses no visible
changes.

[breaking-change]
This commit is contained in:
Corey Richardson 2014-06-18 10:44:20 -07:00
parent 9f5e21da4e
commit cc4213418e
9 changed files with 330 additions and 271 deletions

View file

@ -96,7 +96,7 @@ pub struct Span {
pub static DUMMY_SP: Span = Span { lo: BytePos(0), hi: BytePos(0), expn_info: None };
#[deriving(Clone, PartialEq, Eq, Encodable, Decodable, Hash)]
#[deriving(Clone, PartialEq, Eq, Encodable, Decodable, Hash, Show)]
pub struct Spanned<T> {
pub node: T,
pub span: Span,