Futureproof Rust for fancier suffixed literals. The Rust compiler tokenises a literal followed immediately (no whitespace) by an identifier as a single token: (for example) the text sequences `"foo"bar`, `1baz` and `1u1024` are now a single token rather than the pairs `"foo"` `bar`, `1` `baz` and `1u` `1024` respectively.
The compiler rejects all such suffixes in the parser, except for the 12 numeric suffixes we have now.
I'm fairly sure this will affect very few programs, since it's not currently legal to have `<literal><identifier>` in a Rust program, except in a macro invocation. Any macro invocation relying on this behaviour can simply separate the two tokens with whitespace: `foo!("bar"baz)` becomes `foo!("bar" baz)`.
This implements [RFC 463](https://github.com/rust-lang/rfcs/blob/master/text/0463-future-proof-literal-suffixes.md), and so closes https://github.com/rust-lang/rust/issues/19088.
This common representation for delimeters should make pattern matching easier. Having a separate `token::DelimToken` enum also allows us to enforce the invariant that the opening and closing delimiters must be the same in `ast::TtDelimited`, removing the need to ensure matched delimiters when working with token trees.
https://github.com/rust-lang/rfcs/pull/221
The current terminology of "task failure" often causes problems when
writing or speaking about code. You often want to talk about the
possibility of an operation that returns a Result "failing", but cannot
because of the ambiguity with task failure. Instead, you have to speak
of "the failing case" or "when the operation does not succeed" or other
circumlocutions.
Likewise, we use a "Failure" header in rustdoc to describe when
operations may fail the task, but it would often be helpful to separate
out a section describing the "Err-producing" case.
We have been steadily moving away from task failure and toward Result as
an error-handling mechanism, so we should optimize our terminology
accordingly: Result-producing functions should be easy to describe.
To update your code, rename any call to `fail!` to `panic!` instead.
Assuming you have not created your own macro named `panic!`, this
will work on UNIX based systems:
grep -lZR 'fail!' . | xargs -0 -l sed -i -e 's/fail!/panic!/g'
You can of course also do this by hand.
[breaking-change]
This should be clearer, and fits in better with the `TTNonterminal` variant.
Renames:
- `TTTok` -> `TTToken`
- `TTDelim` -> `TTDelimited`
- `TTSeq` -> `TTSequence`
This came up when working [on the gl-rs generator extension](990383de80/src/gl_generator/lib.rs (L135-L146)).
The new definition of `TTDelim` adds an associated `Span` that covers the whole token tree and enforces the invariant that a delimited sequence of token trees must have an opening and closing delimiter.
A `get_span` method has also been added to `TokenTree` type to make it easier to implement better error messages for syntax extensions.
Spring cleaning is here! In the Fall! This commit removes quite a large amount
of deprecated functionality from the standard libraries. I tried to ensure that
only old deprecated functionality was removed.
This is removing lots and lots of deprecated features, so this is a breaking
change. Please consult the deprecation messages of the deleted code to see how
to migrate code forward if it still needs migration.
[breaking-change]
[breaking-change]
1. The internal layout for traits has changed from (vtable, data) to (data, vtable). If you were relying on this in unsafe transmutes, you might get some very weird and apparently unrelated errors. You should not be doing this! Prefer not to do this at all, but if you must, you should use raw::TraitObject rather than hardcoding rustc's internal representation into your code.
2. The minimal type of reference-to-vec-literals (e.g., `&[1, 2, 3]`) is now a fixed size vec (e.g., `&[int, ..3]`) where it used to be an unsized vec (e.g., `&[int]`). If you want the unszied type, you must explicitly give the type (e.g., `let x: &[_] = &[1, 2, 3]`). Note in particular where multiple blocks must have the same type (e.g., if and else clauses, vec elements), the compiler will not coerce to the unsized type without a hint. E.g., `[&[1], &[1, 2]]` used to be a valid expression of type '[&[int]]'. It no longer type checks since the first element now has type `&[int, ..1]` and the second has type &[int, ..2]` which are incompatible.
3. The type of blocks (including functions) must be coercible to the expected type (used to be a subtype). Mostly this makes things more flexible and not less (in particular, in the case of coercing function bodies to the return type). However, in some rare cases, this is less flexible. TBH, I'm not exactly sure of the exact effects. I think the change causes us to resolve inferred type variables slightly earlier which might make us slightly more restrictive. Possibly it only affects blocks with unreachable code. E.g., `if ... { fail!(); "Hello" }` used to type check, it no longer does. The fix is to add a semicolon after the string.
These `where` clauses are accepted everywhere generics are currently
accepted and desugar during type collection to the type parameter bounds
we have today.
A new keyword, `where`, has been added. Therefore, this is a breaking
change. Change uses of `where` to other identifiers.
[breaking-change]
This adds support to `quote_expr!` and friends for round-trip hygienic
preservation of Ident.
Here are the pieces of the puzzle:
* adding a method for encoding Ident for re-reading into token tree.
* Support for reading such encoded Idents in the lexer. Note that one
must peek ahead for MOD_SEP after scan_embedded_hygienic_ident.
* To ensure that encoded Idents are only read when we are in the midst
of expanding a `quote_expr` or similar, added a
`read_embedded_ident` flag on `StringReader`.
* pprust support for encoding Ident's as (uint,uint) pairs (for hygiene).
This ended up passing through the lexer but dying later on in parsing when it
wasn't handled. The strategy taken was to copy the `str_lit` funciton, but adapt
it for bytes.
Closes#16278
The `type_overflow` lint, doesn't catch the overflow for `i64` because
the overflow happens earlier in the parse phase when the `u64` as biggest
possible int gets casted to `i64` , without checking the for overflows.
We can't lint in the parse phase, so a refactoring of the `LitInt` type
was necessary.
The types `LitInt`, `LitUint` and `LitIntUnsuffixed` where merged to one
type `LitInt` which stores it's value as `u64`. An additional parameter was
added which indicate the signedness of the type and the sign of the value.
This removes a bunch of token types. Tokens now store the original, unaltered
numeric literal (that is still checked for correctness), which is parsed into
an actual number later, as needed, when creating the AST.
This can change how syntax extensions work, but otherwise poses no visible
changes.
[breaking-change]
This shuffles things around a bit so that LIT_CHAR and co store an Ident
which is the original, unaltered literal in the source. When creating the AST,
unescape and postprocess them.
This changes how syntax extensions can work, slightly, but otherwise poses no
visible changes. To get a useful value out of one of these tokens, call
`parse::{char_lit, byte_lit, bin_lit, str_lit}`
[breaking-change]
Rationale: for what appear to be historical reasons only, the PatIdent contains
a Path rather than an Ident. This means that there are many places in the code
where an ident is artificially promoted to a path, and---much more problematically---
a bunch of elements from a path are simply thrown away, which seems like an invitation
to some really nasty bugs.
This commit replaces the Path in a PatIdent with a SpannedIdent, which just contains an ident
and a span.
The lexer already ignores CRLF in between tokens, but it doesn't
properly handle carriage returns inside strings and doc comments. Teach
it to treat CRLF as LF inside these tokens, and to disallow carriage
returns that are not followed by linefeeds. This includes handling an
escaped CRLF inside a regular string token the same way it handles an
escaped LF.
This is technically a breaking change, as bare carriage returns are no
longer allowed, and CRLF sequences are now treated as LF inside strings
and doc comments, but it's very unlikely to actually affect any
real-world code.
This change is necessary to have Rust code compile on Windows the same
way it does on Unix. The mozilla/rust repository explicitly sets eol=lf
for Rust source files, but other Rust repositories don't. Notably,
rust-http cannot be compiled on Windows without converting the CRLF line
endings back to LF.
[breaking-change]
Use ty_rptr/ty_uniq(ty_trait) rather than TraitStore to represent trait types.
Also addresses (but doesn't close) #12470.
Part of the work towards DST (#12938).
[breaking-change] lifetime parameters in `&mut trait` are now invariant. They used to be contravariant.
This removes all remnants of `@` pointers from rustc. Additionally, this removes
the `GC` structure from the prelude as it seems odd exporting an experimental
type in the prelude by default.
Closes#14193
[breaking-change]
A number of functions/methods have been moved or renamed to align
better with rust standard conventions.
syntax::ext::mtwt::xorPush => xor_push
syntax::parse::parser::Parser => Parser::new
[breaking-change]
