Skip to content

Commit 7bd81a6

Browse files
committed
tokens.md: say that lifetime-like tokens can't be immediately followed by '
Forms like 'ab'c are rejected, so we need some way to explain why they don't tokenise as two consecutive LIFETIME_OR_LABEL tokens. Address this by adding "not immediately followed by `'`" to each of the lexer rules for the lifetime-like tokens. This also means there can be no ambiguity between CHAR_LITERAL and these tokens (at present we don't say how such ambiguities are resolved).
1 parent 1afcfd9 commit 7bd81a6

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

src/tokens.md

+4-1
Original file line numberDiff line numberDiff line change
@@ -630,11 +630,14 @@ Examples of reserved forms:
630630

631631
> **<sup>Lexer</sup>**\
632632
> LIFETIME_TOKEN :\
633-
> &nbsp;&nbsp; &nbsp;&nbsp; `'` [IDENTIFIER_OR_KEYWORD][identifier]\
633+
> &nbsp;&nbsp; &nbsp;&nbsp; `'` [IDENTIFIER_OR_KEYWORD][identifier]
634+
> _(not immediately followed by `'`)_\
634635
> &nbsp;&nbsp; | `'_`
636+
> _(not immediately followed by `'`)_
635637
>
636638
> LIFETIME_OR_LABEL :\
637639
> &nbsp;&nbsp; &nbsp;&nbsp; `'` [NON_KEYWORD_IDENTIFIER][identifier]
640+
> _(not immediately followed by `'`)_
638641
639642
Lifetime parameters and [loop labels] use LIFETIME_OR_LABEL tokens. Any
640643
LIFETIME_TOKEN will be accepted by the lexer, and for example, can be used in

0 commit comments

Comments
 (0)