#1. Lexical structure

This chapter covers the rules for breaking rok source code into tokens. It examines every element encountered during tokenization, starting with files and lines and continuing through identifiers, keywords, literals and operator tokens.

Higher layers such as types, expressions and statements receive input that has already been tokenized, so all rules that apply during tokenization are gathered in this one chapter.

#Contents

• 1.1. Source files. Packages, directories, .rok files.
• 1.2. Lexical structure. Lines, whitespace, comments.
• 1.3. Statement terminator. ; and insertion rules.
• 1.4. Identifiers. Name format.
• 1.5. Keywords. List of reserved words.
• 1.6. Numeric literals.
• 1.7. String literals.
• 1.8. Character literals.
• 1.9. Boolean and sentinel literals. true, false, none.
• 1.10. Punctuation and operator tokens.

Edit this page on GitHub