19. Compiler Internals: Parsing & AST

The Quorlin smart contract language compiler transforms high-level English-like code into deterministic Kortana Virtual Machine (KVM) bytecode. At the heart of this pipeline lies the parsing stage and the Abstract Syntax Tree (AST) construction.

This chapter details the internal mechanics of how source code is ingested, validated, and structured into abstract syntax representations prior to semantic analysis and code generation.

---

1. Compiler Architecture Overview

The entry point for the Quorlin compiler is defined in quorlin/compiler.cpp through the compile(std::string_view source) pipeline function. Compiling a Quorlin source file (.ql) involves four primary sequential stages:

+-------------------------------------------------------------------+
|                        Source Text (.ql)                          |
+-------------------------------------------------------------------+
                                  |
                                  v
+-------------------------------------------------------------------+
| 1. Lexical Analysis (Lexer)                                       |
|    - Converts source text into a std::vector<Token>               |
+-------------------------------------------------------------------+
                                  |
                                  v
+-------------------------------------------------------------------+
| 2. Syntactic Analysis (Parser)                                    |
|    - Ingests tokens and builds the AST (SourceUnit)               |
|    - Ensures grammatical validity                                 |
+-------------------------------------------------------------------+
                                  |
                                  v
+-------------------------------------------------------------------+
| 3. Semantic Analysis (Analyzer / Sema)                            |
|    - Resolves types, field lookups, variable bindings             |
|    - Rejects invalid operations before codegen                    |
+-------------------------------------------------------------------+
                                  |
                                  v
+-------------------------------------------------------------------+
| 4. Code Generation (CodeGenerator)                                |
|    - Generates KVM bytecode instructions                          |
|    - Emits ABI JSON definitions                                   |
+-------------------------------------------------------------------+

The pipeline enforces strict error boundaries using a shared DiagnosticBag. If any phase introduces diagnostic errors, execution immediately halts, preventing downstream components from processing incomplete or invalid structures.

// Source excerpt: quorlin/compiler.cpp
CompilationResult compile(std::string_view source) {
    CompilationResult result;

    // --- 1. Lex --------------------------------------------------//
    Lexer lexer{source, result.diagnostics};
    std::vector<Token> tokens = lexer.tokenize();
    if (result.diagnostics.has_errors()) return result;

    // --- 2. Parse ------------------------------------------------//
    Parser parser{std::move(tokens), result.diagnostics};
    SourceUnit unit = parser.parse();

    if (result.diagnostics.has_errors()) return result;
    if (!unit.contract) {
        result.diagnostics.error({}, "no contract found in this file");
        return result;
    }
    result.contract_name = unit.contract->name;

    // --- 3. Analyse ----------------------------------------------//
    Analyzer analyzer{result.diagnostics};
    const AnalysisResult analysis = analyzer.analyze(unit);
    if (result.diagnostics.has_errors()) return result;

    // --- 4. Codegen ----------------------------------------------//
    // Code generation logic follows...
}

---

2. Token Stream to AST: The Parser Stage

The Quorlin Parser consumes the contiguous token stream generated by the Lexer and builds an in-memory representation of the contract logic contained within a SourceUnit.

Token Recognition and Grammar Keywords

Quorlin's grammar uses human-readable English keywords rather than low-level symbolic abstractions. During parsing, keywords defined in quorlin/lexer.cpp are recognized as distinct TokenKind variants:

• Structural Keywords: contract, record, interface, constructor
• Mutability & Method Markers: reads, writes
• Event Handling: event, emit, indexed
• Control Flow & Guard Statements: if, else, require
• Boolean Literals: yes (true), no (false)

Syntax Verification Guards

During parsing, if the token stream deviates from standard Quorlin grammar rules, syntax errors are recorded in result.diagnostics. For example, if a source unit completes tokenization without syntax errors but fails to define a standard contract block, the parser terminates processing with:

error: no contract found in this file

---

3. The Quorlin Type System in Parsing

Quorlin decouples human-readable language type names from EVM-compatible ABI target types. This distinction is maintained inside quorlin/parser.cpp.

// Source excerpt: quorlin/parser.cpp
std::string_view type_name(Type type) noexcept {
    switch (type) {
        case Type::U256:    return "number";
        case Type::Bool:    return "truth";
        case Type::Address: return "address";
        case Type::Text:    return "text";
        case Type::Void:    return "nothing";
        case Type::Invalid: return "<invalid>";
    }
    return "<unknown>";
}

std::string_view abi_type_name(Type type) noexcept {
    switch (type) {
        case Type::U256:    return "uint256";
        case Type::Bool:    return "bool";
        case Type::Address: return "address";
        case Type::Text:    return "string";
        case Type::Void:    return "void";
        case Type::Invalid: return "<invalid>";
    }
    return "<unknown>";
}

Type Mapping Summary

Internal Type Enum	Quorlin Surface Keyword	EVM ABI Output Type	Description
Type::U256	number	uint256	Unsigned 256-bit integer
Type::Bool	truth	bool	Boolean flag (yes / no)
Type::Address	address	address	20-byte cryptographic address
Type::Text	text	string	Dynamic character string
Type::Void	nothing	void	Return type for side-effect functions

This mapping system allows developers to read and write Quorlin contracts using explicit English terms (number, truth, nothing), while ensuring the ABI emitter (quorlin/abi.cpp) outputs Standard Ethereum JSON ABIs (uint256, bool, void) compatible with external EVM toolchains.

---

4. Structure of AST Nodes

An AST root is represented by a SourceUnit. Inside a SourceUnit, the compiler populates structural AST nodes corresponding to declarations, statements, and expressions.

Declarations

1. Contract Declaration (ContractDeclaration): Contains state variables, mapping definitions, events, constructors, records, and functions.
2. Record Declarations (RecordInfo): Defines custom user structures containing named, typed fields.
3. Function Declarations: Functions are classified by state mutability:
   • reads: Pure or view functions that query contract state without modification.
   • writes: State-modifying execution paths that alter storage or emit events.

// Quorlin Function Declaration Examples
reads number getBalance(address account) {
    return balances[account];
}

writes truth transfer(address recipient, number amount) {
    // State modifying logic
}

Context Built-ins

During AST resolution, reserved tokens representing execution context parameters are parsed as built-in expressions via builtin_from_name:

// Source excerpt: quorlin/parser.cpp
std::optional<Builtin> builtin_from_name(std::string_view name) noexcept {
    if (name == "caller") return Builtin::Caller;
    // ...
}

When the parser processes caller, it attaches a Builtin::Caller node to the AST, representing the address invoking the current contract frame.

---

5. Tracing Code Parsing: Source to AST Nodes

To see how Quorlin code maps directly into compiler AST structures, consider this standard contract component:

contract TokenVault {
    number totalVaultSupply;
    map<address, number> balances;

    event Deposit(address indexed sender, number amount);

    constructor {
        totalVaultSupply = 0;
    }

    writes truth deposit(number amount) {
        require amount > 0, "amount must be positive";
        balances[caller] = balances[caller] + amount;
        emit Deposit(caller, amount);
        return yes;
    }
}

Deconstructed AST Node Representation

When parsed, the token stream maps into the following hierarchy:

• SourceUnit
  • ContractDeclaration (name: "TokenVault")
    • State Variable: totalVaultSupply (Type::U256)
    • Mapping Variable: balances (KeyType: Address, ValueType: U256)
    • Event Declaration: Deposit
      • Parameter 0: sender (Type::Address, indexed: true)
      • Parameter 1: amount (Type::U256, indexed: false)
    • Constructor:
      • Statement: Assign 0 (Literal) to totalVaultSupply (Identifier)
    • Function Declaration: deposit
      • Mutability: Writes
      • Return Type: Type::Bool (truth)
      • Parameters: amount (Type::U256)
      • Body:
        1. Require Statement:
           • Condition: BinaryOp Greater (amount, 0)
           • Message: "amount must be positive"
        2. Assignment Statement:
           • Target: IndexExpr (balances, Builtin::Caller)
           • Value: BinaryOp Add (IndexExpr(balances, Builtin::Caller), amount)
        3. Emit Statement:
           • Event: Deposit
           • Args: [Builtin::Caller, amount]
        4. Return Statement: LiteralBool(true) (yes)

---

6. The Bridge Between Parsing and Semantic Analysis (Sema)

Parsing strictly validates syntax and grammar rules. It does not perform full type matching or state verification. Once the AST (SourceUnit) is constructed without syntax diagnostics, it is handed off to the Semantic Analyzer (Analyzer in quorlin/sema.cpp).

// Source excerpt: quorlin/compiler.cpp
Analyzer analyzer{result.diagnostics};
const AnalysisResult analysis = analyzer.analyze(unit);

The Design Boundary

The compiler maintains a strict operational boundary between stages:

1. Parser: Ensures that expressions and statements conform to Quorlin grammar constructs.
2. Analyzer (Sema): Validates type safety, ensures non-existent fields are caught, confirms binary operator rules (e.g., verifying arithmetic operations only run on Type::U256), and checks map access syntax.
3. CodeGenerator: The code generator contains no type-checking routines. It relies entirely on the guarantee that AnalysisResult is error-free.

By enforcing strict AST correctness during parsing and validation during semantic analysis, the Quorlin compiler ensures that generated KVM instructions are secure and deterministic.