Skip to content

eo-parser is cluttered with redundant rules due to ANTLR's limitations in handling EO syntax #5078

Description

@maxonfjvipon

The current ANTLR grammar has grown into an unmaintainable mess, and the root cause is structural: ANTLR is the wrong tool for EO syntax.

Two specific problems drive this:

  1. Indentation. ANTLR has no native support for indent-sensitive grammars. The current workaround injects synthetic TAB/UNTAB tokens via a custom hand-written lexer (EoIndentLexer). This is fragile, hard to debug, and means the grammar implicitly depends on lexer behavior isn't expressed in the grammar itself.

  2. Horizontal/vertical notation. EO allows the same structure to be written on one line or spread across multiple indented lines. ANTLR cannot express "same construct, different layout" without duplicating rules. The result is every rule family exists twice:
    happlication/vapplication, hmethod/vmethod, hformation/etc. — each pair semantically identical but syntactically distinct. The t* family (tbound, tformation, tapplication, ...) is the same problem: semantic constraints on test objects forced into grammar rules, doubling the rule
    count again.

The grammar is currently ~600 lines for a language whose core constructs fit on one page.

Proposed solution

A stack-based line-by-line parser:

  1. Parse each line independently into one of a small set of typed statements (formation, dispatch/application, dispatch continuation, reversed dispatch head)
  2. Use indentation to establish parent-child relationships in the XML tree — more indent = child, same indent = sibling, less indent = close
    levels and resume
  3. Move semantic constraints (test objects can't be top-level, can't nest) to a post-parse validator, not the grammar

This separates concerns cleanly: structure from indentation, syntax from semantics, core language from sugar. Adding new sugar means adding a new line type and a desugaring pass — not rewriting half the grammar.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions