[Phase 5] Add typo suggestions for unknown operations

[andreasronge]

Summary

Add typo suggestions to validation error messages using Jaro-Winkler string similarity. When a user provides an unknown operation or field name, the error message will suggest the closest valid alternatives (e.g., "Unknown operation 'filer'. Did you mean 'filter'?"). This helps LLMs self-correct their generated programs.

Context

Architecture reference: Phase 5: Polish - "Error messages optimized for LLM consumption", "Validation with helpful suggestions"
Dependencies: None - Phase 1-4 are complete
Related issues: None

Current State

Verified via codebase analysis:

Unknown operations (lib/ptc_runner/validator.ex:25):

"Unknown operation '#{op}'"

Unknown comparison fields (lib/ptc_runner/validator.ex:204):
When field name typos occur, no suggestion is given - just a generic error.

Existing validation covers all operations (literal, load, var, let, if, and, or, not, merge, concat, zip, pipe, filter, map, select, eq, neq, gt, gte, lt, lte, contains, get, sum, count, avg, min, max, first, last, nth, reject, call) but provides no "did you mean?" suggestions.

Acceptance Criteria

• Unknown operation names suggest the closest valid operation (if similarity > 0.8)
  • Example: "Unknown operation 'filer'. Did you mean 'filter'?"
  • Example: "Unknown operation 'selct'. Did you mean 'select'?"
  • Example: "Unknown operation 'xyz'" (no suggestion if nothing is close)
• Suggestion uses Jaro-Winkler similarity algorithm (built into Elixir's String module)
• Only suggests when similarity is high enough (> 0.8 threshold) to avoid confusing suggestions
• E2E test demonstrates typo correction feedback
• Existing tests pass
• No external dependencies added (use Elixir stdlib)

Implementation Hints

Files to modify:

• lib/ptc_runner/validator.ex - Add suggestion helper function, update unknown operation error
• test/ptc_runner/validator_test.exs - Add tests for typo suggestions

Patterns to follow:
Use String.jaro_distance/2 which is built into Elixir:

String.jaro_distance("filer", "filter")
# => 0.9555555555555556

Key implementation:

@valid_operations ~w(literal load var let if and or not merge concat zip pipe filter map select eq neq gt gte lt lte contains get sum count avg min max first last nth reject call)

defp suggest_operation(unknown_op) do
  @valid_operations
  |> Enum.map(fn valid -> {valid, String.jaro_distance(unknown_op, valid)} end)
  |> Enum.max_by(fn {_op, score} -> score end)
  |> case do
    {suggested, score} when score > 0.8 -> " Did you mean '#{suggested}'?"
    _ -> ""
  end
end

Edge cases to consider:

• Very short operation names (1-2 chars) may have misleading similarity scores
• Case sensitivity: should "FILTER" suggest "filter"? (recommend: yes, lowercase compare)
• Empty string input: should return no suggestion

Test Plan

Unit tests:

• "filer" suggests "filter" (common typo)
• "selct" suggests "select" (missing letter)
• "filtter" suggests "filter" (extra letter)
• "xyz" has no suggestion (nothing close)
• "FILTER" suggests "filter" (case insensitive)
• "lit" suggests "literal" only if score > 0.8 (may not meet threshold - verify)

E2E test:

test "provides helpful typo suggestion for misspelled operation" do
  program = ~s({"op": "filer", "where": {"op": "eq", "field": "x", "value": 1}})
  assert {:error, {:validation_error, msg}} = PtcRunner.run(program)
  assert msg =~ "Did you mean 'filter'?"
end

Out of Scope

• Field name suggestions (e.g., "wher" -> "where") - separate issue
• Required field suggestions (e.g., "literal requires 'value'") - separate issue
• Path context in error messages (e.g., "at steps[2]") - separate issue
• Structured JSON error output - separate issue
• Parse error improvements - separate issue

[andreasronge]

Issue Review: [Phase 5] Add typo suggestions for unknown operations

Summary: Well-specified issue for adding Jaro-Winkler-based typo suggestions to unknown operation error messages. The implementation approach is verified against the codebase and ready for implementation.

Analysis:

1. Should This Be Done?: Yes. This directly supports the Phase 5 goal of "Error messages optimized for LLM consumption." Typo suggestions help LLMs self-correct, reducing roundtrips. The benefit is clear and maintenance cost is low (single helper function).

2. Simplifications: The approach is already minimal - single helper function using stdlib. No simplification needed.

3. Codebase Research: Verified via exploration:

   • Unknown operation error is at lib/ptc_runner/validator.ex:336-338
   • 34 valid operations are defined across the validator (literal, load, var, let, if, and, or, not, merge, concat, zip, pipe, filter, map, select, eq, neq, gt, gte, lt, lte, contains, get, sum, count, avg, min, max, first, last, nth, reject, call)
   • Error format is {:error, {:validation_error, message}} - consistent throughout
   • String.jaro_distance/2 is confirmed available in Elixir 1.19+ stdlib

4. Reuse Opportunities:

   • Existing helper function pattern with require_field/3 and validate_aggregation_field/2
   • Can follow same pattern for suggest_operation/1

5. Refactoring Needs: None. The catch-all clause at line 336 is the perfect insertion point.

6. Edge Cases: Issue addresses these well:

   • Short operation names (1-2 chars): threshold guards against misleading matches
   • Case sensitivity: recommends lowercase comparison ✓
   • Empty string: threshold naturally handles this
   • No security concerns - this only affects validation error messages

7. Issue Scope: Appropriately scoped. Single feature, ~20 lines of implementation code, clear boundaries. Out-of-scope items are explicitly listed.

8. Test Plan: Comprehensive test plan covering:

   • Common typos (missing letter, extra letter)
   • Edge cases (no suggestion for very different names)
   • Case insensitivity
   • E2E integration test

   One minor suggestion: Consider adding a test for empty string operation name, though this may already be caught earlier in validation.

9. Solution Outline: Clear and actionable:

   • Files to modify identified correctly
   • Code snippets provided and verified against codebase
   • Implementation is copy-paste ready
   • A good junior developer could implement this

Technical Issues Found:

• None. The issue references line 25 for unknown operations but it's actually line 336 - however, this is a minor documentation inconsistency and the code snippet provided is accurate.

Recommended Labels:

• ready-for-implementation - Issue is well-defined and verified
• enhancement - Already present

Verdict: Ready for implementation

[andreasronge]

@claude Please implement this issue:

1. First, read the review comments on this issue
2. If the review identified issues (inconsistencies, missing details, etc.), update the issue description to address them
3. Implement the issue following the (updated) specification
4. Create a PR with your implementation

Make sure your implementation addresses any concerns raised in the review.

[andreasronge]

Implementation Complete

Successfully implemented Jaro-Winkler typo suggestions for unknown operations as specified in the issue.

What Was Implemented

1. Core Feature:

   • Added suggest_operation/1 helper function in lib/ptc_runner/validator.ex
   • Uses String.jaro_distance/2 to find closest matching operation
   • Only suggests when similarity exceeds 0.8 threshold
   • Handles case-insensitive matching

2. Tests Added:

   • 5 unit tests covering typos (missing letter, extra letter, case sensitivity, no match)
   • 1 E2E test demonstrating suggestion in complex pipeline with filter operation

3. Validation:

   • All 242 tests pass ✓
   • No code style issues ✓
   • Precommit checks successful ✓

Examples

• "filer" → suggests "filter" (score: 0.955)
• "selct" → suggests "select" (score: 0.944)
• "filtter" → suggests "filter" (score: 0.945)
• "xyz" → no suggestion (score: 0.0, below 0.8 threshold)

Changes

Files Modified:

• lib/ptc_runner/validator.ex - Added helper and updated error message
• test/ptc_runner_test.exs - Added comprehensive tests

Commit:

• 9296641 - feat: add Jaro-Winkler typo suggestions for unknown operations

PR:

• feat: add Jaro-Winkler typo suggestions for unknown operations (Fixes #43) #44

The implementation addresses all acceptance criteria from the issue and handles edge cases appropriately.