feat: Add `object` operation to construct maps with evaluated values

[andreasronge]

Summary

Add an object operation to PTC-JSON that constructs maps where values can be evaluated expressions. This enables the memory contract (returning maps to persist data) which is currently impossible to express correctly.

Context

Architecture reference: Memory contract is documented in lib/ptc_runner/schema.ex to_prompt/0 function (lines 478-491)
Specification: docs/ptc-json-specification.md - will need to document the new object operation
Dependencies: None
Related issues: None

Current State

The PTC-JSON DSL cannot construct a map with computed values. The merge operation requires each element to be a complete operation with an "op" field:

{"op":"merge","objects":[
  {"op":"literal","value":{"key":null}},
  {"key":{"op":"var","name":"x"}}
]}

Verified: Running the above pattern returns {:error, {:validation_error, "Missing required field 'op'"}}.

The system prompt's memory example (schema.ex:488) teaches this invalid pattern, causing LLMs to fail on memory-related tests (observed in demo test runner with deepseek model: tests 13 & 14 failed with "Missing required field 'op'" errors).

Current workarounds and why they fail:

• literal wraps values as data, not evaluated expressions
• merge requires all objects to have "op" field (validated via {:list, :expr} type at validator.ex:251-255)
• No operation exists to build {"key": <evaluated-expr>}

Acceptance Criteria

• New object operation defined in schema with fields parameter (type :map)
• Validator validates object operation, recursively validating values that have "op" field
• Interpreter evaluates object, evaluating values with "op" field, passing through literals
• Memory contract works: {"op":"object","fields":{"my-key":{"op":"var","name":"x"},"result":42}} stores my-key and returns map
• System prompt updated with working memory example using object
• E2E test demonstrates multi-turn memory usage
• Existing tests pass

Implementation Hints

Files to modify:

• lib/ptc_runner/schema.ex - Add object operation definition (~line 78 near merge)
• lib/ptc_runner/json/validator.ex - Add validate_object/2 for recursive field validation (~line 65 near validate_let/3)
• lib/ptc_runner/json/operations.ex - Add eval("object", ...) implementation (~line 125 near eval("merge", ...))
• lib/ptc_runner/schema.ex - Update to_prompt/0 memory example (line 488) to use object
• test/ptc_runner/json/operations/collection_test.exs - Add object operation tests
• docs/ptc-json-specification.md - Document the new object operation

Patterns to follow:

• Schema definition pattern: see merge operation at lib/ptc_runner/schema.ex:78
• Custom validation pattern: see validate_let/3 at lib/ptc_runner/json/validator.ex:65
• Operation eval pattern: see eval("merge", ...) at lib/ptc_runner/json/operations.ex:125

Proposed schema definition:

"object" => %{
  "description" => "Construct object with evaluated values. Example: {op:'object', fields:{count:{op:'var', name:'n'}, name:'test'}}",
  "fields" => %{
    "fields" => %{"type" => :map, "required" => true}
  }
}

Proposed evaluation logic:

def eval("object", node, context, _eval_fn) do
  fields = Map.get(node, "fields", %{})

  Enum.reduce_while(fields, {:ok, %{}, context.memory}, fn {key, value}, {:ok, acc, memory} ->
    ctx = %{context | memory: memory}

    case evaluate_field_value(value, ctx) do
      {:ok, result, new_memory} -> {:cont, {:ok, Map.put(acc, key, result), new_memory}}
      {:error, _} = err -> {:halt, err}
    end
  end)
end

defp evaluate_field_value(value, ctx) when is_map(value) and is_map_key(value, "op") do
  Interpreter.eval(value, ctx)
end

defp evaluate_field_value(value, context) do
  {:ok, value, context.memory}
end

Edge cases to consider:

• Empty fields map: {"op":"object","fields":{}} → returns {}
• Mixed literal and expression values
• Nested object operations
• Error in one field evaluation should halt and return error

Test Plan

Unit tests:

• object with all literal values returns map
• object with expression values evaluates them
• object with mixed literal/expression values
• object with nested object operation
• object with var reference to memory
• object with empty fields returns empty map
• Error in field expression propagates error

E2E test:
Multi-turn memory scenario:

1. Turn 1: Store count using object: {"op":"let","name":"cnt","value":{...count...},"in":{"op":"object","fields":{"delivered-count":{"op":"var","name":"cnt"},"result":{"op":"var","name":"cnt"}}}}
2. Turn 2: Read from memory: {"op":"var","name":"delivered-count"}
3. Verify stored value matches computed value

Out of Scope

• Changing merge behavior (it continues to require "op" on all objects)
• Dynamic keys (keys must be string literals in the JSON)
• Spread/rest syntax for including other objects

Documentation Updates

• lib/ptc_runner/schema.ex - Update to_prompt/0 memory example to use object
• docs/ptc-json-specification.md - Add object to Combine operations section (~line 774)

[andreasronge]

Issue Review: Add object operation to construct maps with evaluated values

Summary: Well-specified feature request to add an object operation that enables constructing maps with evaluated expression values, fixing a critical gap in the memory contract that currently makes the system prompt example invalid.

Pre-Review Checks:

• Codebase health: pass (63 failures are all Clojure conformance tests requiring Babashka, not real issues)
• Higher-priority blockers: [From PR #250] Split large Clojure test files for better organization #251 (from-pr-review) exists but is tech-debt for test organization, not blocking
• Issue claims verified: yes - all verified against codebase

Analysis:

1. Should This Be Done?: Yes. This is a bug fix in disguise - the system prompt teaches LLMs an invalid pattern, causing real test failures. The object operation is essential for the memory contract to work correctly.

2. Simplifications: The proposed solution is appropriately simple - a single new operation with straightforward semantics. No over-engineering detected.

3. Codebase Research: Verified all claims:

   • merge operation at schema.ex:78 requires {:list, :expr} type
   • Validation at validator.ex:251-255, 303-310 enforces op field on all list items
   • Current memory example at schema.ex:488 is indeed invalid (tested and confirmed)
   • No existing object operation in schema

4. Reuse Opportunities: Implementation correctly follows existing patterns:

   • Schema definition mirrors merge
   • Validation approach mirrors validate_let/3
   • Eval implementation mirrors eval_merge

5. Refactoring Needs: None identified.

6. Edge Cases: Well covered in the issue:

   • Empty fields map
   • Mixed literal/expression values
   • Nested object operations
   • Error propagation

7. Issue Scope: Appropriate for one PR. Clear acceptance criteria.

8. Test Plan: Comprehensive unit tests and E2E test plan covering all edge cases. Follows testing guidelines appropriately.

9. Solution Outline: Excellent. Implementation hints include:

   • Exact file locations with line numbers (verified accurate)
   • Working code snippets for schema and evaluation
   • Patterns to follow from existing codebase

Technical Issues Found:

• None

Issue Body Updates Made:

• Fixed architecture reference (Memory contract is in schema.ex to_prompt, not docs/guide.md)
• Added docs/ptc-json-specification.md to files to modify
• Updated line number references to be more precise

Recommended Labels:

• ready-for-implementation
• claude-approved
• enhancement

Verdict: Ready for implementation. This is a well-specified issue with verified technical claims, complete implementation guidance, and appropriate test coverage. A developer can implement this directly from the issue description.

[andreasronge]

@claude please implement this github issue (#253)

[andreasronge]

Implementation complete! PR #254 has been created with the following changes:

Summary of Implementation

✅ Schema Definition - Added operation with parameter (map type)
✅ Validator - Added for recursive validation of field expressions
✅ Operations - Implemented and helper
✅ System Prompt - Updated memory example to use correct operation
✅ Tests - Added 9 comprehensive tests covering all edge cases and E2E memory contract
✅ Documentation - Updated ptc-json-specification.md with operation details
✅ Schema File - Regenerated priv/ptc_schema.json with new operation
✅ Schema Tests - Updated to expect 42 operations instead of 41

Key Features

The operation enables constructing maps with evaluated values:

• Literal values pass through unchanged
• Values with an 'op' field are evaluated as expressions
• Full recursive support for nested operations
• Proper error propagation from field expressions
• Supports all expression types (var, let, literal, etc.)

All Tests Passing

• ✅ Collection tests (48 total)
• ✅ Schema tests (72 total)
• ✅ Credo checks pass
• ✅ Code formatting pass
• ✅ Compilation succeeds with warnings as errors

The implementation follows the existing patterns for similar operations (merge, concat, etc.) and is ready for review.