Repository navigation
Description
Status: Open — no constraint check in the ReAct answer path.
Impact: Syntactically valid SQL can miss business requirements, and prediction selection can widen scope or accept rows for an empty one.
Evidence
The ReAct agent's run_sql and submit check that a query is a single read, that its tables and columns exist, and that its tables were retrieved. Nothing extracts or checks the question's requirements, and predict writes PQL from the question with the same gap for populations.
Tested on main with the default model:
- Explicit filters held. Status, order type, customer tier, a date window, a distinct count, and a top-N ranking were all exact.
- Literal types aren't checked. On a SQLite source whose columns have no declared type, the model wrote
site_id = '1'for an integer column. SQLite matched nothing, and the agent answered "0 sales orders… no orders were placed" for a site with 9. - Nothing re-checks an empty result before it's reported as a fact.
Actions
Extract formula, literal, temporal, projection, distinctness, ordering, measure and unit, source-role, and population requirements once, from the question and the retrieved definitions. Check submit and predict against them deterministically, and return any failure to the model as a tool error so it revises.
Bind each literal to its column's stored type: use the catalog type, or the sampled type when a SQLite column has no declared one. Treat an empty result from a filtered query as unverified, and probe the filter values before answering. Keep an explicit failure when the evidence is unavailable or the tool budget runs out.
Acceptance
Cover quantity versus share, distinct artifact roles, and same-named legacy and case attributes with disjoint IDs; check exact keys, columns, values, and policy lineage. Retain the intended shortage-child interventions, the cheaper base-child control, and the invalid or superseded versions from #266, enforcing parent and scenario, status, validity, supersession, and commitment. Check predict populations, including empty and out-of-graph selections. Include untyped SQLite columns compared with quoted numeric literals, and an empty result whose filter value exists under another type.
#253 owns required sources, schema, and paths; #263 owns fanout; #334 owns explicit origin.
Review (a630ebd9): Explicit filters and the literal-type miss reproduced on a SQLite supply-chain dataset with nvidia/nemotron-3.5-lightning-30b-a3b.
Activity
[-][P0] Validate material question intent before execution[/-][+]Validate material question intent before execution[/+][-]Validate material question intent before execution[/-][+]Enforce material query constraints before execution[/+]cafzal commentedon Nov 18, 2026
@tomer-levin-nv @liavnave @DinaLaptii, flagging the part of this that's unowned: nothing re-checks an empty result. On
main, "How many sales orders came from site S0001?" ransite_id = '1'against an untyped SQLite integer column, got 0 rows, and answered "0 sales orders… no orders were placed" (the site has 9). The old pipeline probed filter values whenever a query came back empty; ReAct doesn't, and nothing in flight adds it. The smallest fix would treat an empty result from a filtered query as unverified, and probe the filter values once beforesubmitaccepts it. That's in the loop you're changing, so who should own it? I'm looking at the literal-type half separately in the SQLite connector, outside ReAct.