Repository navigation
Feat/deepagent sql construction - #391
Draft
tomer-levin-nv wants to merge 6 commits into
Draft
tomer-levin-nv wants to merge 6 commits into
tomer-levin-nv wants to merge 6 commits into
Conversation
Construction was one LLM call that never saw the database: a wrong filter literal or a misread question could only be caught downstream, after the query had already failed validation. These arms let the model look before it commits, swapped in behind the existing construct_sql_from_candidates node so validation, execution and reconstruction cannot tell which one ran. - sql_agent/loop.py bounds a tool-calling conversation on turns, database calls and wall clock at once, returning the best query seen rather than nothing when any of them expires. - sql_agent/deep.py holds several readings of a question, probes for the evidence separating them and prefers the query that assumes less, which is the only thing the candidate-pool measurement left room for. It returns structured output rather than calling a submit tool, removing both the missed-termination and the collapsed-argument failure modes. - sql_agent/tools.py is the shared toolbox; elaboration.py and the submission gate apply the checks that catch scoring errors to every submission, since a check the model may decline is not a check. - Arms are off unless SQL_AGENT_ENABLED or SQL_DEEP_AGENT_ENABLED says otherwise, so an unconfigured run keeps today's single call and the arms can be A/B'd rather than branched. - state carries reasoning_llm so construction can take the strong model while the rest of the graph stays on the cheap one.
…nstruction # Conflicts: # auto_ontology/dal/custom_analyses.py # auto_ontology/retrieval/text_to_sql/agents/sql_from_semantic.py # auto_ontology/retrieval/text_to_sql/main.py # auto_ontology/retrieval/text_to_sql/state.py # auto_ontology/server/datasources/service.py # auto_ontology/utils/llm_invoke.py # pyproject.toml
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Comment |
ensemble.py groups the deep agent's candidate SQL by the rows each one returns and votes over the groups; config.py carries the arm switches and weights, and the graph wires the pool through the construction node.
… into feat/deepagent-sql-construction # Conflicts: # auto_ontology/retrieval/text_to_sql/tests/test_sql_reconstruction.py
The configuration behind the full BIRD dev Gym run (1,115 vs 1,078 for the schema-only control, +37 over 1,534 questions). - Submission gate: add the wording check, which flags an unused column whose description shares a 3+ word phrase with the question. On a 165-question slice the deep agent scored 104 with it and 98 without. - Deep agent: bounce a gated submission back up to SQL_AGENT_MAX_BOUNCES times instead of accepting the first answer, and merge the leading system messages into one block so the endpoint's prompt cache applies. - Prompts: date without the time of day, so the prompt prefix is cacheable; the "Semantically Important Columns" section off unless SQL_SEMANTIC_COLUMNS is set; shorter guidance for the example SQL pairs. - main.py: keep the full-pipeline retry, and fall back to the reasoning client when no non-reasoning model is configured instead of handing every node a null client.
Collaborator
|
@tomer-levin-nv, suggestion: close, or rebuild on ReAct. #488 removed |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.