Skip to content

✨ agents: add the Codegen specialist, which runs the code it writes - #65

Merged
Jaweria-B merged 2 commits into
mainfrom
feature/codegen-agent
Aug 1, 2026
Merged

Jaweria-B merged 2 commits into
mainfrom
feature/codegen-agent

Conversation

@Jaweria-B

@Jaweria-B Jaweria-B commented Aug 1, 2026 •

Copy link
Copy Markdown
Collaborator

Completes the coding engine: #63 taught the agents to write AiiDA code, #64 built somewhere safe to run it, and this connects them.

Why

The tool surface is fixed, and some questions do not fit it. "Which structures in group screening-2026 contain Ti, and what were their final energies" is three filters and a projection. The QueryBuilder expresses it in six lines; no tool enumerates it. Adding a tool per combination does not converge — the first one nobody anticipated breaks immediately, and the tool list becomes unreadable to the model long before it becomes complete.

So writing the query is the feature. Running it is what makes the feature trustworthy — a snippet nobody executed is a guess with syntax highlighting.

The loop

search_aiida_code → write the smallest snippet → run_aiida_code
                                                       ↓
                           traceback ──→ fix it, run again
                                                       ↓
                    show the user the code AND what it returned

The agent fixes its own mistakes from a real traceback before the user sees anything.

The decision most worth arguing with

run_aiida_code is not approval-gated.

It holds no capability to write — it runs against a profile whose Postgres role has no INSERT privilege — so there is nothing for an approval to protect.

And the alternative is weaker than it looks. A prompt showing twenty lines of unexecuted Python asks a researcher to audit code under time pressure, which is a far worse check than it appears to be. Letting the code run where it can do no harm and showing what it returned converts the same decision into one made from evidence.

Everything that genuinely writes stays on the Execution agent, gated, where the preview is a resolved input a researcher can judge in seconds.

Why a third specialist, when Diagnosis stayed a tool

docs/extending.md requires both an own tool surface and an own prompt:

Own tool surface Own prompt
Diagnosis No — Analysis already held the tools No
Codegen Yes — code execution's blast radius differs in kind from a 12-tool read-only surface Yes — write/run/read-traceback/retry is not "explain what is in a database"

ADR-09's "cap at two specialists" now carries a revision note: the structural decisions are untouched — one routing path, specialists never call each other, approval still enforced by requires_approval, MAX_STEPS unchanged. Only the size of the set the planner chooses from moved.

Two things the tests caught

The pinned MCP surface test failed, because run_aiida_code is discovered but not exported. That was the right catch, and the answer is that it should stay unexported: its safety rests entirely on AIIDA_AGENTS_SANDBOX_PROFILE naming a profile someone verified with sandbox check. An MCP client cannot check that and we cannot see whether it holds. The test now carries a third category, _UNEXPORTED_READ_TOOLS, with the reason beside it — so exporting it later has to be deliberate.

It is inert until aiida-agents sandbox init has been run. With no sandbox profile configured the tool refuses, and instructs the model to say the snippet is unverified rather than falling back to the user's writable profile. That silent fallback is the one failure that would make everything else here worthless, and there is a test on it.

Testing

886 pass (+34), mypy clean across 146 files, ruff and mdformat green.

Not yet exercised against a real model — that needs a configured sandbox profile and provider quota.

Docs

New ADR-11 on executing generated code, including the alternatives rejected (approval-gating it, a scratch profile with copied data, a restricted interpreter). Architecture updated for the third specialist; ADR-09 gained the revision note.

The tool surface is fixed, and some questions do not fit it. "Which
structures in group screening-2026 contain Ti, and what were their final
energies" is three filters and a projection; the QueryBuilder expresses
it in six lines and no tool enumerates it. Adding a tool per combination
does not converge -- the first one nobody anticipated breaks immediately.

So writing the query is the feature, and running it is what makes the
feature trustworthy: a snippet nobody executed is a guess with syntax
highlighting. The agent looks the API up, writes the smallest thing that
answers the question, runs it, and fixes its own mistakes from the real
traceback before the user is shown anything.

Execution is not approval-gated, which is the decision most worth
arguing with. The tool holds no capability to write -- it runs against a
profile whose Postgres role has no INSERT privilege -- so there is
nothing for an approval to protect. And the alternative is weaker than it
looks: a prompt showing twenty lines of unexecuted Python asks a
researcher to audit code under time pressure, which is a far worse check
than it appears. Letting it run where it can do no harm and showing what
it returned makes the same decision one from evidence. Everything that
genuinely writes stays on the Execution agent, gated, where the preview
is a resolved input judged in seconds.

A third specialist on both grounds extending.md requires: its own tool
surface, since code execution's blast radius differs in kind from a
read-only surface, and its own prompt, since write-run-read-retry is a
different job from explaining what is in a database. Diagnosis was
considered for the same promotion and stayed a tool, needing neither.

Deliberately not exported over MCP. The safety of run_aiida_code rests
entirely on AIIDA_AGENTS_SANDBOX_PROFILE naming a profile someone
verified; an MCP client cannot check that and we cannot see whether it
holds. The pinned MCP surface test now carries that exclusion by name, so
exporting it later has to be deliberate.

Inert until `aiida-agents sandbox init` has been run: with no sandbox
profile configured the tool refuses, and says the snippet is unverified
rather than falling back to the user's writable profile. That silent
fallback is the one failure that would make all of the above worthless.
@coderabbitai

coderabbitai Bot commented Aug 1, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@Jaweria-B, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 45 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 76945362-15c4-4be2-88b1-f509c02804fb

📥 Commits

Reviewing files that changed from the base of the PR and between f49f9a3 and a7e82e9.

📒 Files selected for processing (1)
  • docs/adr/11-code-execution.md
📝 Walkthrough

Walkthrough

This change adds a Codegen specialist that writes and executes read-only Python against a configured AiiDA sandbox. It updates planner, factory, and CLI routing, excludes execution from MCP, adds sandbox settings, and documents the architecture and safety model.

Changes

Codegen execution architecture

Layer / File(s) Summary
Sandbox execution contract and tool
src/aiida_agents/_settings.py, src/aiida_agents/tools/codegen/*, docs/adr/11-code-execution.md, tests/agents/codegen/test_codegen.py
Adds sandbox profile and timeout settings. Adds run_aiida_code, profile validation, timeout handling, execution results, and safety tests.
Codegen agent and specialist routing
src/aiida_agents/agents/codegen/*, src/aiida_agents/agents/__init__.py, src/aiida_agents/agents/planner/*, src/aiida_agents/cli/agent.py, tests/agents/codegen/test_codegen.py
Adds the Codegen agent, prompt, read-only tools, factory registration, planner routing, CLI selection, and wiring tests.
Architecture records and MCP boundary
docs/adr/09-agent-orchestration.md, docs/adr/11-code-execution.md, docs/adr/README.md, docs/architecture.md, tests/mcp/test_server.py
Documents Codegen and its execution model. Updates the ADR index and verifies that run_aiida_code is not registered through MCP.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant Planner
  participant CodegenAgent
  participant run_aiida_code
  participant AiiDASandbox
  User->>Planner: submit code or multi-condition data query
  Planner->>CodegenAgent: select codegen specialist
  CodegenAgent->>CodegenAgent: search code and write minimal Python
  CodegenAgent->>run_aiida_code: execute generated Python
  run_aiida_code->>AiiDASandbox: verify profile and run with timeout
  AiiDASandbox-->>run_aiida_code: return output or error
  run_aiida_code-->>CodegenAgent: return execution summary
  CodegenAgent-->>User: report verified code and output
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 68.42% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: adding a Codegen agent that executes generated Python code.
Description check ✅ Passed The description directly explains the Codegen agent, sandboxed execution, integrations, safety controls, documentation, and tests.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Jaweria-B Jaweria-B changed the title Add Codegen agent for executing generated Python code ✨ agents: add the Codegen specialist, which runs the code it writes Aug 1, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
docs/adr/README.md (1)

29-34: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Fix the duplicate ADR-11 number.

Line 30 adds [11](/docs/adr/11-code-execution.md) for "Executing generated code," but Line 31 already uses the plain number 11 for the unrelated "Agent-run provenance" entry. Two different decisions now share the same ADR number in the log table. The status text on Line 34 ("ADR-11 is still exploratory and has no record yet") is also now stale: the new ADR-11 is Accepted, not exploratory.

Renumber the "Agent-run provenance" row (for example, to 12) and update the status line to reflect that ADR-11 (code execution) is written and Accepted.

📝 Proposed fix
 | [10](/docs/adr/10-plugin-extensibility.md)            | Plugin extensibility through one `aiida_agents.plugins` entry point                            |
 | [11](/docs/adr/11-code-execution.md)                  | Executing generated code against a write-refusing database role                                |
-| 11                                                    | Agent-run provenance: persist agent decisions/traces in AiiDA's provenance graph (exploratory) |
+| 12                                                    | Agent-run provenance: persist agent decisions/traces in AiiDA's provenance graph (exploratory) |

 ADR-01 is in effect; ADR-02 and ADR-03 are seeds with direction confirmed (2026-05-22).
-ADR-04 through ADR-10 are written; ADR-11 is still exploratory and has no record yet.
+ADR-04 through ADR-11 are written; ADR-12 is still exploratory and has no record yet.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/adr/README.md` around lines 29 - 34, Resolve the duplicate ADR number in
the README table by renumbering the Agent-run provenance entry to the next
available ADR number, such as 12, while keeping the code-execution entry as
ADR-11. Update the following status text so ADR-11 is described as written and
Accepted rather than exploratory, and adjust the exploratory/no-record wording
to refer to the renumbered provenance ADR.
src/aiida_agents/agents/codegen/prompt.md (1)

62-64: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Label the planner example fence.

The new fenced block has no language identifier. Use text or plaintext to satisfy Markdown linting.

-```
+```text
 codegen: find every relaxation in group `screening-2026` whose final structure contains Ti, and report their total energies

<details>
<summary>🤖 Prompt for AI Agents</summary>

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @src/aiida_agents/agents/codegen/prompt.md around lines 62 - 64, Update the
new planner example fenced block in the prompt to include the text language
identifier, using text or plaintext, while preserving the example content
unchanged.


</details>

<!-- cr-comment:v1:7834d2f17c29d7ff310badb3 -->

_Source: Linters/SAST tools_

</blockquote></details>

</blockquote></details>
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/aiida_agents/agents/codegen/__init__.py`:
- Around line 17-24: Verify and enforce the complete read-only boundary for the
Codegen agent before relying on its ungated execution: constrain all database
write operations, filesystem changes, process execution, and network side
effects in the runner, not only INSERT statements. Update the module’s prompt
and documentation claims to match the controls that are actually enforced, and
retain the approval gate until those protections are verified.

In `@src/aiida_agents/agents/planner/prompt.md`:
- Around line 62-64: Update the Markdown code fence surrounding the Codegen
example in the planner prompt to include a text language identifier, using
either text or plaintext, while preserving the example content unchanged.
- Around line 19-26: Update the specialist-count heading near the `codegen`
section from “The two specialists” to reflect the three specialists defined in
the planner prompt, without changing the surrounding specialist descriptions.

In `@src/aiida_agents/cli/agent.py`:
- Around line 51-54: Update `_as_specialist_literal` to preserve the `"codegen"`
selection instead of mapping it to `"analysis"`, while retaining the existing
`"execution"` and analysis behavior. Add a CLI test covering `--agent codegen`
and verify it routes to the Codegen specialist.

In `@src/aiida_agents/tools/codegen/execution.py`:
- Around line 30-35: Update the broad exception handler in the profile-checking
function around get_config to satisfy Ruff’s BLE001 rule while preserving the
intentional fallback of returning False for malformed or unreadable AiiDA
configuration. Prefer narrowing the exception if the relevant failures are
known; otherwise add an inline BLE001 suppression with a brief justification
alongside the existing coverage pragma.

---

Outside diff comments:
In `@docs/adr/README.md`:
- Around line 29-34: Resolve the duplicate ADR number in the README table by
renumbering the Agent-run provenance entry to the next available ADR number,
such as 12, while keeping the code-execution entry as ADR-11. Update the
following status text so ADR-11 is described as written and Accepted rather than
exploratory, and adjust the exploratory/no-record wording to refer to the
renumbered provenance ADR.

In `@src/aiida_agents/agents/codegen/prompt.md`:
- Around line 62-64: Update the new planner example fenced block in the prompt
to include the text language identifier, using ```text or ```plaintext, while
preserving the example content unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 43458689-7efb-4d54-8a75-94be20552546

📥 Commits

Reviewing files that changed from the base of the PR and between 20780c1 and f49f9a3.

📒 Files selected for processing (16)
  • docs/adr/09-agent-orchestration.md
  • docs/adr/11-code-execution.md
  • docs/adr/README.md
  • docs/architecture.md
  • src/aiida_agents/_settings.py
  • src/aiida_agents/agents/__init__.py
  • src/aiida_agents/agents/codegen/__init__.py
  • src/aiida_agents/agents/codegen/prompt.md
  • src/aiida_agents/agents/planner/__init__.py
  • src/aiida_agents/agents/planner/prompt.md
  • src/aiida_agents/cli/agent.py
  • src/aiida_agents/tools/codegen/__init__.py
  • src/aiida_agents/tools/codegen/execution.py
  • tests/agents/codegen/__init__.py
  • tests/agents/codegen/test_codegen.py
  • tests/mcp/test_server.py

Comment on lines +17 to +24
What it does *not* have is a write tool, and that is deliberate. Executing
generated code sounds like the most dangerous thing in this project, and it
would be if it ran against the user's live profile. It does not: it runs
against one whose database role has no INSERT privilege, so the danger is
handled at the database rather than by asking a user to vet Python at a prompt.
Approving twenty lines of unexecuted code is a weak check. Reading what the
code actually returned is a real one --- which is why this agent is not gated
and every tool that genuinely writes still is, on the Execution agent, where

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

rg -n -C 4 \
  'INSERT|UPDATE|DELETE|TRUNCATE|ALTER|CREATE|DROP|GRANT|REVOKE|read.?only|readonly|subprocess|socket|open\(|Path\(|os\.|shutil|exec\(|eval\(' \
  src/aiida_agents || true

Repository: aiidateam/aiida-agents

Length of output: 50380


🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

printf '--- candidate files ---\n'
git ls-files | rg '(^src/aiida_agents/agents/codegen/__init__\.py$|^src/aiida_agents/sandbox/(runner|guard|setup)\.py$|^src/aiida_agents/agents/codegen/)' || true

printf '\n--- codegen prompt init sections ---\n'
sed -n '1,140p' src/aiida_agents/agents/codegen/__init__.py

printf '\n--- sandbox guard ---\n'
sed -n '1,240p' src/aiida_agents/sandbox/guard.py

printf '\n--- sandbox runner relevant lines ---\n'
sed -n '80,180p' src/aiida_agents/sandbox/runner.py

printf '\n--- sandbox setup read-only role/probe ---\n'
sed -n '34,195p' src/aiida_agents/sandbox/setup.py

printf '\n--- read-only execution CLI usage ---\n'
sed -n '1,220p' src/aiida_agents/cli/sandbox.py

Repository: aiidateam/aiida-agents

Length of output: 251


🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

printf '%s\n' '--- candidate files ---'
git ls-files | rg '(^src/aiida_agents/agents/codegen/__init__.py$|^src/aiida_agents/sandbox/(runner|guard|setup).py$|^src/aiida_agents/agents/codegen/)' || true

printf '\n%s\n' '--- codegen __init__.py relevant section ---'
sed -n '1,140p' src/aiida_agents/agents/codegen/__init__.py

printf '\n%s\n' '--- sandbox guard ---'
sed -n '1,260p' src/aiida_agents/sandbox/guard.py

printf '\n%s\n' '--- sandbox runner relevant lines ---'
sed -n '80,180p' src/aiida_agents/sandbox/runner.py

printf '\n%s\n' '--- sandbox setup read-only role/probe ---'
sed -n '34,210p' src/aiida_agents/sandbox/setup.py

printf '\n%s\n' '--- read-only execution CLI usage ---'
sed -n '1,220p' src/aiida_agents/cli/sandbox.py

Repository: aiidateam/aiida-agents

Length of output: 22868


🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

printf '%s\n' '--- codegen prompt ---'
sed -n '1,260p' src/aiida_agents/agents/codegen/prompt.md

printf '\n%s\n' '--- run_aiida_code codegen tool ---'
sed -n '1,220p' src/aiida_agents/tools/codegen.py

printf '\n%s\n' '--- static verifier: references to sandbox APIs/callers'
rg -n 'run_in_sandbox|run_aiida_code|aiida-agents sandbox|sandbox check|verify_read_only|INSERT|UPDATE|DELETE|read-only|read.?only' \
  src/aiida_agents/agents src/aiida_agents/tools src/aiida_agents/cli src/aiida_agents/agents/codegen/prompt.md src/aiida_agents/sandbox || true

printf '\n%s\n' '--- codegen static verifier snippets from source as text'
python3 - <<'PY'
from pathlib import Path
p = Path('src/aiida_agents/agents/codegen/__init__.py').read_text()
for needle in [
    'run_aiida_code',
    'write-refusing profile',
    'does not need approval',
    'sandbox',
    'INSERT privilege',
]:
    print(f'{needle!r}: {needle in p}')
PY

Repository: aiidateam/aiida-agents

Length of output: 3407


🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

printf '%s\n' '--- codegen prompt ---'
sed -n '1,260p' src/aiida_agents/agents/codegen/prompt.md

printf '\n%s\n' '--- run_aiida_code codegen tool ---'
sed -n '1,220p' src/aiida_agents/tools/codegen.py

printf '\n%s\n' '--- static verifier: references to sandbox APIs/callers'
rg -n 'run_in_sandbox|run_aiida_code|aiida-agents sandbox|sandbox check|verify_read_only|INSERT|UPDATE|DELETE|read-only|read.?only' \
  src/aiida_agents/agents src/aiida_agents/tools src/aiida_agents/cli src/aiida_agents/agents/codegen/prompt.md src/aiida_agents/agents/codegen/__init__.py || true

Repository: aiidateam/aiida-agents

Length of output: 3407


🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

printf '%s\n' '--- locate run_aiida_code ---'
rg -n 'def run_aiida_code|run_aiida_code|run_in_sandbox' src/aiida_agents || true

printf '%s\n' '--- sandbox tool files ---'
git ls-files src/aiida_agents/tools src/aiida_agents/agents | rg '(^src/aiida_agents/tools/|^src/aiida_agents/agents/)' | sed -n '1,200p'

printf '%s\n' '--- all run_aiida_code occurrences ---'
rg -n 'run_aiida_code|SandboxResult|run_in_sandbox|aiida-agents sandbox|sandbox check|verify_read_only|INSERT|UPDATE|DELETE|read-only|read.?only|write privilege' src/aiida_agents || true

Repository: aiidateam/aiida-agents

Length of output: 14342


🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

printf '%s\n' '--- tools.codegen.execution ---'
sed -n '1,140p' src/aiida_agents/tools/codegen/execution.py

printf '%s\n' '--- _settings sandbox setting ---'
sed -n '170,215p' src/aiida_agents/_settings.py

Repository: aiidateam/aiida-agents

Length of output: 5252


Verify the complete read-only execution boundary.

INSERT refusal is not a complete read-only boundary. The runner uses no-writes SQL SQL; that still leaves other database writes and filesystem, process, or network effects possible if those constraints are not enforced. Add the missing controls before removing the approval gate, and align the module/prompt/doc claims with the verified boundary.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/aiida_agents/agents/codegen/__init__.py` around lines 17 - 24, Verify and
enforce the complete read-only boundary for the Codegen agent before relying on
its ungated execution: constrain all database write operations, filesystem
changes, process execution, and network side effects in the runner, not only
INSERT statements. Update the module’s prompt and documentation claims to match
the controls that are actually enforced, and retain the approval gate until
those protections are verified.

Comment on lines +19 to +26
**`codegen`** — writing Python against the user's data and running it. Use it
when a question needs a query no fixed tool expresses: several filters at once,
a combination across groups and elements, a projection of specific properties,
or anything the user asks for *as code*. It looks the API up in the
documentation, runs the snippet against a profile whose database role cannot
write, and reports what actually came back. It cannot submit or change
anything.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Update the specialist-count heading.

The new codegen section defines three specialists, but Line 6 still says The two specialists. Change that heading so the planner prompt does not contain contradictory specialist counts.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/aiida_agents/agents/planner/prompt.md` around lines 19 - 26, Update the
specialist-count heading near the `codegen` section from “The two specialists”
to reflect the three specialists defined in the planner prompt, without changing
the surrounding specialist descriptions.

Comment on lines +62 to +64
```
codegen: find every relaxation in group `screening-2026` whose final structure contains Ti, and report their total energies
```

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Label the Codegen example fence.

The example fence has no language identifier. Change it to ```text or ```plaintext.

🧰 Tools
🪛 markdownlint-cli2 (0.23.1)

[warning] 62-62: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/aiida_agents/agents/planner/prompt.md` around lines 62 - 64, Update the
Markdown code fence surrounding the Codegen example in the planner prompt to
include a text language identifier, using either text or plaintext, while
preserving the example content unchanged.

Source: Linters/SAST tools

Comment on lines +51 to +54
_AGENT_CHOICES = ("auto", "analysis", "execution", "codegen")
#: The specialists a request can actually be run by -- ``auto`` is a decision,
#: not an agent, so it is never passed to ``get_agent``.
_SPECIALISTS = ("analysis", "execution")
_SPECIALISTS = ("analysis", "execution", "codegen")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Preserve codegen in CLI agent selection.

_AGENT_CHOICES accepts "codegen", but _as_specialist_literal at Lines 336-338 maps every value other than "execution" to "analysis". A user who selects --agent codegen is therefore routed to the Analysis agent. Handle "codegen" explicitly and add a CLI test for the selection.

Proposed fix
 def _as_specialist_literal(name: str) -> Specialist:
     """Narrow a ``--agent`` value the Click choice has already constrained."""
-    return "execution" if name == "execution" else "analysis"
+    if name == "execution":
+        return "execution"
+    if name == "codegen":
+        return "codegen"
+    return "analysis"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/aiida_agents/cli/agent.py` around lines 51 - 54, Update
`_as_specialist_literal` to preserve the `"codegen"` selection instead of
mapping it to `"analysis"`, while retaining the existing `"execution"` and
analysis behavior. Add a CLI test covering `--agent codegen` and verify it
routes to the Codegen specialist.

Comment on lines +30 to +35
try:
from aiida.manage.configuration import get_config

return profile in {p.name for p in get_config().profiles}
except Exception: # pragma: no cover - a broken AiiDA config is not our news
return False

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Suppress or justify the blind-exception catch flagged by Ruff.

Static analysis flags Line 34's except Exception: as BLE001. The broad catch looks intentional here — a malformed or unreadable AiiDA configuration should degrade to "not configured" rather than raise — but the # pragma: no cover comment only affects coverage, not lint. Add a # noqa: BLE001 (with a short reason) or narrow the exception type, if the ruff configuration for this repository does not already exempt this rule for this file.

🔧 Proposed fix
-    except Exception:  # pragma: no cover - a broken AiiDA config is not our news
+    except Exception:  # noqa: BLE001 - pragma: no cover - a broken AiiDA config is not our news
         return False
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
try:
from aiida.manage.configuration import get_config
return profile in {p.name for p in get_config().profiles}
except Exception: # pragma: no cover - a broken AiiDA config is not our news
return False
try:
from aiida.manage.configuration import get_config
return profile in {p.name for p in get_config().profiles}
except Exception: # noqa: BLE001 - pragma: no cover - a broken AiiDA config is not our news
return False
🧰 Tools
🪛 Ruff (0.16.0)

[warning] 34-34: Do not catch blind exception: Exception

(BLE001)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/aiida_agents/tools/codegen/execution.py` around lines 30 - 35, Update the
broad exception handler in the profile-checking function around get_config to
satisfy Ruff’s BLE001 rule while preserving the intentional fallback of
returning False for malformed or unreadable AiiDA configuration. Prefer
narrowing the exception if the relevant failures are known; otherwise add an
inline BLE001 suppression with a brief justification alongside the existing
coverage pragma.

Source: Linters/SAST tools

CI's mdformat hook rewrote the title's double quotes to single ones and
failed the run. The frontmatter should not have been there at all: every
other ADR opens on its heading, and myst takes the title from that.

Worth recording why the local check missed it. `pre-commit run
--all-files` operates on files git knows about, so a newly created file
that has not been staged is invisible to every hook. ADR-11 was untracked
when the hooks were run, which is why they passed on a file CI then
rejected. Hooks after `git add`, not before.
@Jaweria-B
Jaweria-B merged commit 76b799d into main Aug 1, 2026
5 checks passed
@Jaweria-B
Jaweria-B deleted the feature/codegen-agent branch August 1, 2026 17:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant