Skip to content

Commit 677d28d

Browse files
qhkmclaude
andcommitted
fix(templates): coder template now uses existing tests and edit_file
The coder template system prompt lacked instructions to check for existing test files before writing new ones, causing the LLM to write its own tests that matched buggy behavior and report false success. Added a 5-step workflow: discover existing tests, run them first, fix in-place with edit_file, verify with existing suite, summarize changes. Closes #345 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 2d39ea2 commit 677d28d

1 file changed

Lines changed: 39 additions & 2 deletions

File tree

src/config/templates.rs

Lines changed: 39 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -98,9 +98,19 @@ fn builtin_coder() -> AgentTemplate {
9898
"well-structured code following best practices for the language and ",
9999
"framework in use. You explain your reasoning, consider edge cases, ",
100100
"handle errors properly, and write tests when appropriate. You prefer ",
101-
"simple, readable solutions over clever ones. ",
101+
"simple, readable solutions over clever ones.\n\n",
102102
"You have access to grep (regex search across files) and find (glob file discovery) ",
103-
"tools for navigating and understanding codebases."
103+
"tools for navigating and understanding codebases.\n\n",
104+
"When asked to fix bugs or modify code, follow this workflow:\n",
105+
"1. Before writing any tests, check for existing test files (e.g., test_*.py, *_test.go, ",
106+
"*Test.java). Use find or grep to discover them.\n",
107+
"2. If existing tests exist, run them FIRST to see which tests fail and understand the ",
108+
"current state of the code.\n",
109+
"3. Fix bugs by editing the original source file in-place using edit_file. Do NOT create ",
110+
"new copies of the file or rewrite it with write_file unless explicitly asked to.\n",
111+
"4. After making changes, verify by re-running the EXISTING test suite — not by writing ",
112+
"your own tests. Only write new tests if no relevant tests exist or the user asks for them.\n",
113+
"5. Show a clear summary of what you changed and why."
104114
)
105115
.to_string(),
106116
model: None,
@@ -427,6 +437,33 @@ mod tests {
427437
assert!(coder.tags.contains(&"coding".to_string()));
428438
}
429439

440+
#[test]
441+
fn test_coder_prompt_instructs_existing_test_workflow() {
442+
let registry = TemplateRegistry::new();
443+
let coder = registry.get("coder").unwrap();
444+
let prompt = &coder.system_prompt;
445+
// Must instruct to check for existing tests before writing new ones
446+
assert!(
447+
prompt.contains("existing test"),
448+
"coder prompt must mention existing tests"
449+
);
450+
// Must instruct to use edit_file for in-place fixes
451+
assert!(
452+
prompt.contains("edit_file"),
453+
"coder prompt must instruct using edit_file"
454+
);
455+
// Must instruct to run existing tests for verification
456+
assert!(
457+
prompt.contains("re-running the EXISTING test suite"),
458+
"coder prompt must instruct re-running existing tests"
459+
);
460+
// Must NOT encourage writing own tests as default verification
461+
assert!(
462+
prompt.contains("Only write new tests if no relevant tests exist"),
463+
"coder prompt must discourage writing new tests when existing ones are available"
464+
);
465+
}
466+
430467
#[test]
431468
fn test_builtin_researcher_exists() {
432469
let registry = TemplateRegistry::new();

0 commit comments

Comments
 (0)