fix(toolkit): strengthen agent skill instruction to enforce SKILL.md reading#1455
Open
octo-patch wants to merge 1 commit intoagentscope-ai:mainfrom
Open
Conversation
…reading
The previous default skill instruction used weak conditional language
("If you want to use a skill") that allowed LLMs to skip reading SKILL.md
and execute tasks from prior knowledge instead. This caused agents to
bypass the skill workflow and call tools directly without following the
documented steps in SKILL.md.
Changes:
- Fix typo: "folds" -> "folders" in the skill instruction
- Replace conditional "If you want to use" with mandatory "You MUST read
SKILL.md BEFORE performing any task related to that skill"
- Add explicit prohibition against using prior knowledge or assumptions
- Strengthen per-skill template from "Check..." to "IMPORTANT: You MUST
read..." with explicit prohibition on skipping SKILL.md
Fixes agentscope-ai#1453
Co-Authored-By: Octopus <liyuan851277048@icloud.com>
|
在Qwen3.5里的效果大差不差,得在系统提示词强制回复之前一定要阅读技能文件,因为只用过Qwen,不知道是模型问题还是框架设计问题。 |
Contributor
Author
|
@lsy-loren 谢谢反馈,确实如此 — 纯提示词强化对小一点的模型(含 Qwen3.5)效果有限,本 PR 只是把工具说明从「条件性建议」改成「强制要求」,把作弊空间收窄一些。要彻底解决可能需要框架层在调用其他工具前强制先调用一次 read_file 读 SKILL.md,那应该是另一个 PR 的范围。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1453
Problem
The default agent skill instruction used weak conditional language that allowed LLMs to skip reading
SKILL.mdand execute tasks from prior knowledge instead:This caused agents to bypass the skill workflow entirely — calling tools directly (e.g.
execute_shell_command) without first reading theSKILL.mdfile that documents the correct procedure. Additionally, the word "folds" in the instruction was a typo for "folders".The per-skill template also used a weak "Check..." phrasing that did not clearly communicate that reading SKILL.md is mandatory before any action.
Solution
folds→foldersin the instruction textSKILL.mdfile BEFORE performing any task related to that skill"SKILL.mdfirst"Check "{dir}/SKILL.md" for how to use this skilltoIMPORTANT: You MUST read "{dir}/SKILL.md" before using this skill. Do not perform any actions for this skill without first reading its SKILL.md file.Testing
The change is to the default prompt strings only. The existing test suite covers the toolkit APIs and no behavior change is expected in functional tests. The improvement is validated by the stronger LLM instruction that prevents models from skipping SKILL.md reading.