Skip to content

fix(toolkit): strengthen agent skill instruction to enforce SKILL.md reading#1455

Open
octo-patch wants to merge 1 commit intoagentscope-ai:mainfrom
octo-patch:fix/issue-1453-strengthen-skill-instruction
Open

fix(toolkit): strengthen agent skill instruction to enforce SKILL.md reading#1455
octo-patch wants to merge 1 commit intoagentscope-ai:mainfrom
octo-patch:fix/issue-1453-strengthen-skill-instruction

Conversation

@octo-patch
Copy link
Copy Markdown
Contributor

Fixes #1453

Problem

The default agent skill instruction used weak conditional language that allowed LLMs to skip reading SKILL.md and execute tasks from prior knowledge instead:

If you want to use a skill, you MUST read its `SKILL.md` file carefully.

This caused agents to bypass the skill workflow entirely — calling tools directly (e.g. execute_shell_command) without first reading the SKILL.md file that documents the correct procedure. Additionally, the word "folds" in the instruction was a typo for "folders".

The per-skill template also used a weak "Check..." phrasing that did not clearly communicate that reading SKILL.md is mandatory before any action.

Solution

  1. Fix typo: foldsfolders in the instruction text
  2. Strengthen instruction: Replace the conditional "If you want to use a skill" with an unconditional mandate: "You MUST read the skill's SKILL.md file BEFORE performing any task related to that skill"
  3. Add explicit prohibition: "Do NOT rely on prior knowledge or make assumptions about how to use a skill — always read its SKILL.md first"
  4. Strengthen template: Change from Check "{dir}/SKILL.md" for how to use this skill to IMPORTANT: You MUST read "{dir}/SKILL.md" before using this skill. Do not perform any actions for this skill without first reading its SKILL.md file.

Testing

The change is to the default prompt strings only. The existing test suite covers the toolkit APIs and no behavior change is expected in functional tests. The improvement is validated by the stronger LLM instruction that prevents models from skipping SKILL.md reading.

…reading

The previous default skill instruction used weak conditional language
("If you want to use a skill") that allowed LLMs to skip reading SKILL.md
and execute tasks from prior knowledge instead. This caused agents to
bypass the skill workflow and call tools directly without following the
documented steps in SKILL.md.

Changes:
- Fix typo: "folds" -> "folders" in the skill instruction
- Replace conditional "If you want to use" with mandatory "You MUST read
  SKILL.md BEFORE performing any task related to that skill"
- Add explicit prohibition against using prior knowledge or assumptions
- Strengthen per-skill template from "Check..." to "IMPORTANT: You MUST
  read..." with explicit prohibition on skipping SKILL.md

Fixes agentscope-ai#1453

Co-Authored-By: Octopus <liyuan851277048@icloud.com>
@lsy-loren
Copy link
Copy Markdown

在Qwen3.5里的效果大差不差,得在系统提示词强制回复之前一定要阅读技能文件,因为只用过Qwen,不知道是模型问题还是框架设计问题。

@octo-patch
Copy link
Copy Markdown
Contributor Author

@lsy-loren 谢谢反馈,确实如此 — 纯提示词强化对小一点的模型(含 Qwen3.5)效果有限,本 PR 只是把工具说明从「条件性建议」改成「强制要求」,把作弊空间收窄一些。要彻底解决可能需要框架层在调用其他工具前强制先调用一次 read_file 读 SKILL.md,那应该是另一个 PR 的范围。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AgentScope SKILL 执行问题

2 participants