Repository navigation
feat(rust): add indirect/tool-output injection rules to CONTEXT_RULES - #3069
Conversation
The CONTEXT_RULES family covered first-person social-engineering variants
but had no coverage for the indirect / tool-output injection class: payloads
that arrive inside retrieved content (web pages, emails, RAG documents,
tool results) and address the model directly.
Add four rules in the existing (regex, ThreatLevel, f64, name) tuple format:
context:instructions_for_ai_reading High/0.9
"instructions for the AI/assistant reading/processing this"
context:system_note_to_assistant High/0.9
"system note / note to the assistant:" impersonating a system channel
context:embedded_tool_directive Medium/0.8
HTML-comment-smuggled directive addressed to the AI (<!-- ai: … -->)
context:retrieved_doc_override High/0.9
"this document/page/email instructs you to [inject verb]"
Scoped to require an explicit injection verb after "you to" so ordinary
form instructions ("this page asks you to fill in…") do not fire.
All patterns are Rust-regex-safe (no lookarounds or backreferences) and
fit the existing family mechanics — the bare name in the tuple becomes the
stable public rule ID "context:<name>" via family_contains_rule_id /
is_known_built_in_rule_id, exactly like the existing rules.
Adds an indirect_injection_rules test module in tests/prompt_injection.rs
with one positive fixture and one benign-negative fixture per rule, plus
a cross-family type check and a disable-list recognition test. All 10 new
tests pass alongside the existing 63-test suite (373 total, 0 failures).
Scope: CONTEXT_RULES only. DIRECT_RULES, audit chain, policy evaluator,
and governance layer are untouched.
References: OWASP LLM Top 10 LLM01 (Prompt Injection),
MITRE ATLAS AML.T0051.001, and the indirect injection patterns discussed
in issue microsoft#3012.
Assisted-by: Claude Sonnet 4.6
Signed-off-by: AgentGymLeader <agentgymleader@users.noreply.github.com>
🤖 AI Agent: docs-sync-checker — Docs Sync
Docs Sync
|
🤖 AI Agent: test-generator — View details
Test coverage looks good. No gaps identified. |
🤖 AI Agent: code-reviewer — View details
TL;DR: 0 blockers, 0 warnings. Clean implementation of new indirect/tool-output injection rules.
No action items. The changes are well-implemented, with comprehensive tests ensuring correctness and no impact on existing functionality. |
🤖 AI Agent: security-scanner — View details
No security issues found. |
🤖 AI Agent: breaking-change-detector — API Compatibility
API CompatibilityNo breaking changes detected. |
|
🟡 Contributor Check: MEDIUM
Automated check by AGT Contributor Check. |
github-actions
Bot
added
the
needs-review:MEDIUM
PR Review Summary
Verdict: AI review comments are untrusted advisory output. The summary reports workflow-generated completion status only, not model-authored pass/fail claims. |
Imran Siddique (imran-siddique)
left a comment
Imran Siddique (imran-siddique)
left a comment
There was a problem hiding this comment.
Reviewed the four new CONTEXT_RULES entries and the mod indirect_injection_rules test block. The implementation is clean and follows existing conventions exactly.
Patterns
All four regexes are ReDoS-safe (no nested quantifiers, no backrefs), and the verb-gated tail on retrieved_doc_override (ignore|disregard|...) is the right call to keep false positives off ordinary form copy. The injection-verb list covers the OWASP LLM01 / ATLAS AML.T0051.001 canonical payload verbs without being so broad it fires on everyday phrasing.
One minor inconsistency worth noting for a follow-up: attention\s*(?:ai|assistant) uses \s* (zero-or-more whitespace) while every other sub-expression in system_note_to_assistant uses \s+. This means attentionAI: matches today. Harmless in practice — the HTML-comment equivalent would be the more realistic attack surface — but aligning to \s+ would be cleaner.
Tests
Positive + benign-negative per rule is exactly the right structure. The cross-family InjectionType::ContextManipulation assertion is important for downstream dispatch and I'm glad it's here. The is_known_built_in_rule_id coverage via the disable-list validator is a nice way to ensure the rule IDs are registered end-to-end, not just pattern-matched.
Scope
Confirmed: DIRECT_RULES, the audit chain, the policy evaluator, and the governance layer are untouched. This is purely additive.
Closes #3012. Approving — the \s* note above is fine to handle in a follow-up if the author prefers not to re-push.
0e231e0
into
microsoft:main
Address all six blocking issues raised by MohammadHaroonAbuomar and confirmed by imran-siddique in PR microsoft#3114. AppArmor profile (agt-sandbox): - Change all deny rules from x to mrx so ld.so-based invocations are blocked in addition to direct exec calls - Add deny mrx for /tmp/** and /workspace/** to close the copy-and-exec bypass vector - Remove the contradictory deny ptrace read rule at L36 which conflicted with the docker-default allow form retained at L60 - Reword the option 3 header to accurately describe exec-level denylist scope and note that in-process network access via Python urllib or similar is not covered by this profile Provider (provider.py): - _apparmor_profile_loaded now checks the mode token and only returns True when the profile is in (enforce) mode; a profile in (complain) mode is not a confinement boundary - Add _fallback_apparmor_profile helper that emits a logger.warning before returning docker-default so operators have a signal when the custom profile was not applied Scope cleanup: - Revert agent-governance-rust/agentmesh/src/prompt_injection.rs and tests/prompt_injection.rs to their main branch state; these indirect injection rule additions are unrelated to the AppArmor work and will be submitted as a separate PR to avoid conflicting with microsoft#3069 Tests (test_docker_sandbox.py): - Add test_returns_false_when_profile_in_complain_mode - Add test_returns_false_when_mode_token_missing - Add test_fallback_emits_warning to verify the warning log fires when the fallback path is taken
A rebase on this branch reverted the indirect and tool output injection rules from microsoft#3069 along with their test module. Restored both files from main so instructions_for_ai_reading, system_note_to_assistant, embedded_tool_directive, and retrieved_doc_override are back in CONTEXT_RULES. Reported by MohammadHaroonAbuomar and carloshvp on microsoft#3114.
…microsoft#3069) The CONTEXT_RULES family covered first-person social-engineering variants but had no coverage for the indirect / tool-output injection class: payloads that arrive inside retrieved content (web pages, emails, RAG documents, tool results) and address the model directly. Add four rules in the existing (regex, ThreatLevel, f64, name) tuple format: context:instructions_for_ai_reading High/0.9 "instructions for the AI/assistant reading/processing this" context:system_note_to_assistant High/0.9 "system note / note to the assistant:" impersonating a system channel context:embedded_tool_directive Medium/0.8 HTML-comment-smuggled directive addressed to the AI (<!-- ai: … -->) context:retrieved_doc_override High/0.9 "this document/page/email instructs you to [inject verb]" Scoped to require an explicit injection verb after "you to" so ordinary form instructions ("this page asks you to fill in…") do not fire. All patterns are Rust-regex-safe (no lookarounds or backreferences) and fit the existing family mechanics — the bare name in the tuple becomes the stable public rule ID "context:<name>" via family_contains_rule_id / is_known_built_in_rule_id, exactly like the existing rules. Adds an indirect_injection_rules test module in tests/prompt_injection.rs with one positive fixture and one benign-negative fixture per rule, plus a cross-family type check and a disable-list recognition test. All 10 new tests pass alongside the existing 63-test suite (373 total, 0 failures). Scope: CONTEXT_RULES only. DIRECT_RULES, audit chain, policy evaluator, and governance layer are untouched. References: OWASP LLM Top 10 LLM01 (Prompt Injection), MITRE ATLAS AML.T0051.001, and the indirect injection patterns discussed in issue microsoft#3012. Assisted-by: Claude Sonnet 4.6 Signed-off-by: AgentGymLeader <agentgymleader@users.noreply.github.com> Co-authored-by: AgentGymLeader <agentgymleader@users.noreply.github.com>
Add four rules to
CONTEXT_RULEScovering the indirect / tool-output injection class (OWASP LLM01, MITRE ATLAS AML.T0051.001). The existing context family only handled first-person social-engineering variants; payloads arriving inside retrieved content (web pages, emails, RAG documents, tool results) that address the model directly had no coverage.The four new rules follow the existing
(regex, ThreatLevel, f64, name)tuple format and produce stable public IDs via the existingfamily_contains_rule_id/is_known_built_in_rule_idpath, exactly like the current rules:context:instructions_for_ai_reading(High / 0.9) — "instructions for the AI/assistant reading/processing this"context:system_note_to_assistant(High / 0.9) — "system note / note to the assistant:" impersonating a system channelcontext:embedded_tool_directive(Medium / 0.8) — HTML-comment-smuggled directive addressed to the AIcontext:retrieved_doc_override(High / 0.9) — "this document/page/email instructs you to [injection verb]", scoped to an explicit injection verb so ordinary form instructions do not fireDIRECT_RULES, the audit chain, the policy evaluator, and the governance layer are untouched.tests/prompt_injection.rsadds amod indirect_injection_rulesblock with one positive fixture and one benign-negative fixture per rule, a cross-family injection type check, and a disable-list recognition test.cargo test --release --workspacepasses with 0 failures before and after the change.Happy to adjust threat levels, pattern scope, or test fixture wording if the review surfaces concerns — let me know.
Closes #3012