Skip to content

Commit df74c55

Browse files
docs: Tutorial 50 - Decision BOM with example (#1783)
Add tutorial and runnable example for the DecisionBOMReconstructor: - Reconstructible view from existing observability signals - Partial to full BOM with progressive signal source addition - Completeness scoring (0-100%) - Field categories: identity, trust, policy, action, context, outcome, lineage - Batch reconstruction for agent time ranges - JSON export for audit reporting - API reference Includes verified runnable demo (examples/decision-bom/). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent a663816 commit df74c55

4 files changed

Lines changed: 539 additions & 0 deletions

File tree

docs/tutorials/50-decision-bom.md

Lines changed: 319 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,319 @@
1+
# Tutorial 50: Decision Bill of Materials (Decision BOM)
2+
3+
Every governance decision has inputs: who requested it, what policies applied,
4+
what the trust score was, what trace it belongs to. The Decision BOM
5+
reconstructs all of these factors on demand, without requiring agents to
6+
report anything extra.
7+
8+
**Prerequisites:** Install AGT with the mesh package:
9+
10+
```bash
11+
pip install agentmesh-platform
12+
```
13+
14+
## Why Decision BOM?
15+
16+
When an auditor asks "Why was this action allowed?", you need more than a
17+
log entry. You need the full picture:
18+
19+
- Which agent requested the action?
20+
- What was their trust score at that moment?
21+
- Which policies were evaluated? What did each one decide?
22+
- Was there a delegation chain involved?
23+
- What was the OTel trace ID for correlation?
24+
25+
The Decision BOM reconstructs this from existing observability signals.
26+
No new data collection required.
27+
28+
## Core Concepts
29+
30+
### Reconstructible View
31+
32+
Unlike approaches that store a pre-built BOM at decision time, AGT reconstructs
33+
the BOM on demand by querying existing data sources:
34+
35+
```
36+
Audit Logs ──┐
37+
Trust Store ──┤── DecisionBOMReconstructor ──> DecisionBOM
38+
Policy Log ───┤
39+
OTel Traces ──┘
40+
```
41+
42+
This is **non-invasive**: agents don't need to change anything. The BOM
43+
infers everything from signals already being collected.
44+
45+
### Signal Sources (Protocols)
46+
47+
The reconstructor uses protocol-based abstractions so any backend can plug in:
48+
49+
| Source | What It Provides |
50+
|--------|-----------------|
51+
| `AuditSource` | Action logs, agent IDs, outcomes, policy decisions |
52+
| `TrustSource` | Trust scores at a point in time, score history |
53+
| `PolicySource` | Which policies evaluated, what they decided |
54+
| `TraceSource` | OTel spans for latency and correlation |
55+
56+
### Completeness Scoring
57+
58+
Every reconstructed BOM gets a completeness score (0.0 to 1.0) based on how
59+
many required fields could be populated. Five fields are required:
60+
61+
1. `agent_identity` - who acted
62+
2. `trust_score_at_decision` - trust level at the time
63+
3. `policy_rules_evaluated` - which policies ran
64+
4. `action_type` - what was attempted
65+
5. `decision_outcome` - allow/deny/alert
66+
67+
## Step 1: Set Up Signal Sources
68+
69+
Create adapters for your existing observability backends:
70+
71+
```python
72+
from datetime import datetime, timedelta, timezone
73+
from agentmesh.governance.decision_bom import (
74+
DecisionBOMReconstructor,
75+
BOMFieldCategory,
76+
)
77+
78+
# Example: wrap your audit log backend
79+
class MyAuditSource:
80+
def __init__(self, audit_log):
81+
self._log = audit_log
82+
83+
def query_by_trace(self, trace_id: str) -> list[dict]:
84+
return self._log.search(trace_id=trace_id)
85+
86+
def query_by_agent(self, agent_id: str, start: datetime, end: datetime) -> list[dict]:
87+
return self._log.search(agent_id=agent_id, start=start, end=end)
88+
```
89+
90+
For this tutorial, we will use in-memory mock sources:
91+
92+
```python
93+
class InMemoryAuditSource:
94+
def __init__(self):
95+
self.entries = []
96+
97+
def add(self, entry: dict):
98+
self.entries.append(entry)
99+
100+
def query_by_trace(self, trace_id: str) -> list[dict]:
101+
return [e for e in self.entries if e.get("trace_id") == trace_id]
102+
103+
def query_by_agent(self, agent_id: str, start: datetime, end: datetime) -> list[dict]:
104+
return [e for e in self.entries
105+
if e.get("agent_did") == agent_id
106+
and start <= e.get("timestamp", start) <= end]
107+
```
108+
109+
## Step 2: Reconstruct a Single Decision
110+
111+
```python
112+
from datetime import datetime, timezone
113+
114+
now = datetime.now(timezone.utc)
115+
116+
# Set up audit source with a recorded decision
117+
audit = InMemoryAuditSource()
118+
audit.add({
119+
"trace_id": "trace-abc-123",
120+
"agent_did": "did:mesh:payment-agent",
121+
"action": "transfer_funds",
122+
"resource": "account:checking",
123+
"outcome": "allow",
124+
"policy_decision": "allow",
125+
"session_id": "session-42",
126+
"timestamp": now,
127+
})
128+
129+
# Create reconstructor
130+
reconstructor = DecisionBOMReconstructor(audit_source=audit)
131+
132+
# Reconstruct the decision BOM
133+
bom = reconstructor.reconstruct(trace_id="trace-abc-123")
134+
135+
print(f"Decision: {bom.decision_id}")
136+
print(f"Agent: {bom.agent_id}")
137+
print(f"Action: {bom.action_requested}")
138+
print(f"Outcome: {bom.outcome}")
139+
print(f"Completeness: {bom.completeness_score:.0%}")
140+
print(f"Sources: {bom.sources_queried}")
141+
```
142+
143+
Expected output:
144+
145+
```
146+
Decision: trace-abc-123
147+
Agent: did:mesh:payment-agent
148+
Action: transfer_funds
149+
Outcome: allow
150+
Completeness: 60%
151+
Sources: ['audit']
152+
```
153+
154+
Completeness is 60% because we have 3 of 5 required fields (agent_identity,
155+
action_type, decision_outcome) but no trust score or policy evaluation data.
156+
157+
## Step 3: Add Trust Context
158+
159+
Add a trust source to increase completeness:
160+
161+
```python
162+
class InMemoryTrustSource:
163+
def __init__(self):
164+
self.scores = {}
165+
self.history = []
166+
167+
def get_score_at(self, agent_id: str, timestamp: datetime) -> float | None:
168+
return self.scores.get(agent_id)
169+
170+
def get_score_history(self, agent_id: str, start: datetime, end: datetime) -> list[dict]:
171+
return [h for h in self.history if h.get("agent_id") == agent_id]
172+
173+
trust = InMemoryTrustSource()
174+
trust.scores["did:mesh:payment-agent"] = 0.85
175+
trust.history = [
176+
{"agent_id": "did:mesh:payment-agent", "score": 0.80},
177+
{"agent_id": "did:mesh:payment-agent", "score": 0.85},
178+
]
179+
180+
reconstructor = DecisionBOMReconstructor(
181+
audit_source=audit,
182+
trust_source=trust,
183+
)
184+
185+
bom = reconstructor.reconstruct(trace_id="trace-abc-123")
186+
print(f"Completeness: {bom.completeness_score:.0%}") # Now 80%
187+
188+
# Inspect trust fields
189+
for f in bom.get_fields_by_category(BOMFieldCategory.TRUST):
190+
inferred = " (inferred)" if f.inferred else ""
191+
print(f" {f.name}: {f.value}{inferred}")
192+
```
193+
194+
Expected output:
195+
196+
```
197+
Completeness: 80%
198+
trust_score_at_decision: 0.85
199+
trust_score_trend: 0.05 (inferred)
200+
```
201+
202+
## Step 4: Full BOM with All Sources
203+
204+
Add policy and trace sources for 100% completeness:
205+
206+
```python
207+
class InMemoryPolicySource:
208+
def __init__(self):
209+
self.evaluations = []
210+
self.active_policies = []
211+
212+
def get_evaluations(self, trace_id: str) -> list[dict]:
213+
return [e for e in self.evaluations if e.get("trace_id") == trace_id]
214+
215+
def get_active_policies_at(self, timestamp: datetime) -> list[dict]:
216+
return self.active_policies
217+
218+
policy = InMemoryPolicySource()
219+
policy.evaluations = [
220+
{"trace_id": "trace-abc-123", "rule_name": "max-transfer", "decision": "allow"},
221+
{"trace_id": "trace-abc-123", "rule_name": "rate-limit", "decision": "allow"},
222+
]
223+
224+
reconstructor = DecisionBOMReconstructor(
225+
audit_source=audit,
226+
trust_source=trust,
227+
policy_source=policy,
228+
)
229+
230+
bom = reconstructor.reconstruct(trace_id="trace-abc-123")
231+
print(f"Completeness: {bom.completeness_score:.0%}") # 100%
232+
print(f"Sources: {bom.sources_queried}")
233+
```
234+
235+
## Step 5: Batch Reconstruction
236+
237+
Reconstruct all decisions by an agent in a time range:
238+
239+
```python
240+
# Add more audit entries
241+
audit.add({
242+
"trace_id": "trace-def-456",
243+
"agent_did": "did:mesh:payment-agent",
244+
"action": "read_balance",
245+
"outcome": "allow",
246+
"timestamp": now - timedelta(seconds=10),
247+
})
248+
249+
boms = reconstructor.reconstruct_batch(
250+
agent_id="did:mesh:payment-agent",
251+
start=now - timedelta(minutes=5),
252+
end=now + timedelta(seconds=1),
253+
)
254+
255+
print(f"\nReconstructed {len(boms)} decisions:")
256+
for bom in boms:
257+
print(f" {bom.action_requested}: {bom.outcome} "
258+
f"(completeness: {bom.completeness_score:.0%})")
259+
```
260+
261+
## Step 6: Export for Audit
262+
263+
The BOM serializes to a dictionary for storage or API responses:
264+
265+
```python
266+
import json
267+
268+
bom_data = bom.to_dict()
269+
print(json.dumps(bom_data, indent=2, default=str))
270+
```
271+
272+
This produces a structured JSON document with all fields, their categories,
273+
sources, confidence levels, and whether they were inferred.
274+
275+
## Field Categories
276+
277+
Every BOM field is categorized for organized audit reporting:
278+
279+
| Category | Fields |
280+
|----------|--------|
281+
| `identity` | agent_identity |
282+
| `trust` | trust_score_at_decision, trust_score_trend |
283+
| `policy` | policy_rules_evaluated, active_policies, policy_decision |
284+
| `action` | action_type, resource_target |
285+
| `context` | session_context, latency_ms |
286+
| `outcome` | decision_outcome |
287+
| `lineage` | delegation_chain, otel_trace_id |
288+
289+
## API Reference
290+
291+
### DecisionBOMReconstructor
292+
293+
| Method | Description |
294+
|--------|-------------|
295+
| `reconstruct(trace_id, agent_id, timestamp)` | Reconstruct a single BOM |
296+
| `reconstruct_batch(agent_id, start, end)` | Reconstruct all BOMs in a range |
297+
| `available_sources` | List configured signal sources |
298+
299+
### DecisionBOM
300+
301+
| Field | Type | Description |
302+
|-------|------|-------------|
303+
| `decision_id` | `str` | Unique identifier (trace_id or agent@time) |
304+
| `timestamp` | `datetime` | When the decision was made |
305+
| `agent_id` | `str` | The agent involved |
306+
| `action_requested` | `str` | What was attempted |
307+
| `outcome` | `str` | allow, deny, or alert |
308+
| `fields` | `list[BOMField]` | All reconstructed fields |
309+
| `completeness_score` | `float` | 0.0 to 1.0 |
310+
| `sources_queried` | `list[str]` | Which backends were used |
311+
312+
## What's Next
313+
314+
- [Tutorial 04 - Audit & Compliance](04-audit-and-compliance.md): Set up the
315+
audit logs that feed BOM reconstruction
316+
- [Tutorial 13 - Observability & Tracing](13-observability-and-tracing.md):
317+
Add OTel traces for full lineage correlation
318+
- [Tutorial 48 - Intent-Based Authorization](48-intent-based-authorization.md):
319+
Combine intent verification with decision BOMs

docs/tutorials/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@ guides.
6565
| 13 | [Observability & Tracing](13-observability-and-tracing.md) | Causal traces, event bus, Prometheus, OpenTelemetry | `agentmesh-runtime` |
6666
| 15 | [RL Training Governance](15-rl-training-governance.md) | GovernedRunner, PolicyReward, Gym-compatible environments | `agentmesh-lightning` |
6767
| 18 | [Compliance Verification](18-compliance-verification.md) | Governance grading, regulatory frameworks, attestation | `agent-governance-toolkit` |
68+
| 50 | [Decision BOM](50-decision-bom.md) | Reconstruct full decision context from observability signals, completeness scoring, batch audit | `agentmesh-platform` |
6869

6970
## Multi-Language Packages
7071

examples/decision-bom/README.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Decision BOM Example
2+
3+
Demonstrates how AGT reconstructs a complete Bill of Materials for any
4+
governance decision, on demand, from existing observability signals.
5+
6+
## Quick Start
7+
8+
```bash
9+
pip install agentmesh-platform
10+
python decision_bom_demo.py
11+
```
12+
13+
## What This Demo Shows
14+
15+
1. **Partial BOM**: Reconstruction with just audit logs (60% completeness)
16+
2. **Full BOM**: All 4 signal sources for 100% completeness
17+
3. **Field Categories**: Identity, trust, policy, action, context, outcome, lineage
18+
4. **Batch Reconstruction**: All decisions by an agent in a time range
19+
5. **JSON Export**: Structured output for audit reporting
20+
21+
## Learn More
22+
23+
- [Tutorial 50: Decision BOM](../../docs/tutorials/50-decision-bom.md)
24+
- [API: decision_bom.py](../../agent-governance-python/agent-mesh/src/agentmesh/governance/decision_bom.py)

0 commit comments

Comments
 (0)