Repository navigation
Report unavailable and truncated policy evidence instead of zeros - #554
Conversation
With no audit query store, the rule hit-count and shadow-review endpoints answered 200 with zero counts and empty summaries. That reads exactly like "this rule never matched" or "this shadow rule would deny nothing", so an operator could delete a rule or promote a shadow rule to enforced deny on evidence that was never read. Cluster mode was always in this state: both aggregates still read only the SQLite query surface. Both endpoints now answer 503 with a stable "audit_evidence_unavailable" reason after their permission checks, and rule preview's existing 503 for the same condition carries the same reason. A store that is present still returns real zeros, which tests now pin separately. Rule preview had no hard bound on how many observations one request could scan. It now stops at MAX_RULE_PREVIEW_SCAN_ROWS (100,000, matching the other audit scan ceilings) and reports scan_truncated and complete in its response. Truncation is declared only when a row past the ceiling is actually reached, so a window that exactly fills the ceiling stays complete. Both fields are additive. The admin UI maps only the gateway's explicit reason to an unavailable evidence state; every other failure still surfaces as an error. The rulebase shows "Evidence unavailable", "Not reported" and "No recorded hits" as distinct text, the shadow review queue lists enabled shadow rules from the policy with unavailable evidence instead of zero counts and labels truncated counts, and the rule preview presents a truncated count as a lower bound. Shadow promotion is intentionally not yet bound to a reviewed revision, window or digest; that remains separate work on #243.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9463e62fda
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| void disableRule(item.ruleId); | ||
| }} | ||
| onPromote={() => { | ||
| void promoteRule(summary.rule_id); | ||
| void promoteRule(item.ruleId); |
There was a problem hiding this comment.
Disable promotion when shadow evidence is unavailable
When /policy/rules/shadow-review returns the new evidence-unavailable 503, loadReview now populates items from the policy and retains write capability, so this callback still lets an operator confirm shadow → deny despite no audit evidence having been read. Previously that request failure cleared the summaries and exposed no promotion control; in cluster mode this now affects every shadow review because the aggregate is always unavailable. Keep the safe Disable action available, but gate Promote while evidence.status === 'unavailable' so an audit-store error cannot become an enforced authorization change.
AGENTS.md reference: AGENTS.md:L53-L53
Useful? React with 👍 / 👎.
Pull Request
What changed and why
Issue #243 slice 2 leftover: evidence honesty. When no audit query store is configured, the hit-count and shadow-review endpoints answered
200with zeros and empty summaries. That looks the same as "this rule never matched" or "this shadow rule would deny nothing", so an operator could delete or promote a rule based on evidence that was never read. Cluster mode was always in this state, because both aggregates still read only the SQLite query surface.Gateway
GET /v1{ADMIN_PREFIX}/policy/rules/hitsandGET /v1{ADMIN_PREFIX}/policy/rules/shadow-reviewnow answer503with{"error": "...", "reason": "audit_evidence_unavailable"}whenquery_storeisNone. This uses the existing{error, reason}admin error shape, through a newpolicy_evidence_unavailablehelper inadmin_responses.rs. Auth and permission checks still run first: the shadow review still returns403withoutadmin:audit:read, even when no store exists.503(no audit store) now carries the samereason. Theerrortext is unchanged.MAX_RULE_PREVIEW_SCAN_ROWS = 100_000inapp_state.rs, next toMAX_RULE_PREVIEW_SAMPLE_LIMIT, matching the other audit scan caps.PolicyRulePreviewResponsegainsscan_truncatedandcomplete. Both are additive. Truncation is reported only when the scan actually reaches a row past the ceiling, so a window that exactly fills the ceiling is stillcomplete.Admin UI
lib/policy.ts:fetchPolicyRuleHitsandfetchPolicyRuleShadowReviewreturnPolicyEvidence<T>(available|unavailable). Only the gateway's explicit503+audit_evidence_unavailablemaps tounavailable. Every other failure, including other 503s, still rejects as an error.RuleTable: each row's evidence readsEvidence unavailable,Not reported(the rule is missing from the counts),No recorded hits(a counted zero; this replaces "never matched"), orN hits. When evidence is unavailable, a notice explains it and the policy still loads.ShadowReviewView: when evidence is unavailable, enabled shadow rules are listed from the policy withEvidence unavailableinstead of zero counts. Truncated counts are labelled(scan truncated), and the banner says counts are lower bounds. A complete zero still reads0 would-deny events.RuleEditorpreview: a truncated result is described as "at least N" and gets aScan truncatedbadge. This is the natural consumer of the new preview fields.Docs
docs/configuration.md(preview, hits and shadow-review contracts) anddocs/deployment/postgres.md(cluster-mode behaviour) are updated, plus a CHANGELOG entry.docs/openapihas no spec for these endpoints, so no OpenAPI changes.Not in this PR (by design): binding shadow promotion to a reviewed revision, window or digest; the centralized privacy projection; the
admin:audit:readsplit for preview samples.Compatibility note: clients that relied on
200with zeros from hits or shadow review without an audit store now get503. This is the intended fail-closed change. The admin UI in this PR handles it.Issue
Part of #243
Checklist
503s for hits and shadow review, real zeros with an empty store, audit-read checked before availability, preview truncation and exact-ceiling completeness, and cluster-mode503s. Vitest:policy.test.ts,RuleTable,ShadowReviewView,RuleEditor.audit,admin-uiandcloudflarenpm audit,connections-acceptance) tracked by the dependency PR