chore: pin the suite-adequacy figures and record the study

check_manuscript pins the coverage, the mutation score, the gap and its interval, and the
per-operator kill rates, and the reproduction guide and changelog record the new scripts and
how they differ from the decision-blindness study.

Author: MohammadThabetHassan

CHANGELOG.md

@@ -40,6 +40,16 @@ All notable changes to TrustWeave are documented in this file. The project follo
   (`docs/cedar-suite-strategy-study-v1.json`, deviations recorded in it). Of 43 real
   earlier-version edits, 24 change a decision and 21 of those weaken one
   (`docs/cedar-real-edits-v1.json`).
+- `scripts/rego_suite_study.py` and `scripts/rego_source_mutation.py` measure what a real
+  policy suite's line coverage is worth, under `docs/SUITE_ADEQUACY_PROTOCOL_REGO.md` (hash
+  fixed in the script): on the 49 OPA Gatekeeper modules that ship an author-written suite, the
+  line coverage `opa test --coverage` reports against the fraction of the module's mutants the
+  same suite kills, every number from the engine. The suites reach 93.8% mean line coverage but
+  kill 70.6% of mutants, a 23.2-point gap concentrated in value-level faults
+  (`docs/rego-suite-adequacy-v1.json`). `rego_source_mutation.py` locates each edit through
+  `opa parse`, a distinct and richer operator set than `rego_mutation.py`'s fixed-text one
+  (it adds deleted rules, dropped conditions and replaced literals, and never edits inside a
+  string literal); the two studies measure different things and are reported separately.
 - `third_party_sample.py oracle-rego` checks the sampled Rego modules against `opa deps`, each
   compiled with its whole repository: 150 agree, the 6 that do not all share their package with
   other files (the engine answers for a package, the adapter for a file), and for 144 the

docs/REPRODUCING_THE_STUDY.md

@@ -195,6 +195,24 @@ python scripts/cedar_exact_study.py edits --cache /tmp/cedar-cache \
 `repos/{owner}/{repo}/commits?path=` returns them; `docs/cedar-real-edits-v1.json` pins every
 pair it measured by commit and SHA-256, so re-deriving the pairs does not depend on it.
 
+### 4d. What a real suite's line coverage is worth (Gatekeeper Rego)
+
+Runs under [`SUITE_ADEQUACY_PROTOCOL_REGO.md`](SUITE_ADEQUACY_PROTOCOL_REGO.md). Needs `opa`
+on `PATH` and the pinned Gatekeeper library (a cloned corpus, commit in the artifact's `corpus`
+block). For each module with an author-written suite it records the suite's line coverage and
+its mutation score, every number from `opa test`.
+
+```bash
+python scripts/rego_suite_study.py study \
+  --corpus /path/to/gatekeeper-library \
+  --json docs/rego-suite-adequacy-v1.json
+```
+
+This is a different measurement from the decision-blindness prediction of §4 above: it reports
+line coverage against a located-source mutation score (`scripts/rego_source_mutation.py`), where
+`scripts/rego_mutation.py` reports a text-search mutation score against the decision-coverage
+flag. The protocol states how the two operator sets differ.
+
 ### 5. The whole gate
 
 ```bash

scripts/check_manuscript.py

@@ -1172,9 +1172,75 @@ def p_value(stratum: str) -> str:
     claims += _summary_claims(docs)
     claims += _cedar_claims(docs)
     claims += _sample_oracle_claims(docs)
+    claims += _rego_suite_claims(docs)
     return claims
 
 
+def _rego_suite_claims(docs: Path) -> list[Claim]:
+    """What a real suite's line coverage is worth against its mutation score (Gatekeeper Rego)."""
+
+    suite = _load(docs, "rego-suite-adequacy-v1")
+    head = suite["headline"]
+    operators = suite["by_operator"]
+
+    def rate(operator: str) -> str:
+        counts = operators[operator]
+        return str(round(100 * counts["killed"] / counts["total"]))
+
+    return [
+        (
+            rf"on the ({_GROUPED}) Gatekeeper modules\s+that ship an author-written suite",
+            (_grouped(suite["population"]["modules_with_a_suite_and_a_decision"]),),
+            "rego suite study: the population",
+        ),
+        (
+            r"a mean of (\d+\.\d)\\% line coverage",
+            (f"{head['mean_coverage']:.1f}",),
+            "rego suite study: mean coverage",
+        ),
+        (
+            r"killing\s+(\d+\.\d)\\% in the mean "
+            rf"\(({_GROUPED}) of ({_GROUPED}) over the ({_GROUPED}) modules",
+            (
+                _pct(head["mean_mutation_score"]),
+                _grouped(head["killed"]),
+                _grouped(head["mutants"]),
+                _grouped(head["modules"]),
+            ),
+            "rego suite study: mean mutation score and pool",
+        ),
+        (
+            r"a per-module gap of (\d+\.\d) points with a 95\\% bootstrap interval of "
+            r"\$\[(\d+\.\d), (\d+\.\d)\]\$",
+            (
+                _pct(head["mean_gap"]),
+                _pct(head["gap_bootstrap_95"][0]),
+                _pct(head["gap_bootstrap_95"][1]),
+            ),
+            "rego suite study: the gap",
+        ),
+        (
+            r"the suites kill (\d+)\\% of deleted rules and (\d+)\\% of removed negations but "
+            r"only (\d+)\\% of changed string literals, (\d+)\\% of changed comparisons and "
+            r"(\d+)\\% of changed numbers",
+            (
+                rate("delete a rule"),
+                rate("remove a negation"),
+                rate("change a string literal"),
+                rate("change a comparison"),
+                rate("change a number literal"),
+            ),
+            "rego suite study: the uneven gap by operator",
+        ),
+        (
+            r"author-written suites for real Rego policy reach (\d+\.\d)\\% line coverage yet "
+            r"kill only (\d+\.\d)\\% of the policies' mutants",
+            (f"{head['mean_coverage']:.1f}", _pct(head["mean_mutation_score"])),
+            "rego suite study: the figures restated in related work",
+        ),
+    ]
+
+
 def _sample_oracle_claims(docs: Path) -> list[Claim]:
     """The sampled Rego modules against the engine's own dependency analysis."""