You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
pdd sync over-regenerates mature modules and breaks public compatibility #1012
pdd sync can turn a narrow prompt change into a broad rewrite of mature generated modules and tests. In PR #1010 for issue #1005, a small prompt change regenerated pdd/update_main.py and tests/test_update_main.py with thousands of lines changed, dropping existing public/helper compatibility.
Broke .pddrc context/strength behavior in update flows.
Narrow generated tests passed while broader CI failed.
Expected Behavior
For mature modules, pdd sync should preserve the existing public surface and behavior unless the prompt explicitly requests a breaking change.
Acceptance Criteria
pdd sync has a compatibility/conformance gate comparing generated public symbols/signatures against the pre-sync module or architecture.json interface.
Existing externally imported helpers remain present unless explicitly removed by the prompt/issue.
Generated test rewrites do not discard broad existing coverage for unrelated behavior.
A sync that would produce broad unrelated churn fails with an actionable diagnostic instead of committing it.
All agent providers failed: anthropic: Exit code 1: {"type":"result","subtype":"success","is_error":true,"api_error_status":429,"duration_ms":615,"duration_api_ms":0,"num_turns":1,"result":"You've hit your limit · resets May 15, 11pm (UTC)","stop_reason":"stop_sequence","session_id":"1f5e06b0-7f37-4acb-8215-10069676b238","total_cost_usd":0,"usage":{"input_tokens":0,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"output_tokens":0,"server_tool_use":{"web_search_requests":0,"web_fetch_requests":0},"service_tier":"standard","cache_creati
Automated fallback comment — agent did not execute for this step.
A similar issue exists at #1011 with the same title (pdd sync over-regenerates mature modules and breaks public compatibility) and substantially the same body content describing the PR #1010 / issue #1005 regression.
Because the related issue is unresolved, this issue (#1012) is not being closed. Per workflow rules, this issue will proceed as a new change request for investigation. The acceptance criteria (compatibility/conformance gate on pdd sync, preserve externally imported helpers, prevent broad unrelated test churn, fail with actionable diagnostic) will be addressed here.
build_conformance_hard_failure_from_error (pdd/agentic_sync_runner.py:466) formats a hard-failure diagnostic block when generated code doesn't expose the expected symbols.
pdd/sync_orchestration.py (lines ~2205–2264) runs an architecture-conformance repair loop (referencing chore: PDD sync for #865 #866) that retries generation when conformance fails and surfaces a hard-block diagnostic if budget is exhausted.
However, this only covers the symbols declared in architecture.json. The acceptance criteria in #1012 ask for a broader gate that:
Compares the pre-sync module's actual public surface (module-level symbols like git, read_fingerprint, calculate_sha256, helper signatures such as derive_basename_and_language) against the post-sync output — not just architecture.json.
Detects broad unrelated churn in generated tests (e.g., test-file rewrites that discard prior coverage for unrelated behavior).
Preserves .pddrc context/strength behavior in update flows.
Fails sync with an actionable diagnostic when these conditions are violated, rather than committing the over-regenerated output.
None of these broader checks exist today. The current gate is symbol-presence-against-architecture-entry only; it does not diff the pre-sync module surface, does not analyze test-coverage regressions, and does not detect bulk-rewrite churn for mature modules.
Conclusion
A partial gate exists (architecture.json symbol conformance + retry loop), but the wider compatibility gate described in this issue — diff against the pre-sync public surface, broad-churn detection on tests, and an actionable diagnostic that blocks the over-regeneration seen in PR #1010 — is not yet implemented. Proceeding with implementation.
Researched Python public-API diffing tools and LLM-regenerated test churn. Mature, well-established tools (Griffe, pidiff) exist for AST-based public-surface diffing; recent LLM-codegen research confirms that test-suite churn (new tests added while valid ones are discarded) is a real, measurable phenomenon — exactly what was observed in PR #1010.
Research Findings
Technical Background
Public-surface diffing in Python is a solved problem at the AST/static-analysis level. The Python inspect module + ast module are enough to extract module-level callables, classes, signatures, and __all__. Higher-level tools wrap this for SemVer-style breakage detection.
Public symbol convention (per Griffe docs): non-underscore-prefixed module attributes are public; imported re-exports are not public unless listed in __all__ or re-imported with from x import y as y. PDD's update_main.py regression (git, read_fingerprint, calculate_sha256) is exactly this case: module-level re-exports used as a stable API by callers.
LLM test-churn is asymmetric: arXiv 2603.23443 / 2507.19390 (ReCatcher) measured ~18.5 tests churned per regeneration (9.7 new, 8.8 discarded) even when behavior is unchanged — driven by surface-level syntactic cues. This matches the chore: PDD sync for #1005 #1010 symptom (narrow generated tests pass; broader CI fails).
Best Practices
Snapshot + diff the public surface before regeneration and compare after. Don't compare against architecture.json alone — architecture.json describes intent; the pre-sync module describes the actual contract callers depend on.
Fail closed, not open: when broad churn is detected, abort the sync with a structured diagnostic listing the removed/changed symbols. Don't auto-repair beyond a bounded loop (PDD already has this pattern for ArchitectureConformanceError).
Allow explicit override via the prompt or issue text ("breaking change: remove read_fingerprint") so intentional removals aren't blocked.
Test-file protection is distinct from module protection: tests should be additive by default. Mass deletion of unrelated test functions is the strongest signal of over-regeneration.
Similar Implementations
Griffe (griffe check -a <old-ref> -b <new-ref>) — extracts the full public surface and reports breakages with file:line precision (e.g. "Parameter kind was changed: positional or keyword -> keyword-only"). MIT-licensed, dependency-light, used by mkdocstrings.
pidiff — wraps inspect over installed package versions and exits non-zero on SemVer-incompatible changes. Pure stdlib otherwise.
astdiff — minimal AST-equality checker; useful for the test-file churn heuristic (count of removed top-level def test_* nodes).
code_diff (GumTree reimpl) — fine-grained AST tree-edit distance; overkill for this gate but useful as a churn metric.
PDD already has an ArchitectureConformanceError gate (pdd/code_generator_main.py:41, repair loop in pdd/sync_orchestration.py:2211, formatter at pdd/agentic_sync_runner.py:466). The new gate is a complement, not a replacement: arch-conformance covers required symbols from intent; this covers preservation of incumbent symbols.
Security/Edge Cases
Dynamic imports / getattr-only access won't show up in AST scans → conservatively treat any name appearing in a top-level import or __all__ as public, not just AST-defined names.
Re-exports via from .x import y must be tracked — these are the exact symbols (git, read_fingerprint) that broke in chore: PDD sync for #1005 #1010.
First-time generation (no pre-sync module exists) → gate must no-op gracefully.
.pddrc context/strength: these are configuration not code symbols. The gate should also assert that prompt-level config (model strength, context globs) from .pddrc is honored by the sync runner; this is orthogonal to symbol diffing and likely lives in the sync orchestrator's config-resolution path.
Generated prompt artifacts ("stray generated prompt artifacts" per the issue): the gate should reject sync outputs that write files outside the declared output paths.
Refined Specifications
Pre-sync snapshot: before any regeneration, snapshot the target module's public surface — module-level names not starting with _, plus everything in __all__ if defined, plus inspect.signature of each public callable. Same for classes (public methods + signatures). Store in-memory; no on-disk artifact required.
Post-sync diff: after regeneration but before the file is committed/written-final, parse the new module and compute the symbol-set delta: removed, signature_changed, kind_changed (function→class, etc.).
Hard fail conditions (abort sync with actionable diagnostic):
Any symbol from the pre-sync __all__ (explicit or implicit) is removed without an explicit prompt directive containing a marker like BREAKING: or remove <symbol>.
More than N (suggest N=3 or 20% of public surface, whichever is larger) public symbols change signature in a single sync.
More than M (suggest M=50% of pre-sync test functions) test_* functions in the paired test file are removed.
Files are written outside the declared target paths.
Diagnostic format: reuse the existing ArchitectureConformanceError / build_conformance_hard_failure_from_error formatter pattern so the agentic runner already knows how to surface it. Add a new exception class (e.g. PublicSurfaceRegressionError) parallel to ArchitectureConformanceError.
Override mechanism: prompt body may contain BREAKING-CHANGE: <symbol> lines (or equivalent in the issue description) to whitelist intentional removals. The gate parses these and allows matching deletions.
.pddrc preservation: extend the conformance check to compare resolved sync config (strength, context globs) before vs. after — if a sync silently bypasses .pddrc for an existing file with an established config, abort.
Alternative Approaches Considered
Vendor Griffe as a hard dependency — pros: battle-tested, rich diff output, handles edge cases. Cons: adds dep weight, may be more than needed; Griffe's diff vocabulary is broader than what PDD needs to gate on.
Pure stdlib (ast + inspect) — pros: zero new deps, full control over what counts as a "breaking" change for PDD's specific surface. Cons: reimplement signature-comparison logic. Recommended — the gate only needs a handful of checks (removed names, signature diff, count thresholds), all easily expressed with ast.NodeVisitor.
Pure-diff/line-count heuristic — pros: trivial. Cons: brittle, doesn't actually understand the public surface, will both false-positive (formatter changes) and false-negative (small but breaking signature change).
Architecture.json as sole source of truth — pros: already implemented. Cons: doesn't reflect actual incumbent re-exports (this is the exact gap that allowed chore: PDD sync for #1005 #1010).
Recommended Approach
Implement a PublicSurfaceRegressionError gate using stdlib ast only:
Snapshot pre-sync public symbols (module-level non-underscore names, __all__, class public methods, function signatures) from the existing module file before sync writes.
After sync candidate is generated, parse it with ast, compute the delta, and abort with a build_conformance_hard_failure_from_error-style diagnostic if any hard-fail condition is hit.
Parse BREAKING-CHANGE: / explicit removal directives from the prompt/issue to whitelist intentional changes.
Wire into the same place as the existing arch-conformance repair loop in pdd/sync_orchestration.py (around line 2211) so the agentic runner picks up the new failure class uniformly.
Apply a separate, looser test-file churn check (% of test_* functions deleted) before accepting a regenerated test file.
This keeps the change focused, dependency-free, and architecturally parallel to the existing conformance gate — minimizing the risk of the fix itself triggering the same over-regeneration symptom it's trying to prevent.
Add a PublicSurfaceRegressionError gate to pdd sync that snapshots the pre-sync module's public symbols/signatures (via stdlib ast), diffs them against the post-sync candidate, and hard-fails with an actionable diagnostic when public symbols are removed or signatures change — unless the prompt contains an explicit BREAKING-CHANGE: opt-in. A complementary churn-percentage threshold blocks broad unrelated test rewrites. Wired parallel to the existing ArchitectureConformanceError repair loop in pdd/sync_orchestration.py.
Assumptions Made
Public surface definition: module-level names not prefixed with _ (functions, classes, assignments, re-exported imports). Matches the symbols cited in the issue (git, read_fingerprint, calculate_sha256).
Signature compatibility: removed required parameters or renamed positional parameters count as breaking; added optional parameters with defaults are non-breaking.
Opt-in marker: explicit BREAKING-CHANGE: token in the prompt (or commit message of the triggering pdd change) bypasses the gate for the listed symbols.
Test churn threshold: default block if >40% of existing test functions in the file are removed or renamed without corresponding source-symbol removal. Configurable via .pddrc.
Repair loop: on gate failure, feed the diagnostic back to the generator for one repair attempt (matching the existing architecture-conformance pattern), then hard-fail.
Mature module heuristic: apply gate to any module that already exists pre-sync with >0 public symbols (no separate "maturity" flag needed for v1).
.pddrc context/strength preservation: treated as a separate but related concern — verified by ensuring the sync orchestration reads .pddrc once and reuses the resolved values across regen/test/example phases.
The new PublicSurfaceRegressionError gate adds a second sync conformance gate parallel to the existing ArchitectureConformanceError loop. Only the README workflow-logic section that documents the existing gate is on the user-facing surface and needs to mention the new gate, the BREAKING-CHANGE: opt-in directive, and the test-churn threshold.
Files to Update
README.md
Section:pdd sync Workflow Logic — step 2 (generate) gate description (currently ~lines 926–930) Change: update — extend the existing architecture-conformance-gate bullet to describe the second gate (public-surface regression + test-churn) added in this change. Draft content:
2. **generate**: Create or update the code module from the prompt. After generation, two **sync conformance gates** validate the output before it is committed:
- **Architecture conformance gate** — validates the generated code against both `architecture.json` and the prompt's `<pdd-interface>` block:
- Each declared symbol must exist in the generated code (architecture.json symbol-existence check).
- For interface entries that declare a paren-list `signature` (`module`, `cli`, and `command` types), each declared parameter name must appear in the matching function/method signature (dotted names like `ContentSelector.select` are resolved through the class body; variadic `*args`/`**kwargs` do not satisfy a declared named parameter).
- **Signature drift** is checked per parameter: annotation drift fires only when both sides specify and differ (conservative — gradually-typed code does not churn), while default drift fires whenever the prompt declares a default and the generated code drops or changes it (strict — a missing default is a runtime contract break for callers omitting the optional kwarg).
- On failure, sync retries the generation step up to `MAX_CONFORMANCE_ATTEMPTS` with a `PDD_REPAIR_DIRECTIVE` that names the function to fix and the parameters/annotations/defaults to add or restore. The retry stops early when the missing-symbol set repeats across attempts, and the final failure is surfaced as a structured `=== architecture conformance failure ===` block.
- **Public-surface regression gate** (mature modules) — snapshots the pre-sync module's public symbols and signatures via stdlib \`ast\`, then diffs the post-sync candidate. Removed top-level names, removed public methods, or backwards-incompatible signature changes (dropped parameters, removed defaults, narrowed kwargs) hard-fail with a structured \`=== public surface regression ===\` block listing each removed/changed symbol. The candidate is rejected before the working tree is touched, preserving incumbent re-exports (e.g. \`git\`, \`read_fingerprint\`, \`calculate_sha256\`) and helpers like \`derive_basename_and_language\`. A test-file churn ratio above the default 40% threshold (configurable via \`PDD_SYNC_TEST_CHURN_THRESHOLD\`) is reported in the same block so broad rewrites of unrelated test coverage are caught before commit. Opt in to an intentional break by adding a \`BREAKING-CHANGE: <reason>\` line to the prompt; the gate then permits the diff and records the directive in the run report. \`.pddrc\` context/strength values that were active at snapshot time are preserved across the repair loop so retries do not silently widen scope.
Files to Create
None.
Associated Documents
(Documents reachable from the modified prompts' <include> graph, depth ≤ 3.)
The change set Steps 3–4 outlined will edit these prompts (Step 6 will confirm the authoritative dev-unit list):
prompts/sync_orchestration_python.prompt — wires the new gate into the existing repair loop
prompts/code_generator_main_python.prompt — defines PublicSurfaceRegressionError parallel to ArchitectureConformanceError
prompts/agentic_sync_runner_python.prompt — adds the hard-failure formatter
Discovered via:prompts/sync_orchestration_python.prompt → <readme><include>README.md</include></readme> (depth 1) Section:pdd sync Workflow Logic — step 2 (generate) gate description Change: update Draft content:(same draft as the Files to Update → README.md entry above — applied once, atomically with the prompt/code edits in Step 9)
docs/whitepaper.md
Discovered via:prompts/sync_orchestration_python.prompt → <whitepaper><include>./docs/whitepaper.md</include></whitepaper> (depth 1) Section:(none) Change: none — the whitepaper describes PDD philosophy at a higher level and does not mention the sync conformance gates; grepping for `conformance`, `architecture.json`, or `public surface` yields zero matches. No edit required.
Conflicts
None.
No Changes Needed
CHANGELOG.md — handled by the release process at PR time, not at change-design time.
docs/whitepaper.md — does not reference the sync conformance gates.
docs/prompting_guide.md — covers prompt authoring; the `BREAKING-CHANGE:` opt-in is a single-line directive whose semantics are fully described in the README gate section. No new prompt convention worth promoting to the guide until the feature is in production.
CONTRIBUTING.md / AGENTS.md / CLAUDE.md — no mention of sync gates; unaffected.
docs/sync_architecture_analysis.md, docs/sync_determination_analysis.md, docs/sync_solution_*.md — design-history documents, not reached via the include graph and not user-facing reference docs.
Add a PublicSurfaceRegressionError gate to pdd sync that snapshots the pre-sync module's public AST surface (and test-file size), diffs it against the regenerated candidate, and hard-fails on removed/renamed symbols, changed signatures, or broad unrelated test churn — unless the prompt carries an explicit BREAKING-CHANGE: marker. The gate runs parallel to the existing ArchitectureConformanceError repair loop across all three pdd sync entry points.
Project Structure (from .pddrc → pdd_cli context)
Source directory:pdd/
Test directory:tests/
Example directory:context/
Language:python
Dev Units to MODIFY
Prompt
Code
Example
Test
prompts/sync_orchestration_python.prompt
pdd/sync_orchestration.py
context/sync_orchestration_example.py
tests/test_sync_orchestration.py
prompts/code_generator_main_python.prompt
pdd/code_generator_main.py
context/code_generator_main_example.py
tests/test_code_generator_main.py
prompts/agentic_sync_runner_python.prompt
pdd/agentic_sync_runner.py
context/agentic_sync_runner_example.py
tests/test_agentic_sync_runner.py
prompts/sync_main_python.prompt
pdd/sync_main.py
context/sync_main_example.py
tests/test_sync_main.py
Per-unit changes:
code_generator_main — Define PublicSurfaceRegressionError(click.UsageError) next to ArchitectureConformanceError (pdd/code_generator_main.py:41). Carries fields: removed_symbols, signature_changed, test_churn_pct, repair_directive, total_cost, model_name, plus the same BREAKING-CHANGE: opt-in semantics.
sync_orchestration — Add the public-surface gate parallel to the architecture-conformance repair loop at pdd/sync_orchestration.py:2211. Sequence per sync iteration: (a) read pre-sync code/test into memory before generate; (b) after each candidate is produced and after the existing conformance check passes, parse pre/post with ast, diff module-level def/class/async def names and their public signatures, compute test-line-delta percentage; (c) raise PublicSurfaceRegressionError if regressions exist and prompt has no BREAKING-CHANGE: directive; (d) feed repair_directive into the same PDD_REPAIR_DIRECTIVE mechanism for retry; (e) preserve .pddrcstrength/temperature/target_coverage across retries (do not reset).
agentic_sync_runner — Add _parse_public_surface_failure() (mirroring _parse_conformance_failure at pdd/agentic_sync_runner.py:202) and build_public_surface_hard_failure_from_error() (mirroring build_conformance_hard_failure_from_error at line 466) so the structured hard-failure block lists removed symbols, changed signatures, and test-churn percent. Reuses MAX_CONFORMANCE_ATTEMPTS cap.
sync_main — Mirror the new gate in the plain CLI sync path at pdd/sync_main.py:947 (parallel to the existing architecture-conformance loop).
Dev Units to CREATE
(None)
Reuse Analysis (REQUIRED for each CREATE)
N/A — no new modules. The new gate is an extension of the existing conformance-repair infrastructure:
code_generator_main already owns ArchitectureConformanceError and is the canonical home for sync-gate exception types. Extension is the simplest option; a new module would fragment the gate taxonomy.
sync_orchestration, sync_main, and agentic_sync_runner each already implement a near-identical conformance-repair loop. The simplest path is to bolt the surface-regression check onto each loop rather than introduce a parallel orchestration module.
ast (stdlib) provides the diff primitives — no third-party dependency needed.
Dependencies (may need interface updates per dependency graph)
Per the dependency graph, the modules below import from the units being modified and should be re-verified (no API breakage planned, but transitive callers should be checked):
prompts/agentic_sync_python.prompt — depends on agentic_sync_runner (re-exports surface-failure formatter)
prompts/durable_sync_runner_python.prompt — depends on agentic_sync_runner
prompts/one_session_sync_python.prompt — depends on agentic_sync_runner
prompts/ci_drift_heal_python.prompt — depends on agentic_sync_runner and sync_orchestration
prompts/agentic_checkup_python.prompt — depends on agentic_sync
prompts/commands/maintenance_python.prompt — depends on agentic_sync
prompts/architecture_sync_python.prompt — depends on code_generator_main (exception import surface)
No signatures change for downstream importers; only additional exports (PublicSurfaceRegressionError, build_public_surface_hard_failure_from_error) are introduced.
Integration Points Discovered
Integration Point
Reason for Update
Prompt File
pdd/sync_main.py:947
Mirror new gate in plain CLI sync path
prompts/sync_main_python.prompt (in MODIFY list)
pdd/agentic_sync_runner.py:202,466,1690
Mirror new gate in agentic runner repair loop and hard-failure block
prompts/agentic_sync_runner_python.prompt (in MODIFY list)
pdd/sync_orchestration.py:2211
Primary new-gate site
prompts/sync_orchestration_python.prompt (in MODIFY list)
grep -rn "from .code_generator_main import" pdd/ --include="*.py" — confirmed only sync_orchestration.py imports the exception class directly; sync_main.py and agentic_sync_runner.py import within local scope, so the same pattern can be reused for PublicSurfaceRegressionError.
ls prompts/sync_*.prompt prompts/*sync*.prompt — enumerated all sync-related prompts to confirm no additional entry point exists.
Frontend Dev Units (Cross-Layer)
Backend endpoints being created/modified: None — this is a CLI-internal change. No frontend or HTTP surface is touched.
Frontend files referencing these endpoints: N/A
Direct Edit Candidates (No Prompt)
(None) — README.md documentation updates were already scoped in Step 5 and will be handled atomically in Step 9. No other prompt-less files require edits for this change.
prompts/ — enumerated all sync_* and *sync*.prompt files; confirmed presence of sync_orchestration_python.prompt, agentic_sync_runner_python.prompt, sync_main_python.prompt, code_generator_main_python.prompt
Activity
prompt-driven-github commentedon May 14, 2026
🚀 Autonomous Solving Started!
Job ID:
XHKxVcNqzm2DPlE7braxTriggered by: @Serhan-Asad
Label:
issueView Live Progress
prompt-driven-github commentedon May 14, 2026
📋 Execution Plan
Commands:
change → syncAttempt: 1/3
Issue: #1012
prompt-driven-github commentedon May 14, 2026
⚡ Running
pdd changeJob ID:
Yab7r4Doi3qzA8Pj3njVView Live Progress
prompt-driven-github commentedon May 14, 2026
Step 3/13: Research to clarify specifications
Status: FAILED
Error Details
Automated fallback comment — agent did not execute for this step.
prompt-driven-github commentedon May 14, 2026
Step 1: Duplicate Check
Status: Similar issue exists (#1011) — original is unresolved, proceeding with this issue
Search Performed
Findings
A similar issue exists at #1011 with the same title (
pdd sync over-regenerates mature modules and breaks public compatibility) and substantially the same body content describing the PR #1010 / issue #1005 regression.Resolution status of related issue #1011:
Because the related issue is unresolved, this issue (#1012) is not being closed. Per workflow rules, this issue will proceed as a new change request for investigation. The acceptance criteria (compatibility/conformance gate on
pdd sync, preserve externally imported helpers, prevent broad unrelated test churn, fail with actionable diagnostic) will be addressed here.Other related context discovered during search:
pdd syncshould repair architecture conformance failures instead of failing genericallypdd update <code>#1005 / PR chore: PDD sync for #1005 #1010 — the originating regression that motivated this issueProceeding to Step 2: Documentation Check
prompt-driven-github commentedon May 14, 2026
Step 2: Documentation Check
Status: Partially Implemented
Documentation Reviewed
README.md,CONTRIBUTING.md,AGENTS.md,CLAUDE.mddocs/architecture_include_validation.mddocs/sync_architecture_analysis.md,docs/sync_determination_analysis.md,docs/sync_solution_o3pro.md,docs/sync_solution_opus.mdpdd/code_generator_main.py(ArchitectureConformanceError)pdd/agentic_sync_runner.py(build_conformance_hard_failure_from_error, repair-directive loop)pdd/sync_main.py,pdd/sync_orchestration.py(architecture-conformance repair loop, ref. chore: PDD sync for #865 #866)pdd/architecture_include_validation.pyand prompts (pdd checkup --validate-arch-includes)Findings
Similar code exists for an architecture-conformance gate during
pdd sync:ArchitectureConformanceError(pdd/code_generator_main.py:41) carries structuredexpected_symbols,found_symbols,missing_symbolsfields.build_conformance_hard_failure_from_error(pdd/agentic_sync_runner.py:466) formats a hard-failure diagnostic block when generated code doesn't expose the expected symbols.pdd/sync_orchestration.py(lines ~2205–2264) runs an architecture-conformance repair loop (referencing chore: PDD sync for #865 #866) that retries generation when conformance fails and surfaces a hard-block diagnostic if budget is exhausted.pdd checkup --validate-arch-includesvalidatesarchitecture.jsondependency edges against module<include>tags.However, this only covers the symbols declared in
architecture.json. The acceptance criteria in #1012 ask for a broader gate that:git,read_fingerprint,calculate_sha256, helper signatures such asderive_basename_and_language) against the post-sync output — not justarchitecture.json..pddrccontext/strength behavior in update flows.None of these broader checks exist today. The current gate is symbol-presence-against-architecture-entry only; it does not diff the pre-sync module surface, does not analyze test-coverage regressions, and does not detect bulk-rewrite churn for mature modules.
Conclusion
A partial gate exists (architecture.json symbol conformance + retry loop), but the wider compatibility gate described in this issue — diff against the pre-sync public surface, broad-churn detection on tests, and an actionable diagnostic that blocks the over-regeneration seen in PR #1010 — is not yet implemented. Proceeding with implementation.
Proceeding to Step 3: Identify Dev Units
prompt-driven-github commentedon May 14, 2026
Step 3: Research & Clarification
Status: Research Complete
Summary
Researched Python public-API diffing tools and LLM-regenerated test churn. Mature, well-established tools (Griffe, pidiff) exist for AST-based public-surface diffing; recent LLM-codegen research confirms that test-suite churn (new tests added while valid ones are discarded) is a real, measurable phenomenon — exactly what was observed in PR #1010.
Research Findings
Technical Background
inspectmodule +astmodule are enough to extract module-level callables, classes, signatures, and__all__. Higher-level tools wrap this for SemVer-style breakage detection.__all__or re-imported withfrom x import y as y. PDD'supdate_main.pyregression (git,read_fingerprint,calculate_sha256) is exactly this case: module-level re-exports used as a stable API by callers.Best Practices
architecture.jsonalone —architecture.jsondescribes intent; the pre-sync module describes the actual contract callers depend on.ArchitectureConformanceError).read_fingerprint") so intentional removals aren't blocked.Similar Implementations
griffe check -a <old-ref> -b <new-ref>) — extracts the full public surface and reports breakages with file:line precision (e.g. "Parameter kind was changed: positional or keyword -> keyword-only"). MIT-licensed, dependency-light, used by mkdocstrings.inspectover installed package versions and exits non-zero on SemVer-incompatible changes. Pure stdlib otherwise.def test_*nodes).PDD already has an
ArchitectureConformanceErrorgate (pdd/code_generator_main.py:41, repair loop inpdd/sync_orchestration.py:2211, formatter atpdd/agentic_sync_runner.py:466). The new gate is a complement, not a replacement: arch-conformance covers required symbols from intent; this covers preservation of incumbent symbols.Security/Edge Cases
getattr-only access won't show up in AST scans → conservatively treat any name appearing in a top-levelimportor__all__as public, not just AST-defined names.from .x import ymust be tracked — these are the exact symbols (git,read_fingerprint) that broke in chore: PDD sync for #1005 #1010..pddrccontext/strength: these are configuration not code symbols. The gate should also assert that prompt-level config (model strength, context globs) from.pddrcis honored by the sync runner; this is orthogonal to symbol diffing and likely lives in the sync orchestrator's config-resolution path.Refined Specifications
_, plus everything in__all__if defined, plusinspect.signatureof each public callable. Same for classes (public methods + signatures). Store in-memory; no on-disk artifact required.removed,signature_changed,kind_changed(function→class, etc.).__all__(explicit or implicit) is removed without an explicit prompt directive containing a marker likeBREAKING:orremove <symbol>.test_*functions in the paired test file are removed.ArchitectureConformanceError/build_conformance_hard_failure_from_errorformatter pattern so the agentic runner already knows how to surface it. Add a new exception class (e.g.PublicSurfaceRegressionError) parallel toArchitectureConformanceError.BREAKING-CHANGE: <symbol>lines (or equivalent in the issue description) to whitelist intentional removals. The gate parses these and allows matching deletions..pddrcpreservation: extend the conformance check to compare resolved sync config (strength, context globs) before vs. after — if a sync silently bypasses.pddrcfor an existing file with an established config, abort.Alternative Approaches Considered
ast+inspect) — pros: zero new deps, full control over what counts as a "breaking" change for PDD's specific surface. Cons: reimplement signature-comparison logic. Recommended — the gate only needs a handful of checks (removed names, signature diff, count thresholds), all easily expressed withast.NodeVisitor.Recommended Approach
Implement a
PublicSurfaceRegressionErrorgate using stdlibastonly:__all__, class public methods, function signatures) from the existing module file before sync writes.ast, compute the delta, and abort with abuild_conformance_hard_failure_from_error-style diagnostic if any hard-fail condition is hit.BREAKING-CHANGE:/ explicit removal directives from the prompt/issue to whitelist intentional changes.pdd/sync_orchestration.py(around line 2211) so the agentic runner picks up the new failure class uniformly.test_*functions deleted) before accepting a regenerated test file.This keeps the change focused, dependency-free, and architecturally parallel to the existing conformance gate — minimizing the risk of the fix itself triggering the same over-regeneration symptom it's trying to prevent.
Sources
Proceeding to Step 4: Documentation Changes
prompt-driven-github commentedon May 14, 2026
Step 4: Requirements Clarity Check
Status: Requirements Clear
Clarity Assessment
Understanding Summary
Add a
PublicSurfaceRegressionErrorgate topdd syncthat snapshots the pre-sync module's public symbols/signatures (via stdlibast), diffs them against the post-sync candidate, and hard-fails with an actionable diagnostic when public symbols are removed or signatures change — unless the prompt contains an explicitBREAKING-CHANGE:opt-in. A complementary churn-percentage threshold blocks broad unrelated test rewrites. Wired parallel to the existingArchitectureConformanceErrorrepair loop inpdd/sync_orchestration.py.Assumptions Made
_(functions, classes, assignments, re-exported imports). Matches the symbols cited in the issue (git,read_fingerprint,calculate_sha256).BREAKING-CHANGE:token in the prompt (or commit message of the triggeringpdd change) bypasses the gate for the listed symbols..pddrc..pddrccontext/strength preservation: treated as a separate but related concern — verified by ensuring the sync orchestration reads.pddrconce and reuses the resolved values across regen/test/example phases.Proceeding to Step 5: Documentation Changes
prompt-driven-github commentedon May 14, 2026
Step 5: Documentation Changes
Status: Documentation Updates Required
Summary
The new
PublicSurfaceRegressionErrorgate adds a second sync conformance gate parallel to the existingArchitectureConformanceErrorloop. Only the README workflow-logic section that documents the existing gate is on the user-facing surface and needs to mention the new gate, theBREAKING-CHANGE:opt-in directive, and the test-churn threshold.Files to Update
README.mdSection:
pdd syncWorkflow Logic — step 2 (generate) gate description (currently ~lines 926–930)Change: update — extend the existing architecture-conformance-gate bullet to describe the second gate (public-surface regression + test-churn) added in this change.
Draft content:
Files to Create
None.
Associated Documents
(Documents reachable from the modified prompts'
<include>graph, depth ≤ 3.)The change set Steps 3–4 outlined will edit these prompts (Step 6 will confirm the authoritative dev-unit list):
prompts/sync_orchestration_python.prompt— wires the new gate into the existing repair loopprompts/code_generator_main_python.prompt— definesPublicSurfaceRegressionErrorparallel toArchitectureConformanceErrorprompts/agentic_sync_runner_python.prompt— adds the hard-failure formatterTraversing
<include>directives (depth ≤ 3,.md/.rst/.txtonly, cycle-safe):README.mdDiscovered via:
prompts/sync_orchestration_python.prompt→<readme><include>README.md</include></readme>(depth 1)Section:
pdd syncWorkflow Logic — step 2 (generate) gate descriptionChange: update
Draft content: (same draft as the
Files to Update → README.mdentry above — applied once, atomically with the prompt/code edits in Step 9)docs/whitepaper.mdDiscovered via:
prompts/sync_orchestration_python.prompt→<whitepaper><include>./docs/whitepaper.md</include></whitepaper>(depth 1)Section: (none)
Change: none — the whitepaper describes PDD philosophy at a higher level and does not mention the sync conformance gates; grepping for `conformance`, `architecture.json`, or `public surface` yields zero matches. No edit required.
Conflicts
None.
No Changes Needed
CHANGELOG.md— handled by the release process at PR time, not at change-design time.docs/whitepaper.md— does not reference the sync conformance gates.docs/prompting_guide.md— covers prompt authoring; the `BREAKING-CHANGE:` opt-in is a single-line directive whose semantics are fully described in the README gate section. No new prompt convention worth promoting to the guide until the feature is in production.CONTRIBUTING.md/AGENTS.md/CLAUDE.md— no mention of sync gates; unaffected.docs/sync_architecture_analysis.md,docs/sync_determination_analysis.md,docs/sync_solution_*.md— design-history documents, not reached via the include graph and not user-facing reference docs.Proceeding to Step 6: Identify Dev Units
prompt-driven-github commentedon May 14, 2026
Step 6: Dev Units Identified
Status: Found 4 dev units
Change Summary
Add a
PublicSurfaceRegressionErrorgate topdd syncthat snapshots the pre-sync module's public AST surface (and test-file size), diffs it against the regenerated candidate, and hard-fails on removed/renamed symbols, changed signatures, or broad unrelated test churn — unless the prompt carries an explicitBREAKING-CHANGE:marker. The gate runs parallel to the existingArchitectureConformanceErrorrepair loop across all threepdd syncentry points.Project Structure (from
.pddrc→pdd_clicontext)pdd/tests/context/pythonDev Units to MODIFY
prompts/sync_orchestration_python.promptpdd/sync_orchestration.pycontext/sync_orchestration_example.pytests/test_sync_orchestration.pyprompts/code_generator_main_python.promptpdd/code_generator_main.pycontext/code_generator_main_example.pytests/test_code_generator_main.pyprompts/agentic_sync_runner_python.promptpdd/agentic_sync_runner.pycontext/agentic_sync_runner_example.pytests/test_agentic_sync_runner.pyprompts/sync_main_python.promptpdd/sync_main.pycontext/sync_main_example.pytests/test_sync_main.pyPer-unit changes:
code_generator_main— DefinePublicSurfaceRegressionError(click.UsageError)next toArchitectureConformanceError(pdd/code_generator_main.py:41). Carries fields:removed_symbols,signature_changed,test_churn_pct,repair_directive,total_cost,model_name, plus the sameBREAKING-CHANGE:opt-in semantics.sync_orchestration— Add the public-surface gate parallel to the architecture-conformance repair loop atpdd/sync_orchestration.py:2211. Sequence per sync iteration: (a) read pre-sync code/test into memory before generate; (b) after each candidate is produced and after the existing conformance check passes, parse pre/post withast, diff module-leveldef/class/async defnames and their public signatures, compute test-line-delta percentage; (c) raisePublicSurfaceRegressionErrorif regressions exist and prompt has noBREAKING-CHANGE:directive; (d) feedrepair_directiveinto the samePDD_REPAIR_DIRECTIVEmechanism for retry; (e) preserve.pddrcstrength/temperature/target_coverageacross retries (do not reset).agentic_sync_runner— Add_parse_public_surface_failure()(mirroring_parse_conformance_failureatpdd/agentic_sync_runner.py:202) andbuild_public_surface_hard_failure_from_error()(mirroringbuild_conformance_hard_failure_from_errorat line 466) so the structured hard-failure block lists removed symbols, changed signatures, and test-churn percent. ReusesMAX_CONFORMANCE_ATTEMPTScap.sync_main— Mirror the new gate in the plain CLI sync path atpdd/sync_main.py:947(parallel to the existing architecture-conformance loop).Dev Units to CREATE
(None)
Reuse Analysis (REQUIRED for each CREATE)
N/A — no new modules. The new gate is an extension of the existing conformance-repair infrastructure:
code_generator_mainalready ownsArchitectureConformanceErrorand is the canonical home for sync-gate exception types. Extension is the simplest option; a new module would fragment the gate taxonomy.sync_orchestration,sync_main, andagentic_sync_runnereach already implement a near-identical conformance-repair loop. The simplest path is to bolt the surface-regression check onto each loop rather than introduce a parallel orchestration module.ast(stdlib) provides the diff primitives — no third-party dependency needed.Dependencies (may need interface updates per dependency graph)
Per the dependency graph, the modules below import from the units being modified and should be re-verified (no API breakage planned, but transitive callers should be checked):
prompts/agentic_sync_python.prompt— depends onagentic_sync_runner(re-exports surface-failure formatter)prompts/durable_sync_runner_python.prompt— depends onagentic_sync_runnerprompts/one_session_sync_python.prompt— depends onagentic_sync_runnerprompts/ci_drift_heal_python.prompt— depends onagentic_sync_runnerandsync_orchestrationprompts/agentic_checkup_python.prompt— depends onagentic_syncprompts/commands/maintenance_python.prompt— depends onagentic_syncprompts/architecture_sync_python.prompt— depends oncode_generator_main(exception import surface)No signatures change for downstream importers; only additional exports (
PublicSurfaceRegressionError,build_public_surface_hard_failure_from_error) are introduced.Integration Points Discovered
pdd/sync_main.py:947prompts/sync_main_python.prompt(in MODIFY list)pdd/agentic_sync_runner.py:202,466,1690prompts/agentic_sync_runner_python.prompt(in MODIFY list)pdd/sync_orchestration.py:2211prompts/sync_orchestration_python.prompt(in MODIFY list)Search performed:
grep -rn "ArchitectureConformanceError\|conformance" pdd/agentic_sync_runner.py pdd/code_generator_main.py pdd/sync_main.py pdd/sync_orchestration.pygrep -rn "from .code_generator_main import" pdd/ --include="*.py"— confirmed onlysync_orchestration.pyimports the exception class directly;sync_main.pyandagentic_sync_runner.pyimport within local scope, so the same pattern can be reused forPublicSurfaceRegressionError.ls prompts/sync_*.prompt prompts/*sync*.prompt— enumerated all sync-related prompts to confirm no additional entry point exists.Frontend Dev Units (Cross-Layer)
Backend endpoints being created/modified: None — this is a CLI-internal change. No frontend or HTTP surface is touched.
Frontend files referencing these endpoints: N/A
Direct Edit Candidates (No Prompt)
(None) —
README.mddocumentation updates were already scoped in Step 5 and will be handled atomically in Step 9. No other prompt-less files require edits for this change.Files Explored
.pddrc(project structure resolution;pdd_clicontext)prompts/— enumerated allsync_*and*sync*.promptfiles; confirmed presence ofsync_orchestration_python.prompt,agentic_sync_runner_python.prompt,sync_main_python.prompt,code_generator_main_python.promptpdd/sync_orchestration.py(lines 2180–2280: existing conformance repair loop)pdd/agentic_sync_runner.py(lines 42, 188–500, 1316, 1667–1736: conformance parsing, repair, hard-failure)pdd/code_generator_main.py(line 41+:ArchitectureConformanceErrorclass)pdd/sync_main.py(lines 940–1030: parallel conformance loop)context/— confirmed example files exist for all four modulestests/— confirmed test files exist for all four modulesProceeding to Step 7: Architecture Review
99 remaining items
fix(sync): external review iter-8 follow-ups for PR #1015
docs(sync): document PDD_ALLOW_EMPTY_GENERATION in CHANGELOG + archit…
fix(sync): iter-14: address cloud-test regressions for PR #1015
Merge pull request #1015 from promptdriven/change/issue-1012
fix(#1149): wire clean_restart end-to-end + restore regressed paths
fix(#1149): wire clean_restart end-to-end + restore regressed paths
feat(sync): add public-surface and test-churn gates to pdd sync (prom…
fix(sync): external review iter-8 follow-ups for PR promptdriven#1015
docs(sync): document PDD_ALLOW_EMPTY_GENERATION in CHANGELOG + archit…
fix(sync): iter-14: address cloud-test regressions for PR promptdrive…
Merge pull request promptdriven#1015 from promptdriven/change/issue-1012
fix(promptdriven#1149): wire clean_restart end-to-end + restore regre…
feat(conformance): extract the gate layer into pdd/conformance (Phase B)