Skip to content

Commit da6aeae

Browse files
committed
feat: add in-process local loops and repair automation
1 parent 2207bff commit da6aeae

File tree

15 files changed

+433
-33
lines changed

15 files changed

+433
-33
lines changed

CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,9 @@ All notable changes to this project are documented in this file.
2121
- Added provider-driven `componentFlows` plus richer `applyMocks(...)` context so generated React/Next adapters can emit async behavioral flow contracts with mocked fetch/timer control.
2222
- Added provider preset wrappers for router, React Query, Zustand, and Redux-style app patterns in `themis.generate.js` / `themis.generate.cjs`, plus richer inferred async input/submit/loading/success component flows.
2323
- Added an incremental `themis migrate <jest|vitest>` scaffold and runtime compatibility imports for `@jest/globals`, `vitest`, and `@testing-library/react`.
24+
- Added a zero-IPC `--isolation in-process` test mode plus `--cache` for faster local rerun loops and in-process watch execution.
25+
- Deepened provider/app adapter presets with Next navigation and auth/session wrapper metadata alongside the existing router, React Query, Zustand, and Redux presets.
26+
- Strengthened failure-to-fix automation with richer `.themis/fix-handoff.json` entries (`repairStrategy`, `candidateFiles`, `autofixCommand`) and added `.themis/migration-report.json` for migration inventories and next actions.
2427
- Added config-level `testIgnore` discovery patterns so repos can keep generated output, fixture sandboxes, and other local test noise out of default suite runs deterministically.
2528
- Added an in-repo VS Code extension scaffold for artifact-driven result viewing, reruns, and HTML report opening.
2629
- Expanded the VS Code extension scaffold with generated-review navigation for source/test/hint mappings and unresolved generation backlog.

README.md

Lines changed: 19 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ Themis is built for modern Node.js and TypeScript projects:
5656
- `testIgnore` patterns for deterministic discovery boundaries
5757
- first-party mocks, spies, and deterministic UI primitives
5858
- compatibility imports for `@jest/globals`, `vitest`, and `@testing-library/react`
59-
- `--watch` and `--rerun-failed` for tight local and agent rerun loops
59+
- `--watch`, `--rerun-failed`, `--isolation in-process`, and `--cache` for tight local and agent rerun loops
6060

6161
## Visuals
6262

@@ -88,7 +88,7 @@ npx themis test --agent
8888
Stay in a local rerun loop while editing:
8989

9090
```bash
91-
npx themis test --watch --reporter next
91+
npx themis test --watch --isolation in-process --cache --reporter next
9292
```
9393

9494
Incrementally migrate existing Jest/Vitest suites:
@@ -119,7 +119,7 @@ Generated files land under `tests/generated` by default. Each generated test:
119119

120120
Themis also supports per-file generation hints with sidecars like `src/components/Button.themis.json` so humans and agents can provide props, component flows, args, route requests, and route context. When those sidecars do not exist yet, `--write-hints` can scaffold them automatically from the current source analysis.
121121

122-
For repo-wide generation defaults, add `themis.generate.js` or `themis.generate.cjs` at the project root. Providers in that file can match source paths, supply shared props/args/flow plans, register runtime mocks for generated UI scenarios, and wrap generated component renders so generated DOM contracts run inside the same provider shells humans use in app tests. Providers can also declare preset wrapper metadata for router, React Query, Zustand, and Redux-style app state patterns.
122+
For repo-wide generation defaults, add `themis.generate.js` or `themis.generate.cjs` at the project root. Providers in that file can match source paths, supply shared props/args/flow plans, register runtime mocks for generated UI scenarios, and wrap generated component renders so generated DOM contracts run inside the same provider shells humans use in app tests. Providers can also declare preset wrapper metadata for router, Next navigation, auth/session shells, React Query, Zustand, and Redux-style app state patterns.
123123

124124
For CI and agent loops, Themis can also enforce generation quality instead of only writing files. Strict runs emit a structured backlog, fail on unresolved scan debt, and hand back exact remediation commands.
125125

@@ -148,9 +148,19 @@ Every generation run also writes:
148148
- `.themis/generate-handoff.json`: a compact agent handoff artifact with prompt-ready next actions
149149
- `.themis/generate-backlog.json`: unresolved skips, conflicts, and confidence debt with suggested fixes
150150

151+
Local test loops can also opt into a zero-IPC execution path:
152+
153+
- `npx themis test --isolation in-process`: executes suites in-process instead of worker mode
154+
- `npx themis test --watch --isolation in-process --cache`: keeps a fast local rerun loop with file-level result caching
155+
- `npx themis test --isolation worker`: keeps process isolation for CI or global-heavy suites
156+
151157
When generated tests fail, Themis also writes:
152158

153-
- `.themis/fix-handoff.json`: a deduped failure-to-fix artifact that maps generated failures back to source files, categories, and remediation commands
159+
- `.themis/fix-handoff.json`: a deduped failure-to-fix artifact that maps generated failures back to source files, categories, repair strategies, candidate files, and remediation commands
160+
161+
Migration scaffolds also write:
162+
163+
- `.themis/migration-report.json`: a machine-readable inventory of detected Jest/Vitest compatibility imports and recommended next actions
154164

155165
## Why Themis
156166

@@ -201,7 +211,10 @@ See [`docs/why-themis.md`](docs/why-themis.md) for positioning, differentiators,
201211
- `npx themis test --reporter html`: generates a next-gen HTML report file.
202212
- `npx themis test --reporter html --html-output reports/themis.html`: writes HTML report to a custom path.
203213
- `npx themis test --watch`: reruns the suite when watched project files change.
214+
- `npx themis test --watch --isolation in-process --cache`: runs a zero-IPC cached local loop for fast edit/rerun cycles.
204215
- `npx themis test --workers 8`: overrides worker count (positive integer).
216+
- `npx themis test --isolation in-process`: runs test files in-process instead of worker processes.
217+
- `npx themis test --cache`: enables file-level result caching for in-process local loops.
205218
- `npx themis test --environment jsdom`: runs tests in a browser-like DOM environment.
206219
- `npx themis test --stability 3`: runs the suite three times and classifies each test as `stable_pass`, `stable_fail`, or `unstable`.
207220
- `npx themis test --match "intent DSL"`: runs only tests whose full name matches regex.
@@ -224,9 +237,10 @@ Each run writes artifacts to `.themis/`:
224237
- `run-diff.json`: diff against the previous run, including new and resolved failures.
225238
- `run-history.json`: rolling recent-run history for agent comparison loops.
226239
- `fix-handoff.json`: source-oriented repair handoff for generated test failures.
240+
- `migration-report.json`: compatibility inventory and next actions for migrated Jest/Vitest suites.
227241
- `report.html`: interactive HTML verdict report.
228242

229-
`--agent` output includes deterministic failure fingerprints, grouped `analysis.failureClusters`, stability classifications, previous-run comparison data, and a direct pointer to `.themis/fix-handoff.json` so AI agents can jump from generated failures to exact regeneration commands.
243+
`--agent` output includes deterministic failure fingerprints, grouped `analysis.failureClusters`, stability classifications, previous-run comparison data, and a direct pointer to `.themis/fix-handoff.json` so AI agents can jump from generated failures to exact regeneration commands. Fix handoff entries also carry repair strategies, candidate files, and autofix commands for tighter failure-to-fix loops.
230244

231245
Machine-facing reporters intentionally emit compact JSON. Agents and tooling should parse the payloads rather than depend on whitespace formatting.
232246

docs/api.md

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ Default behavior:
3535
- React and Next component adapters also emit direct DOM-state contract assertions that capture visible text, inferred roles, non-event attributes, and interaction-driven UI changes
3636
- React and Next component adapters can also emit async behavioral flow contracts when `componentFlows` are inferred or supplied, including richer inferred input/submit/loading/success flows for common async forms
3737
- project-level providers from `themis.generate.js` / `themis.generate.cjs` can match source files, inject shared fixture data, register runtime mocks, and wrap generated component renders for provider-aware DOM contracts and behavioral flow coverage
38-
- provider presets can declare router, React Query, Zustand, and Redux-style wrapper metadata without hand-writing every wrapper shell
38+
- provider presets can declare router, Next navigation, auth/session, React Query, Zustand, and Redux-style wrapper metadata without hand-writing every wrapper shell
3939
- `.themis/generate-map.json` records source-to-generated-test mappings plus scenario metadata
4040
- `.themis/generate-last.json` stores the full machine-readable generate payload
4141
- `.themis/generate-handoff.json` stores a compact prompt-ready handoff payload for agents
@@ -87,6 +87,8 @@ Project-level provider modules are supported via `themis.generate.js` or `themis
8787
- `include` / `exclude` / `files`: source matching rules
8888
- any of the same static fixture keys as sidecars (`componentProps`, `componentInteractions`, `componentFlows`, `hookArgs`, `hookInteractions`, `serviceArgs`, `routeRequests`, `routeContext`, `scenarios`)
8989
- `router`: preset router wrapper metadata (`path`, `params`, `search`)
90+
- `nextNavigation`: preset Next navigation wrapper metadata (`pathname`, `params`, `searchParams`)
91+
- `auth`: preset auth/session wrapper metadata (`user`, `session`, `state`)
9092
- `reactQuery`: preset React Query wrapper metadata (`clientName`, `state`, `cache`)
9193
- `zustand`: preset Zustand wrapper metadata (`name`, `state`)
9294
- `redux`: preset Redux wrapper metadata (`slice`, `state`)
@@ -119,6 +121,8 @@ Project-level provider modules are supported via `themis.generate.js` or `themis
119121
- `element`
120122
- `withProviderShell(type, element, attrs)`
121123
- `withReactRouter(element, config?)`
124+
- `withNextNavigation(element, config?)`
125+
- `withAuthSession(element, config?)`
122126
- `withReactQuery(element, config?)`
123127
- `withZustandStore(element, config?)`
124128
- `withReduxStore(element, config?)`
@@ -132,6 +136,7 @@ Behavior:
132136
- writes or updates `themis.config.json`
133137
- adds `tests/setup.themis.js` to `setupFiles`
134138
- adds `test:themis` to `package.json` when missing
139+
- writes `.themis/migration-report.json` with detected compatibility imports and next actions
135140
- relies on built-in runtime compatibility for `@jest/globals`, `vitest`, and `@testing-library/react`
136141

137142
## `themis test` options
@@ -144,6 +149,8 @@ Behavior:
144149
| `--reporter spec\|next\|json\|agent\|html` | string | Explicit reporter override. |
145150
| `--workers <N>` | positive integer | Override worker count. Invalid values fail fast. |
146151
| `--environment node\|jsdom` | string | Override the configured test environment. |
152+
| `--isolation worker\|in-process` | string | Select worker isolation or a zero-IPC in-process execution mode. |
153+
| `--cache` | flag | Enable file-level result caching for in-process local loops. |
147154
| `-w`, `--watch` | flag | Rerun the selected suite when watched project files change. |
148155
| `--stability <N>` | positive integer | Run selected tests `N` times and classify stability (`stable_pass`, `stable_fail`, `unstable`). |
149156
| `--html-output <path>` | string | Output path for `--reporter html` (default: `.themis/report.html`). |
@@ -156,8 +163,17 @@ Migration compatibility:
156163
- imports from `@jest/globals` are supported at runtime
157164
- imports from `vitest` are supported at runtime
158165
- imports from `@testing-library/react` are supported via Themis `render`, `screen`, `fireEvent`, `waitFor`, `cleanup`, and `act`
166+
- `themis migrate <jest|vitest>` also emits `.themis/migration-report.json` with detected files and recommended next actions
167+
168+
Additional option:
169+
159170
| `--lexicon classic\|themis` | string | Human reporter terminology mode for `next/spec`. |
160171

172+
Execution note:
173+
174+
- `--watch --isolation in-process --cache` is the fastest local rerun mode
175+
- `--isolation worker` remains the safer mode for CI and global-heavy suites
176+
161177
Snapshot note:
162178

163179
- Themis no longer supports first-party snapshot files or `-u` update flows. Prefer direct assertions and generated contract tests.
@@ -182,6 +198,7 @@ Each run writes to `.themis/`:
182198
- `run-diff.json`: diff against the previous run
183199
- `run-history.json`: rolling recent-run history
184200
- `fix-handoff.json`: deduped repair artifact for generated test failures (`themis.fix.handoff.v1`)
201+
- `migration-report.json`: migration inventory for Jest/Vitest bridge scaffolds (`themis.migration.report.v1`)
185202

186203
Formal schemas:
187204

@@ -204,6 +221,7 @@ Agent payload details:
204221
- `analysis.stability` captures multi-run classifications and per-test status sequences
205222
- `analysis.comparison` reports delta stats plus new and resolved failures against the previous run
206223
- `artifacts.fixHandoff` points to `.themis/fix-handoff.json` for generated failure repair loops
224+
- fix handoff entries include `repairStrategy`, `candidateFiles`, and `autofixCommand` for machine repair loops
207225

208226
## UI Test Utilities
209227

docs/schemas/fix-handoff.v1.json

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -57,8 +57,11 @@
5757
"category",
5858
"failureCount",
5959
"failedTests",
60+
"repairStrategy",
61+
"candidateFiles",
6062
"suggestedAction",
61-
"suggestedCommand"
63+
"suggestedCommand",
64+
"autofixCommand"
6265
],
6366
"properties": {
6467
"file": { "type": "string" },
@@ -83,8 +86,14 @@
8386
"type": "array",
8487
"items": { "type": "string" }
8588
},
89+
"repairStrategy": { "type": "string" },
90+
"candidateFiles": {
91+
"type": "array",
92+
"items": { "type": "string" }
93+
},
8694
"suggestedAction": { "type": "string" },
87-
"suggestedCommand": { "type": ["string", "null"] }
95+
"suggestedCommand": { "type": ["string", "null"] },
96+
"autofixCommand": { "type": ["string", "null"] }
8897
}
8998
}
9099
},

index.d.ts

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,8 @@ export interface RunOptions {
7676
environment?: TestEnvironment;
7777
setupFiles?: string[];
7878
tsconfigPath?: string | null;
79+
isolation?: 'worker' | 'in-process';
80+
cache?: boolean;
7981
}
8082

8183
export interface ThemisConfig {
@@ -95,6 +97,23 @@ export interface MigrationResult {
9597
setupPath: string;
9698
packageJsonPath: string | null;
9799
packageUpdated: boolean;
100+
reportPath: string;
101+
report: {
102+
schema: 'themis.migration.report.v1';
103+
source: 'jest' | 'vitest';
104+
createdAt: string;
105+
summary: {
106+
matchedFiles: number;
107+
jestGlobals: number;
108+
vitest: number;
109+
testingLibraryReact: number;
110+
};
111+
files: Array<{
112+
file: string;
113+
imports: string[];
114+
}>;
115+
nextActions: string[];
116+
};
98117
}
99118

100119
export interface GenerateOptions {

src/artifacts.js

Lines changed: 43 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -256,8 +256,11 @@ function buildFixHandoffPayload(cwd, result, context) {
256256
category,
257257
failureCount: 1,
258258
failedTests: [failedTest.fullName],
259+
repairStrategy: resolveRepairStrategy(category, generatedEntry, backlogMatch),
260+
candidateFiles: buildFixCandidateFiles(generatedEntry, backlogMatch),
259261
suggestedAction: resolveFixAction(category, generatedEntry, backlogMatch),
260-
suggestedCommand: resolveFixCommand(category, generatedEntry, backlogMatch)
262+
suggestedCommand: resolveFixCommand(category, generatedEntry, backlogMatch),
263+
autofixCommand: resolveAutofixCommand(category, generatedEntry, backlogMatch)
261264
});
262265
}
263266

@@ -317,6 +320,45 @@ function resolveFixCommand(category, entry, backlogMatch) {
317320
return null;
318321
}
319322

323+
function resolveAutofixCommand(category, entry, backlogMatch) {
324+
if (backlogMatch && backlogMatch.suggestedCommand) {
325+
return backlogMatch.suggestedCommand;
326+
}
327+
if (category === 'source-drift' && entry && entry.sourceFile) {
328+
return `npx themis generate ${entry.sourceFile} --update && npx themis test --match ${JSON.stringify(path.basename(entry.testFile || entry.sourceFile))}`;
329+
}
330+
if (entry && entry.sourceFile) {
331+
return `npx themis generate ${entry.sourceFile} --update`;
332+
}
333+
return null;
334+
}
335+
336+
function resolveRepairStrategy(category, entry, backlogMatch) {
337+
if (category === 'source-drift') {
338+
return 'regenerate-source';
339+
}
340+
if (backlogMatch && backlogMatch.hintsFile) {
341+
return 'tighten-hints';
342+
}
343+
return 'inspect-contract';
344+
}
345+
346+
function buildFixCandidateFiles(entry, backlogMatch) {
347+
const files = [];
348+
if (entry && entry.sourceFile) {
349+
files.push(entry.sourceFile);
350+
}
351+
if (entry && entry.testFile) {
352+
files.push(entry.testFile);
353+
}
354+
if (entry && entry.hintsFile) {
355+
files.push(entry.hintsFile);
356+
} else if (backlogMatch && backlogMatch.hintsFile) {
357+
files.push(backlogMatch.hintsFile);
358+
}
359+
return [...new Set(files)];
360+
}
361+
320362
function buildFixNextActions(summary) {
321363
const actions = [];
322364
if (summary.generatedFailures > 0) {

0 commit comments

Comments
 (0)