Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
d1dfaac
feat(storybook-reporter): add package structure and introduce first f…
akornmeier Nov 9, 2025
115c64b
feat(storybook-reporter): implement storybook reporter package with f…
akornmeier Nov 9, 2025
cc89997
chore: move adr info to appropriate directory remove unnecessary docs…
akornmeier Nov 10, 2025
5a605a4
docs: add storybook reporter integration test design
akornmeier Nov 11, 2025
a2d9c9b
feat(storybook-reporter): implement real integration tests with Story…
akornmeier Nov 11, 2025
7f2a36b
fix(storybook-reporter): correct test-runner executable path
akornmeier Nov 11, 2025
a3896fe
fix(storybook-reporter): use direct path to test-runner in node_modules
akornmeier Nov 11, 2025
e3aa4a2
fix(storybook-reporter): use require.resolve after npm install hoisting
akornmeier Nov 11, 2025
70f56dd
fix(storybook-reporter): increase beforeAll timeout to 120s
akornmeier Nov 11, 2025
ca30bd0
test: add debug logging for storybook test-runner output
akornmeier Nov 11, 2025
44fbc74
feat(storybook-reporter): start dev server before running tests
akornmeier Nov 11, 2025
789faa6
fix(storybook-reporter): correct npx path to /usr/local/bin
akornmeier Nov 11, 2025
26e01d4
fix(storybook-reporter): use storybook bin from reporter node_modules
akornmeier Nov 11, 2025
23f6f29
fix(storybook-reporter): use hoisted storybook bin from root
akornmeier Nov 11, 2025
1ab6369
fix(integration-test): await async reporter.run() calls
akornmeier Nov 11, 2025
b0de5f6
fix(storybook-reporter): update test expectations and fix port conflicts
akornmeier Nov 13, 2025
de775e0
test: increase beforeAll timeout to 240s for slower container environ…
akornmeier Nov 13, 2025
fb9bc9e
docs: add Storybook to documentation and remove debug output
akornmeier Nov 13, 2025
5a80c36
docs: consolidate Storybook implementation details into ADR-008
akornmeier Nov 13, 2025
11f6960
fix(integration-test): correct PHPUnit reporter name lookup
akornmeier Nov 13, 2025
5e4455d
chore: remove Serena configuration from version control
akornmeier Nov 13, 2025
bcb70fb
chore: remove Gemfile.lock from RSpec reporter
akornmeier Nov 13, 2025
02404c0
fix(security): resolve js-yaml vulnerability in @storybook/test-runner
nizos Nov 16, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,27 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
curl \
&& apt-get clean && rm -rf /var/lib/apt/lists/*

# Install Playwright browser dependencies for Storybook test-runner
RUN apt-get update && apt-get install -y --no-install-recommends \
libnss3 \
libnspr4 \
libatk1.0-0 \
libatk-bridge2.0-0 \
libcups2 \
libdrm2 \
libdbus-1-3 \
libxkbcommon0 \
libxcomposite1 \
libxdamage1 \
libxfixes3 \
libxrandr2 \
libgbm1 \
libpango-1.0-0 \
libcairo2 \
libasound2 \
libatspi2.0-0 \
&& apt-get clean && rm -rf /var/lib/apt/lists/*

# Configure locale
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && \
locale-gen
Expand Down
3 changes: 2 additions & 1 deletion .devcontainer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,13 @@ A consistent, isolated environment for developing TDD Guard.

| Component | Purpose |
| ------------------ | ----------------------------------------------- |
| Node.js 22 | Main CLI & Vitest/Jest reporters |
| Node.js 22 | Main CLI & Vitest/Jest/Storybook reporters |
| Python 3.11 + pipx | Pytest reporter |
| PHP 8.2 + Composer | PHPUnit reporter |
| Go 1.24 | Go reporter |
| Rust 1.89 | Rust reporter with cargo-nextest |
| Ruby 3.x + Bundler | RSpec reporter |
| Playwright deps | Browser automation for Storybook test-runner |
| Claude Code | AI assistance |
| Dev tools | Git, zsh, fzf, Docker, gh, vim, nano, git-delta |

Expand Down
3 changes: 2 additions & 1 deletion .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -58,5 +58,6 @@
},
"workspaceMount": "source=${localWorkspaceFolder},target=/workspace,type=bind,consistency=delegated",
"workspaceFolder": "/workspace",
"postCreateCommand": "/usr/local/bin/setup-dev-environment.sh"
"postCreateCommand": "/usr/local/bin/setup-dev-environment.sh",
"postStartCommand": "npx playwright install chromium --only-shell 2>/dev/null || echo 'Playwright already installed'"
}
11 changes: 7 additions & 4 deletions .devcontainer/scripts/init-firewall.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,9 @@
# - Rust: crates.io, static.crates.io, index.crates.io
# - Claude/Anthropic services: api.anthropic.com, sentry.io, statsig.com
# - JetBrains plugin marketplace: plugins.jetbrains.com
# - VS Code extension galleries: marketplace.visualstudio.com, github.gallerycdn.vsassets.io,
# - VS Code extension galleries: marketplace.visualstudio.com, github.gallerycdn.vsassets.io,
# ms-python.gallerycdn.vsassets.io, anthropic.gallerycdn.vsassets.io, mobile.events.data.microsoft.com
# - Playwright browser downloads: cdn.playwright.dev, playwright.download.prss.microsoft.com
# - Host network (for Docker operations)
#
# BLOCKED CONNECTIONS:
Expand Down Expand Up @@ -128,7 +129,9 @@ for domain in \
"github.gallerycdn.vsassets.io" \
"ms-python.gallerycdn.vsassets.io" \
"anthropic.gallerycdn.vsassets.io" \
"mobile.events.data.microsoft.com"; do
"mobile.events.data.microsoft.com" \
"cdn.playwright.dev" \
"playwright.download.prss.microsoft.com"; do
echo "Resolving $domain..."
# Retry DNS resolution with exponential backoff
for attempt in 1 2 3 4 5; do
Expand All @@ -142,12 +145,12 @@ for domain in \
sleep $delay
fi
done

if [ -z "$ips" ]; then
echo "ERROR: Failed to resolve $domain after 5 attempts"
exit 1
fi

while read -r ip; do
# Skip CNAME records and other non-IP entries
if [[ ! "$ip" =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then
Expand Down
7 changes: 6 additions & 1 deletion .devcontainer/scripts/setup-dev-environment.sh
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,12 @@ bundle install --gemfile=reporters/rspec/Gemfile
# 8. Build Rust reporter
echo ""
echo "🦀 Building Rust reporter..."
cargo build --release --manifest-path reporters/rust/Cargo.toml
cargo build --release --manifest-path reporters/rust/Cargo.toml || echo "⚠️ Rust reporter build failed (non-fatal)"

# 9. Install Playwright browsers for Storybook test-runner
echo ""
echo "🎭 Installing Playwright browsers for Storybook test-runner..."
npx playwright install chromium --only-shell

echo ""
echo "✅ Development environment setup complete!"
Expand Down
5 changes: 3 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ reporters/ # Language-specific test reporters
├── pytest/ # tdd-guard-pytest - Pytest reporter (pip)
├── rspec/ # tdd-guard-rspec - RSpec reporter (gem)
├── rust/ # tdd-guard-rust - Rust test reporter (crates.io)
├── storybook/ # tdd-guard-storybook - Storybook test-runner reporter (npm)
├── test/ # Shared test artifacts and integration tests
└── vitest/ # tdd-guard-vitest - Vitest reporter (npm)

Expand Down Expand Up @@ -65,7 +66,7 @@ docs/ # Documentation (ADRs, configuration, etc.)
TDD Guard is organized as a TypeScript project with integrated language-specific reporters:

- **src/**: Core functionality including contracts, config, storage, and validation
- **reporters/**: Language-specific test reporters (go, jest, phpunit, pytest, vitest)
- **reporters/**: Language-specific test reporters (go, jest, phpunit, pytest, rspec, rust, storybook, vitest)
- **test/**: Comprehensive test suite with integration tests and utilities

### Testing
Expand All @@ -80,7 +81,7 @@ TDD Guard is organized as a TypeScript project with integrated language-specific
#### Commands

```bash
npm run build # Build main package and workspace reporters (jest, vitest)
npm run build # Build main package and workspace reporters (jest, storybook, vitest)
npm run test # All unit tests and base integration tests
npm run test:unit # Fast unit tests only
npm run test:reporters # Test all reporter implementations
Expand Down
34 changes: 32 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ TDD Guard ensures Claude Code follows Test-Driven Development principles. When y
- **Test-First Enforcement** - Blocks implementation without failing tests
- **Minimal Implementation** - Prevents code beyond current test requirements
- **Lint Integration** - Enforces refactoring using your linting rules
- **Multi-Language Support** - TypeScript, JavaScript, Python, PHP, Go, and Rust
- **Multi-Language Support** - TypeScript, JavaScript, Python, PHP, Go, Rust, and Storybook
- **Customizable Rules** - Adjust validation rules to match your TDD style
- **Flexible Validation** - Choose faster or more capable models for your needs
- **Session Control** - Toggle on and off mid-session
Expand All @@ -33,7 +33,7 @@ TDD Guard ensures Claude Code follows Test-Driven Development principles. When y

- Node.js 22+
- Claude Code or Anthropic API key
- Test framework (Jest, Vitest, pytest, PHPUnit, Go 1.24+, or Rust with cargo/cargo-nextest)
- Test framework (Jest, Vitest, Storybook, pytest, PHPUnit, Go 1.24+, or Rust with cargo/cargo-nextest)

## Quick Start

Expand Down Expand Up @@ -117,6 +117,36 @@ export default config
- [Vitest configuration](reporters/vitest/README.md#configuration)
- [Jest configuration](reporters/jest/README.md#configuration)

#### Storybook

Install the [tdd-guard-storybook](https://www.npmjs.com/package/tdd-guard-storybook) reporter in your project:

```bash
npm install --save-dev tdd-guard-storybook
```

Add to your `.storybook/test-runner.js`:

```javascript
const { getJestConfig } = require('@storybook/test-runner')
const { StorybookReporter } = require('tdd-guard-storybook')

module.exports = {
...getJestConfig(),
reporters: [
'default',
[
StorybookReporter,
{
projectRoot: '/Users/username/projects/my-app',
},
],
],
}
```

**Note:** Storybook test-runner uses Jest under the hood, so the reporter integrates via Jest's reporter API. Specify the project root path when your Storybook config is not at the project root. See the [Storybook reporter configuration](reporters/storybook/README.md#configuration) for more details.

</details>

<details>
Expand Down
184 changes: 184 additions & 0 deletions docs/adr/008-storybook-reporter-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
# ADR-008: Storybook Reporter Design

## Status

Accepted

## Context

Storybook interaction tests (stories with `play` functions) provide valuable component test coverage, but TDD Guard didn't recognize these test failures as valid RED-phase evidence. This prevented developers from using TDD workflows with Storybook for component testing.

We needed to design a reporter that:

- Captures test results from Storybook's `@storybook/test-runner`
- Writes results in TDD Guard's standard format
- Handles the unique characteristics of Storybook's test model
- Follows the established patterns from other reporters (Vitest, Jest)

We considered several design decisions:

1. **Test granularity** - Should we report all stories or only those with tests?
2. **Module hierarchy** - How should we group stories into modules?
3. **Reporter lifecycle** - Should we write on each story or accumulate results?
4. **Render failures** - How should we handle stories that crash during render?
5. **Data extraction** - Should we query the page or use provided context?

## Decision

We will implement `tdd-guard-storybook` with the following design:

### Test Granularity

**Only report stories with `play` functions as tests.**

Stories without `play` functions are just renders and don't provide meaningful RED/GREEN signals. This aligns with how other reporters work - they don't report non-test files. TDD Guard requires actual test failures as evidence.

### Module/Test Hierarchy

**One module per story file, multiple stories become tests within that module.**

Structure:

```json
{
"testModules": [
{
"moduleId": "src/Button.stories.tsx",
"tests": [
{
"name": "Primary",
"fullName": "Button > Primary",
"state": "passed"
}
]
}
]
}
```

This matches how Vitest/Jest group tests by file. One source file = one module is intuitive and simpler to implement.

### Reporter Lifecycle

**Accumulate results during `postVisit`, write once on `onExit`.**

API:

```typescript
async postVisit(page, context) {
await reporter.onStoryResult(context)
}
async onExit() {
await reporter.onComplete()
}
```

This matches the Vitest/Jest pattern, is more efficient (one write vs N writes), provides cleaner separation of concerns, and better handles interruptions.

### Render Failure Handling

**Create synthetic failed test for stories that fail to render.**

A story with a `play` function that crashes during render is still a failing test. This matches VitestReporter pattern for module load failures and provides visibility into all failures, not just interaction test failures.

### Data Extraction Strategy

**Extract minimal data from test-runner context directly, no additional page queries.**

Start simple with what test-runner provides out of the box. Less complexity = fewer failure points. Can enhance later if needed.

### Interruption Handling

**Best-effort save on exit with "interrupted" status.**

Approach:

- Register process exit handler to catch interruptions
- Write accumulated results when possible
- Mark overall status as "interrupted"
- Partial results better than no results

This matches the Vitest/Jest pattern and TDD Guard already handles "interrupted" status.

## Consequences

### Positive

- **Consistent patterns** - Follows established Vitest/Jest reporter patterns
- **Efficient** - Single write operation instead of multiple
- **Complete coverage** - Captures both interaction test failures and render failures
- **Simple API** - Easy to configure with minimal code
- **Robust** - Handles interruptions gracefully
- **TDD-aligned** - Only reports actual tests, not passive renders

### Negative

- **Stories without `play` not tracked** - But these aren't tests, so this is intentional
- **Accumulates in memory** - Could be an issue for extremely large test suites, but matches other reporters
- **Storybook-specific** - Requires `@storybook/test-runner` as peer dependency

### Neutral

- Users need to configure `.storybook/test-runner.ts` with the reporter
- Follows same security validations as other reporters (absolute paths, project root validation)
- Results saved to standard location: `.claude/tdd-guard/data/test.json`

## Implementation Details

### Integration Test Architecture

The Storybook reporter integration tests follow the same real-execution pattern as Jest, Vitest, PHPUnit, Pytest, Go, and Rust reporters.

**Test Flow:**

```
Integration Test
Factory (storybook.ts)
Copy story files → Write configs → Spawn test-runner
@storybook/test-runner (Jest + Playwright)
Executes stories → Calls StorybookReporter hooks
StorybookReporter.onStoryResult() → Save to FileStorage
Factory reads results → Returns to test assertions
```

### Test Artifacts Structure

```
reporters/test/storybook/
├── .storybook/
│ └── main.js # Storybook configuration
├── Calculator.js # Simple component module
├── single-passing.stories.js # Story with passing assertions
├── single-failing.stories.js # Story with failing expect()
└── single-import-error.stories.js # Story importing non-existent module
```

Stories use JavaScript (`.stories.js`) to match the repository's test file convention (`.test.js`) and reduce complexity.

### Factory Implementation

The `reporters/test/factories/storybook.ts` factory:

1. **Copies test artifacts** - Uses existing `copyTestArtifacts` helper
2. **Generates test-runner config** - Writes `test-runner-jest.config.js` with reporter
3. **Starts Storybook dev server** - Spawns `storybook dev` on random port (8000-8999 range)
4. **Executes test-runner** - Spawns `test-storybook` command via `spawnSync`
5. **Captures results** - Reporter saves to FileStorage, factory reads back

Port range 8000-8999 avoids Chrome's unsafe port list (e.g., 6697 for IRC).

### Key Design Decisions from Implementation

**Test-runner integration:** Uses `@storybook/test-runner` which integrates with Jest's reporter API through `postVisit` and `onExit` hooks. The reporter accumulates results during story execution and writes once on completion.

**Async server startup:** Factory implements timeout-based Storybook server detection by monitoring stdout for "Local:" or port-specific URLs. 240-second timeout accommodates slower container environments.

**Optional reporter execution:** Integration tests use `Promise.allSettled` to make reporters optional, allowing tests to run even when some language-specific reporters aren't installed.

**Error handling:** The reporter captures both interaction test failures and render failures (import errors, component crashes) as synthetic failed tests, ensuring complete coverage of all failure modes.
Loading
Loading