Skip to content

Conversation

@ashutosh-rath02
Copy link
Contributor

@ashutosh-rath02 ashutosh-rath02 commented Jul 16, 2025

Summary

This PR adds support for exporting test results as an HTML report via the --output-html flag in the Magnitude CLI. The report includes a summary, detailed per-test results, step/action/check breakdowns, and agent thoughts (reasoning) for each step. Related to #90 #11

Changes

  • Add --output-html <file> flag to the CLI. (npx magnitude --output-html results.html)
  • Generate a detailed HTML report after test runs, including:
    • Test summary (passed/failed/total)
    • Per-test results with status and errors
    • Step/action/check breakdowns
    • Agent thoughts (if available) under each step
  • Add HTML template and rendering logic to src/runner/htmlReport.ts.
  • The output results.html is stored in root directory (location and report name TBD)

Notes

  • Agent thoughts will appear if the agent produces reasoning during test execution.
  • No frameworks used for HTML generation—just a simple string template.

Screenshots

Below is the screenshot of a dummy test that I created

import { test } from 'magnitude-test';

test('visit example.com and check title', { url: 'https://example.com' }, async (agent) => {
    await agent.check('should see Example Domain');
}); 
image

It would be great if any of the maintainers clone my branch to run an actual test with llm runs to test the thought. I do not have api keys as of now😅.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant