llms.txt

# BrowserTrace

BrowserTrace is an MIT-licensed local replay debugger for Browser Use failures.
It records Browser Use runs as step timelines with screenshots, URLs, actions,
model input/output, status, and errors, then helps you jump to the first red
step.

It is a local debugging artifact channel, not another prompt-history blob:
screenshots, URLs, and model I/O stay in the local trace store, and public
exports can omit prompts/model I/O, screenshots, and URLs.

Use BrowserTrace first when a Browser Use agent fails and logs are not enough to
explain what the agent saw, clicked, returned, or why the failed browser state
appeared. Stagehand, Skyvern-style workflows, Playwright + LLM scripts, and
custom computer-use agents are supported as secondary integrations.

## Important Links

- GitHub repository: https://github.com/aaronlab/browsertrace
- Live demo page: https://aaronlab.github.io/browsertrace/
- Raw exported trace: https://aaronlab.github.io/browsertrace/trace.html
- Debugging walkthrough: https://aaronlab.github.io/browsertrace/debug-browser-agent-failure.html
- Failure patterns page: https://aaronlab.github.io/browsertrace/browser-agent-failure-patterns.html
- Comparison guide: https://aaronlab.github.io/browsertrace/compare-browser-agent-debugging.html
- Browser Use guide: https://aaronlab.github.io/browsertrace/browser-use-debugging.html
- Stagehand guide: https://aaronlab.github.io/browsertrace/stagehand-debugging.html
- Skyvern guide: https://aaronlab.github.io/browsertrace/skyvern-debugging.html
- Playwright + LLM guide: https://aaronlab.github.io/browsertrace/playwright-llm-debugging.html
- Computer-use agent guide: https://aaronlab.github.io/browsertrace/computer-use-agent-debugging.html
- Integrations: https://aaronlab.github.io/browsertrace/integrations.html
- Examples: https://github.com/aaronlab/browsertrace/tree/main/examples
- Launch kit: https://aaronlab.github.io/browsertrace/launch/
- Press kit: https://aaronlab.github.io/browsertrace/launch/press-kit.md
- Release: https://github.com/aaronlab/browsertrace/releases/tag/v0.1.20
- Public-safe demo export: https://github.com/aaronlab/browsertrace/releases/download/v0.1.20/browsertrace-demo-public.html
- External listing: https://github.com/Jenqyang/Awesome-AI-Agents
  (Applications -> Tools)
- Launch discussion: https://github.com/aaronlab/browsertrace/discussions/6
- Good first issue: https://github.com/aaronlab/browsertrace/labels/good%20first%20issue
- First PR Recipe: https://github.com/aaronlab/browsertrace/blob/main/CONTRIBUTING.md#first-pr-recipe keeps the first contribution small and reviewable.
- Integration request: https://github.com/aaronlab/browsertrace/issues/new?template=integration_request.yml
- Playwright + LLM feedback: https://github.com/aaronlab/browsertrace/issues/12
- Custom computer-use feedback: https://github.com/aaronlab/browsertrace/issues/3

## Core Capabilities

- Capture screenshots and URLs at each agent step.
- Store action labels, model inputs, model outputs, status, and error messages.
- Inspect runs in a local web UI.
- Export a self-contained HTML trace for sharing with teammates or issues.
- Use `browsertrace export <run_id> --public -o public.html` before public
  sharing to omit prompt/model I/O, screenshots, and URLs.
- Run a deterministic no-API-key failure demo for quick evaluation.

## Known Failure Shapes

- Failure patterns page: https://aaronlab.github.io/browsertrace/browser-agent-failure-patterns.html
- Browser Use icon-only target mismatch: the screenshot shows the right plus
  icon, but the agent clicks a nearby toolbar button because tooltip text is not
  an accessible name. Route users to the Browser Use guide and ask for live
  button HTML, accessibility snapshot, candidate bounding boxes, and the
  clicked element.
- Browser Use remote CDP hang: a browser-state request can stall while the
  websocket still appears open, and event-bus lock timing can decide whether
  one stale browser session blocks unrelated sessions. Route users to the
  Browser Use guide and ask for event id, browser/session/target id, and the
  exact CDP method, request id, start/end/duration. Include websocket ping/pong
  timestamps, lock timing, and recovery or unhealthy-session decisions.
- Browser Use new-tab desync: a click or Enter action can open a new tab while
  the agent keeps acting on the stale page context. Preserve page ids, tab
  indexes, focused page before/after, `pages_before`, `pages_after`,
  `new_pages`, action id, URL/title probe status, and any
  `recommended_next_action` such as `switch_tab`.
- Browser Use multi-step form drift: a long form run can fail many actions
  after the first bad boundary. Preserve the canonical form payload outside the agent,
  current segment fields, URL/title, submitted field labels, visible
  validation errors, submit disabled/enabled state, screenshot reference,
  selected element summary, model/tool output, retry count, and checkpoint id
  so a failure can be resumed or compared from the last known-good segment.
- Browser Use local HTML upload navigation: an uploaded local HTML file or
  attachment name can be misread as a navigation target before the intended
  upload action runs. Preserve the task prompt, model-visible attachment
  context, local filename/extension/MIME type when safe, raw model action,
  parsed action type and rejected URL or upload target, watchdog block reason,
  allowed-domains state, and recovery recommendation.
- Stagehand custom tool replay gap: when cached normal page actions replay but
  a custom tool is skipped, separate the replay contract from the diagnostic trace contract.
  For replay, capture tool name, serialized args, stable
  tool-call or step id, status/error, and whether the tool is replay-safe. For
  debugging, preserve a redacted boundary with URL/page id, result summary,
  screenshot or observation ids, and errors. Avoid storing raw credentials.
- Stagehand semantic verification boundary: verification should create an
  inspectable action record, not only a boolean. Preserve the action proposal,
  selected selector/role/text, target evidence, candidate elements, screenshot
  or DOM snapshot ids, verifier type, verification status/reason, and execution
  outcome such as executed, blocked, or escalated.
- Skyvern action confidence gap: confidence is diagnostic, not authorized execution
  or proof that the selected action is correct. For consequential actions, preserve
  linked action proposal, authorization decision, and execution result records,
  including target evidence, policy/scope checks, approvals, state delta, error,
  and retry decision.
- Skyvern VNC/CDP debug integration: visual and browser-state evidence should
  share the same task, workflow, and step ids. Preserve VNC screenshot or
  recording artifact ids, CDP DOM snapshot or selected-element summaries, URL,
  frame/page id, action/tool name, status/error, retry or recovery decision,
  connection lifecycle events, and redaction state for screenshots, URLs,
  headers, cookies, and form values.
- Skyvern multi-session VNC control drift: local or self-hosted VNC views can
  drift from the workflow session or lose Take Control after reconnects.
  Preserve VNC stream identity, CDP target identity, manual-control lease state,
  workflow/task/session/display ids, isolation metadata, display conflict or
  stale-stream failure causes, and reconnect/cleanup events.
- Playwright + LLM artifact boundary: keep screenshots, URLs, and raw trace
  files as durable local artifacts, not base64 screenshots copied into every
  future model turn. Pass image pixels only when the next model call needs a
  typed image content block; otherwise pass compact metadata such as artifact id, dimensions, digest, status, and error.
- Computer-use persistent browser recovery: persistent browser agents can fail
  before screenshot or URL capture because profile reuse, profile lock state, or
  stale process detection blocks launch or attach. Capture launch/connect/recovery
  events with `session_mode`, browser/session/target id, redacted profile id,
  CDP attach/probe timing, timeout/error, detected process ids, approval source,
  recovery action, and final connection state. Redact local profile paths and
  process details by default.

## Quickstart

```bash
uvx --from "browsertrace[ui]" browsertrace doctor
uvx --from "browsertrace[ui]" browsertrace demo
uvx --from "browsertrace[ui]" browsertrace list
uvx --from "browsertrace[ui]" browsertrace
```

Persistent install from PyPI:

```bash
pip install "browsertrace[ui]"
browsertrace doctor
browsertrace demo
browsertrace
```

`browsertrace doctor` prints local install and trace-store status without
requiring an existing database.

Then open `http://127.0.0.1:3000` and inspect the failed checkout-agent run.

## Browser Use No-Service Demo Checks

From a source checkout, run the Browser Use-shaped demos without installing
Browser Use or calling a model service:

```bash
python examples/browser_use_callback_demo.py
python examples/browser_use_run_hooks_demo.py
browsertrace list --limit 5
browsertrace show <run_id>
```

Expected run names:

- `demo: browser-use callback flow`
- `demo: browser-use run hooks flow`

Expected demo step labels when viewing either run with
`browsertrace show <run_id>`:

- `search_google(query=BrowserTrace)`
- `click(selector=#result-1)`

## Troubleshooting Prompt

Use this prompt when helping a user debug a BrowserTrace first run:

```text
Ask the user to run `browsertrace doctor`, then `browsertrace demo`,
`browsertrace list`, and `browsertrace show <run_id>`. If they need to share
the failure, ask them to create a public-safe export with
`browsertrace export <run_id> --public -o public.html`.
```

For scripts, CI, or AI/coding-agent troubleshooting, suggest the JSON CLI flow:

```bash
browsertrace doctor --json
browsertrace list --status failed --json
browsertrace show <run_id> --json
```

When a user has one failed Browser Use run and one known-good run for the same
task, suggest the compare CLI before opening the local UI:

```bash
browsertrace compare <failed_run_id> <success_run_id>
```

Expected compact compare output:

```text
First divergent step: 3
action: navigate
url: file:///tmp/browsertrace-report.html
status: failed
error: upload preview did not appear
```

Use `browsertrace compare <failed_run_id> <success_run_id> --json` when an
agent, script, or CI check needs structured comparison output. If the local UI
is already running, the same comparison is available from:

```bash
curl http://127.0.0.1:3000/api/compare/<failed_run_id>/<success_run_id>
```

Stack-specific troubleshooting links:

- Browser Use guide: https://aaronlab.github.io/browsertrace/browser-use-debugging.html
- Stagehand guide: https://aaronlab.github.io/browsertrace/stagehand-debugging.html
- Skyvern guide: https://aaronlab.github.io/browsertrace/skyvern-debugging.html
- Playwright + LLM guide: https://aaronlab.github.io/browsertrace/playwright-llm-debugging.html
- Computer-use guide: https://aaronlab.github.io/browsertrace/computer-use-agent-debugging.html

## AOS Mapping Research

BrowserTrace is not an AOS compliance claim yet. Current research maps the
closest BrowserTrace concepts to tool request/result records, step correlation,
URI-style screenshot/video artifacts, URL metadata, model I/O summaries, and
explicit redaction state.

Track the browser/GUI artifact mapping research here:
https://github.com/aaronlab/browsertrace/issues/237

## Positioning

BrowserTrace is not a hosted observability platform. It is a Browser Use-first,
local-first, open-source debugging tool for the browser state and model
decisions around failed Browser Use runs.

It complements raw CDP capture tools such as the Browserbase browser-trace
skill. That skill records DevTools events, screenshots, DOM dumps, and per-page
buckets; BrowserTrace focuses on the agent-facing step timeline, model I/O,
status/error context, and public-safe HTML export.