-
Notifications
You must be signed in to change notification settings - Fork 1.8k
feat: add agent-analytics-starter example #1461
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Gdewilde
wants to merge
4
commits into
vercel:main
Choose a base branch
from
Gdewilde:add-agent-analytics-starter
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
158ca1b
feat: add agent-analytics-starter example
Gdewilde 772cd89
chore: add turbo.json + .npmrc, bump @apideck/agent-analytics to ^0.1.1
Gdewilde 33f40f5
chore(deps): bump @apideck/agent-analytics to ^0.2.0
Gdewilde d658143
refactor: bump @apideck/agent-analytics to ^0.3.0, rename trackDocVie…
Gdewilde File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| # PostHog project API key — the public key used by the JS SDK. Find it at: | ||
| # https://app.posthog.com/project/settings | ||
| NEXT_PUBLIC_POSTHOG_KEY= | ||
|
|
||
| # PostHog ingestion host. Use: | ||
| # https://us.i.posthog.com — PostHog US cloud | ||
| # https://eu.i.posthog.com — PostHog EU cloud | ||
| # https://svc.yoursite.com — your own reverse-proxy (dodges ad-blockers) | ||
| NEXT_PUBLIC_POSTHOG_HOST=https://us.i.posthog.com |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| { | ||
| "root": true, | ||
| "extends": "next/core-web-vitals" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| node_modules | ||
| .next | ||
| out | ||
| dist | ||
| .DS_Store | ||
| *.log | ||
| .env | ||
| .env.local | ||
| .env*.local | ||
| .vercel | ||
| .turbo |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| # Enabled to avoid deps failing to use next@canary in the monorepo. | ||
| legacy-peer-deps=true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| MIT License | ||
|
|
||
| Copyright (c) 2026 Apideck | ||
|
|
||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
|
|
||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
|
|
||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,181 @@ | ||
| --- | ||
| name: Agent Analytics Starter | ||
| slug: agent-analytics-starter | ||
| description: Track AI agent traffic (ClaudeBot, GPTBot, Perplexity, and 20+ more) in PostHog via middleware. Serves clean Markdown to agents on the same URLs — @apideck/agent-analytics wired in. | ||
| framework: Next.js | ||
| useCase: Edge Middleware | ||
| css: Plain CSS | ||
| deployUrl: https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Fapideck-libraries%2Fagent-analytics-nextjs-starter&env=NEXT_PUBLIC_POSTHOG_KEY,NEXT_PUBLIC_POSTHOG_HOST&envDescription=PostHog%20project%20API%20key%20and%20host&envLink=https%3A%2F%2Fgithub.com%2Fapideck-libraries%2Fagent-analytics-nextjs-starter%23environment-variables&project-name=agent-analytics-starter&repository-name=agent-analytics-starter | ||
| demoUrl: https://agent-analytics-nextjs-starter.vercel.app | ||
| relatedTemplates: | ||
| - bot-protection-datadome | ||
| - bot-protection-botd | ||
| --- | ||
|
|
||
| <div align="center"> | ||
|
|
||
| <img src="./public/hero.svg" alt="Agent Analytics — see the agents your JavaScript can't." width="100%" /> | ||
|
|
||
| # Agent Analytics — Next.js starter | ||
|
|
||
| ### Next.js 15 starter that tracks AI agent traffic in PostHog — drop in your API key, deploy, watch ClaudeBot show up in your dashboard. | ||
|
|
||
| **One-click deploy.** Sample `/docs/` routes that serve as HTML to browsers and **clean Markdown to AI agents**. Every Markdown fetch fires a `doc_view` event with `is_ai_bot`, `source`, and `user_agent` — ready to segment in PostHog. | ||
|
|
||
| [](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Fapideck-libraries%2Fagent-analytics-nextjs-starter&env=NEXT_PUBLIC_POSTHOG_KEY,NEXT_PUBLIC_POSTHOG_HOST&envDescription=PostHog%20project%20API%20key%20and%20host&envLink=https%3A%2F%2Fgithub.com%2Fapideck-libraries%2Fagent-analytics-nextjs-starter%23environment-variables&project-name=agent-analytics-starter&repository-name=agent-analytics-starter) | ||
|
|
||
| [**Live demo**](https://agent-analytics-nextjs-starter.vercel.app) · [**@apideck/agent-analytics**](https://github.com/apideck-libraries/agent-analytics) · [**The pattern, explained**](https://addyosmani.com/blog/agentic-engine-optimization/) | ||
|
|
||
| </div> | ||
|
|
||
| --- | ||
|
|
||
| ## What this template does | ||
|
|
||
| AI crawlers don't run JavaScript — so your client-side analytics never see them. This template closes that gap: | ||
|
|
||
| 1. **`middleware.ts`** uses [`@apideck/agent-analytics`](https://www.npmjs.com/package/@apideck/agent-analytics) to detect 20+ known AI bots (ClaudeBot, GPTBot, PerplexityBot, Google-Extended, Applebot-Extended, Bytespider, Cursor, Windsurf, and more) and capture a `doc_view` event in PostHog on every request. | ||
| 2. **`/docs/` routes** are served as clean Markdown when an agent asks (via `.md` suffix, `Accept: text/markdown`, or a known bot UA) — otherwise HTML. Same URL, two representations. | ||
| 3. **Every Markdown response** carries `Content-Signal`, `Vary: accept`, and `x-markdown-tokens` headers so agents can budget context before parsing. | ||
|
|
||
| Runs on Vercel's Fluid Compute. Zero infrastructure to manage, events land in PostHog seconds after deploy. | ||
|
|
||
| ## Quick start | ||
|
|
||
| ### 1. Deploy | ||
|
|
||
| Click the Deploy button above, or: | ||
|
|
||
| ```bash | ||
| npx create-next-app --example https://github.com/apideck-libraries/agent-analytics-nextjs-starter my-app | ||
| cd my-app | ||
| vercel --prod | ||
| ``` | ||
|
|
||
| ### 2. Set env vars | ||
|
|
||
| In the Vercel deploy prompt (or your project's env settings): | ||
|
|
||
| | Variable | Required | Example | | ||
| | -------------------------- | ------------------------- | --------------------------------------------------------------------------------- | | ||
| | `NEXT_PUBLIC_POSTHOG_KEY` | yes | `phc_xxxxxxxx` — from PostHog project settings | | ||
| | `NEXT_PUBLIC_POSTHOG_HOST` | no (defaults to US cloud) | `https://us.i.posthog.com`, `https://eu.i.posthog.com`, or your own reverse-proxy | | ||
|
|
||
| If `NEXT_PUBLIC_POSTHOG_KEY` is absent the middleware silently no-ops — nothing breaks, events just don't flow. | ||
|
|
||
| ### 3. Verify | ||
|
|
||
| ```bash | ||
| # From your local terminal, pointed at the deployment: | ||
| curl -A "ClaudeBot/1.0 probe-$(date +%s)" https://<your-deployment>.vercel.app/docs/intro | ||
| ``` | ||
|
|
||
| Open PostHog → Activity and filter events by `event = doc_view`. You should see one event with `is_ai_bot: true`, `source: ua-rewrite`, and the probe UA you sent. | ||
|
|
||
| --- | ||
|
|
||
| ## How it works | ||
|
|
||
| ``` | ||
| Agent / Browser middleware.ts PostHog | ||
| ──────────────── ──────────────────────────────────────── ────────── | ||
| │ │ | ||
| │ GET /docs/intro │ | ||
| │ Accept: text/markdown (or .md suffix, or AI-bot UA) │ | ||
| ├──────────────────────┐ │ | ||
| │ ▼ │ | ||
| │ markdownServeDecision(req) → reason │ | ||
| │ │ │ | ||
| │ ┌────────────────┴───────────────┐ │ | ||
| │ ▼ ▼ │ | ||
| │ trackDocView(req, { NextResponse.rewrite( │ | ||
| │ analytics, req.nextUrl → /md/... │ | ||
| │ source: reason, ) │ | ||
| │ properties: {...} │ | ||
| │ }) ────fire-and-forget──────keepalive fetch──►───────────► │ | ||
| │ │ | ||
| │ ◄──── 200 text/markdown ──────────────── │ | ||
| │ Content-Signal, x-markdown-tokens │ | ||
| │ │ | ||
| ``` | ||
|
|
||
| Key properties: | ||
|
|
||
| - **Fire-and-forget** — the capture is non-blocking. `keepalive: true` lets it survive after the response returns. | ||
| - **No person profiles** — `$process_person_profile: false` tells PostHog not to create one per unique bot fingerprint. | ||
| - **Stable anon distinct_id** — djb2 hash of `ip:ua` collapses repeat fetches from the same agent into one visitor. | ||
|
|
||
| ## Structure | ||
|
|
||
| ``` | ||
| . | ||
| ├── middleware.ts # The star of the show | ||
| ├── app/ | ||
| │ ├── layout.tsx | ||
| │ ├── page.tsx # Landing page with probe instructions | ||
| │ ├── globals.css | ||
| │ └── docs/[slug]/page.tsx # Human-facing docs (HTML) | ||
| ├── public/ | ||
| │ ├── md/docs/ | ||
| │ │ ├── intro.md # Agent-facing Markdown mirror | ||
| │ │ └── usage.md | ||
| │ └── llms.txt # Agent-friendly site index | ||
| ├── .env.example | ||
| ├── package.json | ||
| └── README.md | ||
| ``` | ||
|
|
||
| ## Customising | ||
|
|
||
| **Add a new `/docs/` page**: | ||
|
|
||
| 1. Create `public/md/docs/<slug>.md` — what agents see. | ||
| 2. Add an entry to the `DOCS` object in `app/docs/[slug]/page.tsx` — what browsers see. | ||
| 3. Update `public/llms.txt` so agents can discover it. | ||
|
|
||
| **Extend the mirror to cover other routes** (e.g. `/blog/*`, `/guides/*`): | ||
|
|
||
| Edit `resolveMirrorPath` in `middleware.ts`: | ||
|
|
||
| ```ts | ||
| function resolveMirrorPath(pathname: string): string | null { | ||
| if (pathname.startsWith('/docs/')) return `/md${pathname}.md` | ||
| if (pathname.startsWith('/blog/')) return `/md${pathname}.md` // ← add this | ||
| return null | ||
| } | ||
| ``` | ||
|
|
||
| Then create matching `public/md/blog/*.md` files. | ||
|
|
||
| **Swap analytics backends** — replace the PostHog adapter with a webhook, Mixpanel, or your own callback: | ||
|
|
||
| ```ts | ||
| import { trackDocView, webhookAnalytics } from '@apideck/agent-analytics' | ||
|
|
||
| const analytics = webhookAnalytics({ | ||
| url: 'https://collector.example.com/events', | ||
| headers: { Authorization: `Bearer ${process.env.COLLECTOR_TOKEN}` }, | ||
| }) | ||
| ``` | ||
|
|
||
| Any `{ capture(event) }` object is a valid adapter. | ||
|
|
||
| ## Environment variables | ||
|
|
||
| <a id="environment-variables"></a> | ||
|
|
||
| | Variable | Description | | ||
| | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| | `NEXT_PUBLIC_POSTHOG_KEY` | PostHog project API key (the public key used by the JS SDK). Find it under _Project settings → Project API Key_. | | ||
| | `NEXT_PUBLIC_POSTHOG_HOST` | PostHog ingestion host. Defaults to `https://us.i.posthog.com`. Set to `https://eu.i.posthog.com` for EU cloud, or your own reverse-proxy domain (e.g. `https://svc.example.com`) to dodge ad-blockers. | | ||
|
|
||
| ## Learn more | ||
|
|
||
| - [`@apideck/agent-analytics` on GitHub](https://github.com/apideck-libraries/agent-analytics) — the library powering this template | ||
| - [Agentic Engine Optimization](https://addyosmani.com/blog/agentic-engine-optimization/) — the case for agent-ready docs sites | ||
| - [contentsignals.org](https://contentsignals.org) — the `Content-Signal` header spec | ||
| - [developers.apideck.com](https://developers.apideck.com) — the production docs site this pattern was extracted from | ||
|
|
||
| ## License | ||
|
|
||
| MIT © Apideck | ||
66 changes: 66 additions & 0 deletions
66
edge-middleware/agent-analytics-starter/app/docs/[slug]/page.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| import Link from 'next/link' | ||
| import { notFound } from 'next/navigation' | ||
|
|
||
| const DOCS: Record<string, { title: string; intro: string }> = { | ||
| intro: { | ||
| title: 'Intro', | ||
| intro: | ||
| 'This page is served as HTML for browsers, and as clean Markdown for AI agents. The switch happens in middleware.ts — no framework changes, no JSX-in-Markdown dance.', | ||
| }, | ||
| usage: { | ||
| title: 'Usage', | ||
| intro: | ||
| 'Spoof any of ClaudeBot, GPTBot, PerplexityBot, Google-Extended, Applebot-Extended, Cursor, or Windsurf as the User-Agent and you\u2019ll receive the Markdown mirror. A `doc_view` event fires into PostHog on every Markdown fetch.', | ||
| }, | ||
| } | ||
|
|
||
| export function generateStaticParams() { | ||
| return Object.keys(DOCS).map((slug) => ({ slug })) | ||
| } | ||
|
|
||
| export default async function DocPage({ | ||
| params, | ||
| }: { | ||
| params: Promise<{ slug: string }> | ||
| }) { | ||
| const { slug } = await params | ||
| const doc = DOCS[slug] | ||
| if (!doc) notFound() | ||
|
|
||
| return ( | ||
| <main> | ||
| <p className="muted"> | ||
| <Link href="/">← Back</Link> | ||
| </p> | ||
| <span className="badge">human view (HTML)</span> | ||
| <h1>{doc.title}</h1> | ||
| <p>{doc.intro}</p> | ||
|
|
||
| <h2>See the agent view</h2> | ||
| <p> | ||
| Same URL, append <code>.md</code> or send{' '} | ||
| <code>Accept: text/markdown</code>: | ||
| </p> | ||
| <pre> | ||
| <code>{`curl /docs/${slug}.md | ||
| curl -H "Accept: text/markdown" /docs/${slug} | ||
| curl -A "ClaudeBot/1.0" /docs/${slug}`}</code> | ||
| </pre> | ||
|
|
||
| <p> | ||
| The agent gets the pre-built Markdown in{' '} | ||
| <code>public/md/docs/{slug}.md</code>, with{' '} | ||
| <code>Content-Type: text/markdown</code>, <code>Content-Signal</code>, | ||
| and <code>x-markdown-tokens</code> headers. A <code>doc_view</code>{' '} | ||
| event is captured in PostHog with <code>source</code> ={' '} | ||
| <code>md-suffix</code>, <code>accept-header</code>, or{' '} | ||
| <code>ua-rewrite</code> depending on how the agent arrived. | ||
| </p> | ||
|
|
||
| <footer> | ||
| Edit <code>app/docs/[slug]/page.tsx</code> to customize this template, | ||
| and <code>public/md/docs/*.md</code> to change what agents see. | ||
| </footer> | ||
| </main> | ||
| ) | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
README, page.tsx, and hero.svg still reference the renamed
trackDocViewfunction which no longer exists in@apideck/agent-analytics^0.3.0, causing broken copy-paste imports for users.