Skip to content

Commit 57b12fb

Browse files
aaspinwallclaude
aaspinwall
and
claude
authored
feat(addon): build & test the add-on locally as a system add-on (#955)
* WIP build locally * feat(addon): build the system add-on against a local backend Add a `:local` build path that produces the system/built-in add-on (system id, dev-mode bundle) wired to a locally running Send backend + client, so the system-add-on code path can be exercised end-to-end against localhost. - build:dev:system:local / sync:builtin:local / dev:builtin:local scripts set ADDON_ENV=local. - scripts/build.sh, on ADDON_ENV=local, builds in Vite development mode (import.meta.env.MODE === 'development' so the frontend probes the local backend for storage type and prod Sentry stays off) and forces localhost URLs (VITE_SEND_SERVER_URL/CLIENT_URL, stage OIDC) as exported VITE_* vars, which Vite's loadEnv prioritizes over the .env prod block. Each is overridable inline for different local ports. - README documents the workflow, that CORS already works (the backend auto-allows moz-extension:// origins), and three ways to get past the tls-dev-proxy self-signed cert (OS trust + enterprise roots, a persistent profile exception, or targeting the plain-HTTP backend on :8080). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent f20c40e commit 57b12fb

6 files changed

Lines changed: 321 additions & 10 deletions

File tree

‎packages/addon/README.md‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,3 +16,117 @@ lerna run build:dev --scope=addon
1616
This outputs an xpi file at `packages/addon`, you should see something like `tbpro-add-on-0.1.23.xpi`.
1717

1818
You can use this xpi file to install the addon in your Thunderbird for testing.
19+
20+
## Testing as the built-in / system add-on (local comm-central build)
21+
22+
The standalone xpi above runs under the standalone add-on id (`tbpro-add-on@thunderbird.net`).
23+
To test the add-on the way Thunderbird ships it — as the **built-in / system add-on**
24+
(id `tbpro-system-add-on@thunderbird.net`, loaded from `resource://builtin-addons/thundermail/`,
25+
enabled by default, no install flow) — sync a system-id build into a local comm-central checkout and
26+
rebuild Thunderbird.
27+
28+
The add-on is vendored in comm-central at
29+
`comm/mail/extensions/builtin-addons/thundermail/extension/` and packaged into `messenger.jar` via
30+
that directory's `jar.mn`. These scripts refresh that copy from this repo's `dist/`:
31+
32+
```sh
33+
# Point at your comm-central source root (the dir containing the `comm/` subdir). Required.
34+
export TB_COMM_SRC=/path/to/your/tb-build/source
35+
36+
# One-shot: build with the system id + rsync into the comm tree
37+
pnpm --filter addon sync:builtin
38+
39+
# …or watch mode: auto re-syncs on every source change
40+
pnpm --filter addon dev:builtin
41+
```
42+
43+
### Pointing the built-in add-on at a local backend
44+
45+
By default a system build bakes in the prod/stage Send hosts from your `.env`. To build the
46+
**same system add-on** but wired to a locally running Send backend + client, use the `:local`
47+
variants:
48+
49+
```sh
50+
# One-shot: system-id build against localhost + rsync into the comm tree
51+
pnpm --filter addon sync:builtin:local
52+
53+
# …or watch mode
54+
pnpm --filter addon dev:builtin:local
55+
56+
# (or just build, without syncing)
57+
pnpm --filter addon build:dev:system:local
58+
```
59+
60+
These set `ADDON_ENV=local`, which makes `scripts/build.sh` build in Vite's **development** mode
61+
(so `import.meta.env.MODE === 'development'` — the frontend then probes the local backend for its
62+
storage type instead of assuming bucket storage, and prod Sentry stays off) and force the localhost
63+
URLs into the bundle:
64+
65+
- `VITE_SEND_SERVER_URL=https://localhost:8088` (also used for `wss://…/api/ws` uploads)
66+
- `VITE_SEND_CLIENT_URL=http://localhost:5173`
67+
- `VITE_OIDC_ROOT_URL=https://auth-stage.tb.pro/realms/tbpro/` (no local Keycloak needed)
68+
69+
They're exported as `VITE_*` process env vars, which Vite's `loadEnv` prioritizes over `.env`, so
70+
they win over the prod block your `.env` ends with. Override any of them inline if your local ports
71+
differ, e.g. `VITE_SEND_SERVER_URL=https://localhost:9000 pnpm --filter addon build:dev:system:local`.
72+
73+
**CORS is already handled** — the Send backend auto-allows any `moz-extension://` origin
74+
(`packages/send/backend/src/origins.ts`), and the built-in/system add-on still runs under a
75+
`moz-extension://` origin, so its requests pass CORS with no add-on-side change. Just make sure the
76+
local backend has `SEND_BACKEND_CORS_ORIGINS` set (it throws on startup otherwise) and including
77+
`http://localhost:5173`.
78+
79+
**The self-signed cert is the real gotcha.** The Send dev stack serves `https://localhost:8088`
80+
through the tls-dev-proxy (`packages/send/backend/tls-dev-proxy/`) using a self-signed cert.
81+
Thunderbird's extension `fetch` can't skip cert validation and gets no interactive prompt at
82+
startup, so an untrusted cert fails the TLS handshake — and Gecko reports that as
83+
`CORS request did not succeed` with `Status code: (null)` (a misleading message: it is *not* a CORS
84+
rejection, which would carry a real status and an "Access-Control-Allow-Origin missing" error). Pick
85+
one of these:
86+
87+
- **Trust the cert at the OS level (recommended — survives `--temp-profile`, keeps `wss` uploads
88+
working).** Import the proxy cert into the macOS keychain and enable Gecko's enterprise-roots
89+
support per run:
90+
91+
```sh
92+
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain \
93+
packages/send/backend/tls-dev-proxy/certs/localhost.crt
94+
./mach run --temp-profile --setpref security.enterprise_roots.enabled=true
95+
```
96+
97+
`enterprise_roots` makes Gecko read the macOS keychain; passing it as a `--setpref` means it
98+
applies even to a fresh temp profile.
99+
100+
- **Add a permanent cert exception in a persistent profile.** Thunderbird → Settings → Privacy &
101+
Security → Certificates → Manage Certificates → *Servers* → Add Exception → `https://localhost:8088`.
102+
Downside: a `--temp-profile` run discards it, which fights the stale-cache advice below — use a
103+
named profile (`./mach run -P <profile>`) instead.
104+
105+
- **Skip TLS entirely (fastest; breaks `wss` uploads).** Point the add-on at the plain-HTTP backend
106+
on `:8080`. `http://localhost` is a trustworthy secure context in Gecko, so there's no cert and no
107+
mixed-content block, and it works with `--temp-profile`:
108+
109+
```sh
110+
VITE_SEND_SERVER_URL=http://localhost:8080 pnpm --filter addon build:dev:system:local
111+
```
112+
113+
Caveat: the file-upload WebSocket is hardcoded to `wss://` (`send/frontend/src/lib/helpers.ts`), so
114+
uploads fail over plain http; trpc / auth / storage-type / dashboard all work.
115+
116+
Then, in the comm tree, repackage and run:
117+
118+
```sh
119+
cd "$TB_COMM_SRC"
120+
./mach build faster # repackages messenger.jar + regenerates built_in_addons.json
121+
./mach run --temp-profile # fresh profile each launch — avoids stale built-in caches
122+
```
123+
124+
Notes:
125+
126+
- `build:dev:system` rewrites only the built `dist/manifest.json` to the system id (via
127+
`scripts/set-system-id.ts`); the source `public/manifest.json` keeps the standalone id.
128+
- The absolute `url(/fonts/...)` → relative rewrite that the built-in CSS needs (Bug 2036665) is
129+
already handled by `scripts/build.sh`, so the synced build passes Thunderbird's
130+
`browser_parsable_css.js` static check.
131+
- Use `--temp-profile` after rebuilds; a stale `addonStartup.json.lz4` / startup cache can otherwise
132+
drop or mis-resolve the built-in (Bugs 1651838 / 1964408).

‎packages/addon/package.json‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,17 +11,23 @@
1111
"scripts": {
1212
"build": "NODE_ENV=production ./scripts/build.sh",
1313
"build:dev": "./scripts/build.sh",
14+
"build:dev:system": "ADDON_VARIANT=system ./scripts/build.sh",
15+
"build:dev:system:local": "ADDON_VARIANT=system ADDON_ENV=local ./scripts/build.sh",
1416
"build:dev:watch": "nodemon -e vue,ts,js,json --ignore 'dist*/' --ignore manifest.json --watch './{src,public}/**/*' --watch '../send/frontend/src/**/*' --exec 'pnpm build:dev'",
1517
"build:watch": "nodemon -e vue,ts,js,json --watch './{src,public}/**/*' --exec 'pnpm build'",
1618
"ci:validate": "pnpm lint:all; pnpm prettier:all; pnpm run test",
1719
"deploy-xpi": "bun run ./scripts/deploy_xpi.ts",
1820
"dev": "vite --host",
21+
"dev:builtin": "nodemon -e vue,ts,js,json --ignore 'dist*/' --ignore manifest.json --watch './{src,public}/**/*' --watch '../send/frontend/src/**/*' --exec 'pnpm sync:builtin'",
22+
"dev:builtin:local": "nodemon -e vue,ts,js,json --ignore 'dist*/' --ignore manifest.json --watch './{src,public}/**/*' --watch '../send/frontend/src/**/*' --exec 'pnpm sync:builtin:local'",
1923
"lint:all": "pnpx eslint 'src/**/*.{ts,vue,js}' 'public/api/**/*.js' --fix",
2024
"prettier:all": "prettier --write .",
2125
"preview": "vite preview",
2226
"release": "./scripts/release.sh",
2327
"setup": "./scripts/setup.sh",
2428
"sort-package-json": "sort-package-json",
29+
"sync:builtin": "pnpm build:dev:system && ./scripts/sync-to-builtin.sh",
30+
"sync:builtin:local": "pnpm build:dev:system:local && ./scripts/sync-to-builtin.sh",
2531
"test": "VITE_TESTING=true vitest run --silent",
2632
"test:watch": "VITE_TESTING=true vitest --watch",
2733
"test-debug": "VITE_TESTING=true vitest --inspect-brk --single-thread",

‎packages/addon/scripts/build.sh‎

Lines changed: 63 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,50 @@ else
77
echo 'Starting development build 🐣'
88
fi
99

10+
# Optional Vite mode forwarded to every `vite build`. Empty by default so the
11+
# existing builds keep running in Vite's default production mode.
12+
MODE_ARG=""
13+
14+
### Local backend target (ADDON_ENV=local, via build:dev:system:local).
15+
### Points the bundle at a locally running Send backend + client instead of the
16+
### prod/stage hosts baked into the developer's .env.
17+
if [ "$ADDON_ENV" = "local" ]; then
18+
echo 'Targeting local backend (localhost) 🏠'
19+
### Vite `build` defaults to production mode regardless of NODE_ENV. Force
20+
### development so import.meta.env.MODE === 'development': that makes the
21+
### frontend probe the local backend for its storage type (api.ts), sets
22+
### IS_DEV, and keeps prod Sentry off (sentry.ts). Production mode would
23+
### instead assume bucket storage and skip that probe.
24+
MODE_ARG="--mode development"
25+
26+
### Force localhost URLs. Exported so Vite's loadEnv reads them from
27+
### process.env, where VITE_* vars take precedence over the .env file — whose
28+
### prod block (send.tb.pro / send-backend.tb.pro) would otherwise win via
29+
### last-key-wins. Assigned with := so a developer can still override any of
30+
### them inline (e.g. a different local port) before invoking the build.
31+
: "${VITE_SEND_SERVER_URL:=https://localhost:8088}"
32+
: "${VITE_SEND_CLIENT_URL:=http://localhost:5173}"
33+
### No local Keycloak: authenticate against the stage identity provider.
34+
: "${VITE_OIDC_CLIENT_ID:=desktop}"
35+
: "${VITE_OIDC_ROOT_URL:=https://auth-stage.tb.pro/realms/tbpro/}"
36+
### Keep local builds out of the shared Sentry / PostHog projects.
37+
: "${VITE_SENTRY_DSN:=}"
38+
: "${VITE_POSTHOG_PROJECT_KEY:=}"
39+
export VITE_SEND_SERVER_URL VITE_SEND_CLIENT_URL \
40+
VITE_OIDC_CLIENT_ID VITE_OIDC_ROOT_URL \
41+
VITE_SENTRY_DSN VITE_POSTHOG_PROJECT_KEY
42+
43+
echo " VITE_SEND_SERVER_URL=$VITE_SEND_SERVER_URL"
44+
echo " VITE_SEND_CLIENT_URL=$VITE_SEND_CLIENT_URL"
45+
echo " VITE_OIDC_ROOT_URL=$VITE_OIDC_ROOT_URL"
46+
### CORS: the Send backend auto-allows any moz-extension:// origin
47+
### (origins.ts), and the built-in/system add-on still runs under a
48+
### moz-extension:// origin, so its requests pass CORS unchanged. The local
49+
### backend must still have SEND_BACKEND_CORS_ORIGINS set (it throws
50+
### otherwise) and include $VITE_SEND_CLIENT_URL, and Thunderbird must trust
51+
### the self-signed https://localhost cert (used for fetch + wss uploads).
52+
fi
53+
1054
# Get version from package.json and replace dots with hyphens
1155
VERSION=$(jq -r .version < package.json | sed 's/\./-/g')
1256

@@ -36,7 +80,7 @@ node scripts/subset-fonts.mjs
3680
echo "================================================================"
3781
echo "=============== extension UI ==================================="
3882
### Extension UI
39-
vite build --config vite.config.extension.js
83+
vite build --config vite.config.extension.js $MODE_ARG
4084
cp -R dist/extension/assets/* dist/assets/
4185
cp -R dist/extension/*.* dist/
4286
if [ -d dist/extension/chunks ]; then
@@ -48,7 +92,7 @@ rm -rf dist/extension
4892
echo "================================================================"
4993
echo "=============== management page================================="
5094
### Management page, commenting out for now
51-
vite build --config vite.config.management.js
95+
vite build --config vite.config.management.js $MODE_ARG
5296
cp -R dist/pages/assets/* dist/assets/
5397
cp -R dist/pages/*.* dist/
5498
if [ -d dist/pages/chunks ]; then
@@ -100,14 +144,30 @@ find dist/assets -name '*.css' -exec perl -pi -e \
100144
echo "================================================================"
101145
echo "=============== background.js =================================="
102146
### Build `background.js` as a library
103-
vite build --config vite.config.background.js
147+
vite build --config vite.config.background.js $MODE_ARG
104148
cp -R dist/background/* dist/
105149
# cp -R dist/background/*.map dist/f
106150
# cp -R dist/background/manifest.json dist/
107151
rm -rf dist/background
108152

109153
rm -rf dist/pages
110154

155+
### When building the system/built-in add-on variant (ADDON_VARIANT=system),
156+
### rewrite the built dist manifest's prod id to the system add-on id
157+
### (tbpro-system-add-on@thunderbird.net) so the local build matches the id
158+
### Thunderbird ships it under and exercises the id-keyed runtime guards
159+
### (installGate.ts / selfUninstall.ts). Operates on the built copy only — the
160+
### source public/manifest.json keeps the prod id. Reuses set-system-id.ts, the
161+
### same script CI uses when repacking the system XPI.
162+
if [ "$ADDON_VARIANT" = "system" ]; then
163+
echo "================================================================"
164+
echo "=============== system add-on id ==============================="
165+
### --allow-stage: a dev build's dist manifest carries the STAGE id (set-id.ts
166+
### only runs for prod), so accept STAGE too. CI's prod-XPI repack omits the
167+
### flag and keeps its strict PROD-only guard.
168+
bun run scripts/set-system-id.ts dist/manifest.json --allow-stage
169+
fi
170+
111171
cd dist
112172
# Create xpi with version number
113173
zip -r -FS ../tbpro-addon-${VERSION}.xpi *

‎packages/addon/scripts/set-system-id.ts‎

Lines changed: 34 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ import * as fs from 'fs';
22
import * as path from 'path';
33
import { pathToFileURL } from 'url';
44

5-
import { ADDON_ID_PROD, ADDON_ID_SYSTEM } from '../src/addonIds';
5+
import { ADDON_ID_PROD, ADDON_ID_STAGE, ADDON_ID_SYSTEM } from '../src/addonIds';
66

77
/**
88
* Rewrite a built add-on manifest's production id to the built-in/system add-on
@@ -13,18 +13,42 @@ import { ADDON_ID_PROD, ADDON_ID_SYSTEM } from '../src/addonIds';
1313
* Throws if the production id is not present, so a manifest that can't be
1414
* rewritten fails loudly instead of silently shipping a non-system XPI.
1515
*
16-
* Usage: bun run scripts/set-system-id.ts <path/to/manifest.json>
16+
* `allowStage` relaxes this for local built-in testing only: a plain dev build
17+
* (scripts/build.sh with no NODE_ENV) copies the source public/manifest.json,
18+
* which carries the STAGE id, so the local `build:dev:system` flow has no PROD
19+
* id to rewrite. With `allowStage` the STAGE id is also accepted and a manifest
20+
* already on the system id is a no-op (dev rebuilds re-run this). CI never sets
21+
* it, so the prod-XPI repack keeps its strict PROD-only guard.
22+
*
23+
* Usage: bun run scripts/set-system-id.ts <path/to/manifest.json> [--allow-stage]
1724
* Defaults to this package's public/manifest.json when no path is given.
1825
*/
19-
export function setSystemId(manifestPath: string): void {
26+
export function setSystemId(
27+
manifestPath: string,
28+
{ allowStage = false }: { allowStage?: boolean } = {}
29+
): void {
2030
const before = fs.readFileSync(manifestPath, 'utf8');
21-
const after = before
31+
let after = before
2232
.split(`"id": "${ADDON_ID_PROD}"`)
2333
.join(`"id": "${ADDON_ID_SYSTEM}"`);
34+
if (allowStage) {
35+
after = after
36+
.split(`"id": "${ADDON_ID_STAGE}"`)
37+
.join(`"id": "${ADDON_ID_SYSTEM}"`);
38+
}
2439

2540
if (after === before) {
41+
// Local dev rebuilds re-run this on a dist manifest that may already carry
42+
// the system id; treat that as success rather than failing the build.
43+
if (allowStage && before.includes(`"id": "${ADDON_ID_SYSTEM}"`)) {
44+
console.log(`Manifest ${manifestPath} already on the system add-on id`);
45+
return;
46+
}
47+
const ids = allowStage
48+
? `prod id "${ADDON_ID_PROD}" or stage id "${ADDON_ID_STAGE}"`
49+
: `prod id "${ADDON_ID_PROD}"`;
2650
throw new Error(
27-
`Could not find prod id "${ADDON_ID_PROD}" in ${manifestPath}; ` +
51+
`Could not find ${ids} in ${manifestPath}; ` +
2852
`manifest not rewritten to the system add-on id.`
2953
);
3054
}
@@ -42,10 +66,13 @@ function isInvokedDirectly(): boolean {
4266
}
4367

4468
if (isInvokedDirectly()) {
69+
const cliArgs = process.argv.slice(2);
70+
const allowStage = cliArgs.includes('--allow-stage');
4571
const manifestPath =
46-
process.argv[2] ?? path.resolve(__dirname, '../public/manifest.json');
72+
cliArgs.find((arg) => !arg.startsWith('--')) ??
73+
path.resolve(__dirname, '../public/manifest.json');
4774
try {
48-
setSystemId(manifestPath);
75+
setSystemId(manifestPath, { allowStage });
4976
} catch (error) {
5077
console.error('Failed to set system add-on id:', error);
5178
process.exit(1);

‎packages/addon/scripts/sync-to-builtin.sh‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Sync a built (system-id) dist/ into a local comm-central checkout's vendored
4+
# built-in add-on directory, so the add-on can be tested as the real built-in /
5+
# system add-on (loaded under resource://builtin-addons/thundermail/, enabled by
6+
# default, no install flow).
7+
#
8+
# The add-on is vendored in comm-central at
9+
# comm/mail/extensions/builtin-addons/thundermail/extension/
10+
# and packaged into messenger.jar via that dir's jar.mn. This script refreshes
11+
# that copy from this repo's dist/, then you rebuild + run from the comm tree:
12+
#
13+
# export TB_COMM_SRC=/path/to/your/tb-build/source
14+
# pnpm --filter addon sync:builtin # build (system id) + sync
15+
# cd "$TB_COMM_SRC" && ./mach build faster # repackage messenger.jar
16+
# ./mach run --temp-profile # fresh profile avoids stale builtin cache
17+
#
18+
# Run `pnpm --filter addon dev:builtin` to auto-sync on every source change.
19+
#
20+
# Requires TB_COMM_SRC to point at the comm-central source root (the dir that
21+
# contains the `comm/` subdir). No default — there can be several build trees.
22+
23+
set -euo pipefail
24+
25+
if [ -z "${TB_COMM_SRC:-}" ]; then
26+
echo "ERROR: TB_COMM_SRC is not set." >&2
27+
echo " Set it to your comm-central source root, e.g.:" >&2
28+
echo " export TB_COMM_SRC=/Users/you/WebApps/tb-build/source" >&2
29+
exit 1
30+
fi
31+
32+
DEST="$TB_COMM_SRC/comm/mail/extensions/builtin-addons/thundermail/extension"
33+
34+
if [ ! -d "$DEST" ]; then
35+
echo "ERROR: vendored add-on dir not found:" >&2
36+
echo " $DEST" >&2
37+
echo " Check TB_COMM_SRC ('$TB_COMM_SRC') points at the comm-central source root" >&2
38+
echo " and that the thundermail built-in add-on is present in that tree." >&2
39+
exit 1
40+
fi
41+
42+
# scripts/ lives in packages/addon, so dist/ is one level up from here.
43+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
44+
DIST="$SCRIPT_DIR/../dist"
45+
46+
if [ ! -f "$DIST/manifest.json" ]; then
47+
echo "ERROR: $DIST/manifest.json not found — run the build first (pnpm build:dev:system)." >&2
48+
exit 1
49+
fi
50+
51+
# Guard: the synced build must carry the system add-on id, otherwise the
52+
# built-in would run under the wrong identity and the id-keyed guards wouldn't
53+
# be exercised. sync:builtin runs build:dev:system, which sets this.
54+
if ! grep -q "tbpro-system-add-on@thunderbird.net" "$DIST/manifest.json"; then
55+
echo "ERROR: $DIST/manifest.json does not contain the system add-on id." >&2
56+
echo " Build with the system variant: pnpm build:dev:system" >&2
57+
exit 1
58+
fi
59+
60+
echo "Syncing $DIST/ -> $DEST/"
61+
# --delete so files removed from the build don't linger in the vendored copy.
62+
rsync -a --delete "$DIST/" "$DEST/"
63+
64+
echo "Synced. Now: cd \"\$TB_COMM_SRC\" && ./mach build faster && ./mach run --temp-profile"

0 commit comments

Comments
 (0)