Warn or remove repeated symbols in editable fields #459

Workflow file for this run

	name: CI

	on:
	push:
	branches: ["main"]
	pull_request:
	branches: ["main"]
	workflow_dispatch:

	# Read by default. Only "publish-latest" widens this, and only for itself, so
	# no job that executes pull-request-controlled code ever holds a write token.
	permissions:
	contents: read

	env:
	CARGO_TERM_COLOR: always
	EXTENSION_PACKAGE_DIR: extension-package/zhtw-mcp-extension
	EXTENSION_ZIP: zhtw-mcp-extension.zip

	concurrency:
	group: "ci-${{ github.ref }}"
	# Off for main so a run is never killed mid-publish. This is not free:
	# pushes to main now queue behind the previous run's whole pipeline. Only the
	# in-progress run is protected; a superseded pending run is still cancelled,
	# which is what keeps the queue to one.
	cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}

	jobs:
	check:
	strategy:
	# One platform's failure must not hide another's. The run that made this
	# necessary died on a macOS-only ruleset gate and cancelled the Linux and
	# Windows legs mid-test, so nothing said whether either of those two also
	# had something to report. "build-latest" below has carried this setting
	# all along, for the same reason.
	fail-fast: false
	matrix:
	os: [ubuntu-24.04, macos-15, windows-2025]

	runs-on: ${{ matrix.os }}
	timeout-minutes: 20

	steps:
	- uses: actions/checkout@v7
	with:
	# The ruleset provenance check reads the PR base revision. The
	# "Resolve the ruleset baseline" step below leans on this covering an
	# ordinary push too, and fetches by hand only what a force-push strands.
	fetch-depth: 0

	- name: Install Rust toolchain
	run: rustup toolchain install stable --profile minimal --component rustfmt,clippy

	- name: Rust cache
	uses: Swatinem/rust-cache@v2
	with:
	prefix-key: "v1-rust"
	shared-key: ${{ matrix.os }}

	- name: Set up Python
	uses: actions/setup-python@v7
	with:
	python-version: "3.x"

	# "make check" runs black through scripts/indent.sh, and the runner image
	# does not carry black. Only this job needs it; the build jobs run the
	# table generator, which black never sees.
	- name: Install black
	run: pip install black

	# The shell lanes of "make check". shfmt takes its style from
	# .editorconfig, so installing it here is the whole configuration. Linux
	# only, and that is deliberate: formatting does not vary by platform, and
	# checking it on one runner is what the other two would repeat. Both lanes
	# report a skip where the tool is absent, which is what the macOS and
	# Windows legs then do.
	- name: Install the shell checkers
	if: matrix.os == 'ubuntu-24.04'
	run: sudo apt-get update && sudo apt-get install -y shellcheck shfmt

	# commentflow settles comment width, which no other formatter here does:
	# rustfmt leaves a short-wrapped comment short and shfmt does not touch
	# comment text at all. scripts/indent.sh skips its reflow lane when the
	# binary is missing, and ZHTW_REQUIRE_TOOLS below turns that skip into a
	# failure, so this step is what the formatting gate stands on.
	#
	# Built from a pinned revision rather than downloaded from the release
	# tagged "latest". That tag is rolling, so the asset behind it moves, and a
	# digest read from the same release metadata proves the transfer arrived
	# intact without saying anything about which build arrived. A commit is
	# immutable and readable; bumping it is an edit to this line, which is
	# exactly the review a formatter that rewrites every comment in the tree
	# deserves. It also drops the "sudo" and the write into /usr/local/bin that
	# the tarball needed: cargo installs into ~/.cargo/bin, which is on PATH.
	#
	# The cost is about a minute of compile on a cold cache, and rust-cache
	# above carries ~/.cargo across runs, so most runs pay nothing.
	- name: Install commentflow
	if: matrix.os == 'ubuntu-24.04'
	env:
	COMMENTFLOW_REV: 329ad4178e65e700528ef111cb11a075addb4eba
	run: \|
	set -euo pipefail

	# cargo records the source revision it installed from, so a warm
	# ~/.cargo from the cache above makes this a no-op rather than a rebuild.
	cargo install --git https://github.com/sysprog21/commentflow \
	--rev "$COMMENTFLOW_REV" --locked commentflow
	commentflow --help > /dev/null

	# Every platform, because both check steps below gate on provenance and a
	# gate that means one thing on Linux and another on Windows is not a gate.
	# Each leg is its own runner with its own checkout, so each resolves the
	# baseline for itself. Writing it to GITHUB_ENV rather than repeating the
	# expression keeps the policy in one place: a baseline source added here
	# reaches both consumers, or neither.
	- name: Resolve the ruleset baseline
	shell: bash
	env:
	# The provenance gate compares against the revision before the change.
	# workflow_dispatch has neither a PR base nor a push's before SHA and
	# leaves this empty, which the Makefile and the Windows leg below both
	# read as HEAD.
	RULESET_BASELINE: ${{ github.event.pull_request.base.sha \|\| github.event.before }}
	run: \|
	# A push that creates the branch reports all zeros for the revision
	# before it. That names no commit for the remote to serve, and it
	# means what the empty value means, so default it rather than asking
	# upload-pack for a ref nobody has.
	if [ "$RULESET_BASELINE" = 0000000000000000000000000000000000000000 ]; then
	RULESET_BASELINE=
	fi
	# The checkout above takes every ref, which covers a pull request base
	# and an ordinary push alike. A force-push is the gap it does not
	# cover: the before SHA is left on no ref at all, so ask the remote for
	# that one object rather than substitute an ancestor of HEAD, which
	# would narrow a multi-commit push and let its earlier rules evade
	# provenance. Asking git first keeps every other run to a local lookup.
	if [ -n "$RULESET_BASELINE" ] &&
	! git cat-file -e "$RULESET_BASELINE^{commit}" 2> /dev/null; then
	git fetch --no-tags origin "$RULESET_BASELINE"
	fi
	echo "RULESET_BASELINE=$RULESET_BASELINE" >> "$GITHUB_ENV"

	- name: make check
	if: matrix.os != 'windows-2025'
	env:
	ZHTW_REQUIRE_TOOLS: ${{ matrix.os == 'ubuntu-24.04' && '1' \|\| '' }}
	run: make check

	# "make" is not dependable on the Windows runner (see build-latest's
	# "Build and package" step below), so run the same checks "make check"
	# would directly instead. Two of them are missing here on purpose:
	# scripts/indent.sh and the hook suite drive tar, mktemp, git worktrees and
	# four hook wrappers through whatever sh the runner happens to provide, and
	# neither a formatting rule nor a commit rule varies by platform. The Linux
	# leg is where both run; the direct "cargo fmt --check" and "black --check"
	# below are what this leg keeps of the formatter chain. The ruleset lint
	# is not one of the two: it takes the baseline the step above resolved, so
	# provenance means the same thing here as it does on the other legs.
	#
	# The rest of the list is what caught #124 and #126 turning
	# into recurring Windows-only regressions: those bugs only ever
	# surfaced on a contributor's own Windows machine because nothing in CI
	# ran the test suite there.
	- name: make check (Windows)
	if: matrix.os == 'windows-2025'
	shell: bash
	run: \|
	python scripts/gen-s2t-tables.py
	rustfmt src/engine/s2t_data.rs
	cargo metadata --locked --format-version 1 >/dev/null
	cargo test
	./scripts/clippy-lanes.sh
	cargo fmt --check
	black --check .
	python scripts/check-ruleset.py --lint \
	--baseline-ref "${RULESET_BASELINE:-HEAD}"

	# The commit-msg hook binds whoever installed it. This binds everyone else: a
	# rebase, an amend, "--no-verify", or a subject typed into the GitHub merge box
	# all reach the branch unread otherwise. It runs the same script the pre-push
	# hook runs, so a contributor with hooks and a contributor without are judged
	# by one list.
	#
	# Pull requests only, and deliberately. "base.sha..HEAD" is the work being
	# proposed, which is the thing these rules are about. The push event's range
	# is not: a merge to main replays every commit of the merged branch, so a rule
	# this log only started following recently would reject history rather than the
	# change in front of it.
	#
	# Its own job rather than a step in "check": commit messages do not vary by
	# platform, so folding it into that matrix would run one second of shell three
	# times, and this is skipped outside a pull request while "check" is not.
	commit-log:
	if: github.event_name == 'pull_request'
	runs-on: ubuntu-24.04
	timeout-minutes: 5

	steps:
	- uses: actions/checkout@v7
	with:
	fetch-depth: 0

	- name: Check the commit messages
	env:
	BASE_SHA: ${{ github.event.pull_request.base.sha }}

	# The pull request's own tip, not the checkout's HEAD. For a
	# pull_request event the checkout is a merge ref that GitHub built to
	# test the change, and its commit is authored by nobody and subjected
	# to nothing: "Merge <sha> into <sha>" is 92 columns and belongs to no
	# author. Ending the range at the head keeps this judging the commits
	# the change proposes.
	HEAD_SHA: ${{ github.event.pull_request.head.sha }}
	run: \|
	set -euo pipefail

	# The width rules are measured by a python3 helper, and the hook treats
	# a missing python3 as a lane to skip. That is right on a laptop and
	# wrong here: this job is the only thing binding a contributor without
	# hooks, so a runner image without python3 has to fail rather than
	# quietly stop counting columns.
	python3 --version

	# An unfetched base is not an empty range. Say which one this is rather
	# than reporting "nothing to check" for a check that could not run.
	for sha in "$BASE_SHA" "$HEAD_SHA"; do
	if ! git cat-file -e "${sha}^{commit}" 2>/dev/null; then
	echo "commit $sha is not in this checkout" >&2
	exit 1
	fi
	done
	# The checker comes from the base revision rather than the checkout. A
	# pull request that edits scripts/check-commit-log.sh or the rules it
	# calls would otherwise be graded by its own copy of them, which is not
	# a gate. Both scripts land in one directory because the checker finds
	# the message hook beside itself.
	#
	# Falling back to the checkout is not a hole: it only happens when the
	# base has no checker at all, and a policy that does not exist yet
	# cannot be weakened by the change introducing it.
	trusted=$(mktemp -d)
	for script in check-commit-log.sh git-commit-msg.sh; do
	if git cat-file -e "$BASE_SHA:scripts/$script" 2>/dev/null; then
	git show "$BASE_SHA:scripts/$script" > "$trusted/$script"
	else
	echo "::notice::$script is new in this change; using its own copy"
	cp "scripts/$script" "$trusted/$script"
	fi
	chmod +x "$trusted/$script"
	done

	# Merges included, matching the pre-push hook: their subjects are exempt
	# from the style rules and not from the character ones.
	git rev-list "$BASE_SHA..$HEAD_SHA" \| "$trusted/check-commit-log.sh"

	# Reads Cargo.lock against the RustSec advisory DB. Deliberately has no
	# Swatinem/rust-cache step: this job never compiles the workspace, and sharing
	# the check job's cache key would let it save an empty target/ tree over the
	# real cache.
	audit:
	runs-on: ubuntu-24.04
	timeout-minutes: 10

	steps:
	- uses: actions/checkout@v7

	- name: Install cargo-audit
	uses: taiki-e/install-action@cargo-audit

	- name: cargo audit
	run: cargo audit

	# No "needs: check". This measures a binary, it does not publish one, and
	# "publish-latest" names "check" itself, so waiting for the slowest leg of a
	# three-platform matrix bought no safety and cost a pull request about two
	# minutes before it heard its size verdict.
	check-size:
	runs-on: ubuntu-24.04
	timeout-minutes: 15

	steps:
	- uses: actions/checkout@v7

	# rustfmt because "make check-size" goes through the generator rule, which
	# runs it over the tables it just wrote. Asking for the component is what
	# keeps that from borrowing whichever one the runner image happens to ship.
	- name: Install Rust toolchain
	run: rustup toolchain install stable --profile minimal --component rustfmt

	# Its own key: rust-cache never overwrites an existing entry, so a key
	# shared with the check job on this runner would keep whichever of the two
	# trees finished first, and the release tree this job builds is not the one
	# that wins that race.
	- name: Rust cache
	uses: Swatinem/rust-cache@v2
	with:
	prefix-key: "v1-rust"
	shared-key: "ubuntu-24.04-size"

	- name: Set up Python
	uses: actions/setup-python@v7
	with:
	python-version: "3.x"

	- name: make check-size
	run: make check-size

	# Builds the WASM scanner, runs the extension's own tests, and packages the
	# unpacked and zipped extension. It lives here rather than in a workflow of
	# its own because the zip ships on the same "latest" release as the binaries,
	# and one release wants one owner: "publish-latest" below checksums it,
	# uploads it, and verifies it alongside everything else.
	package-extension:
	runs-on: ubuntu-24.04
	timeout-minutes: 20

	steps:
	- uses: actions/checkout@v7

	- name: Install Rust toolchain
	run: \|
	rustup toolchain install stable --profile minimal --component rustfmt
	rustup target add wasm32-unknown-unknown

	- name: Rust cache
	uses: Swatinem/rust-cache@v2
	with:
	prefix-key: "v1-extension-rust"
	shared-key: "ubuntu-24.04"

	- name: Set up Node.js
	uses: actions/setup-node@v7
	with:
	node-version: "22"

	- name: Install extension test dependencies
	run: npm ci --prefix extension

	- name: Install headless Chromium
	run: npx --prefix extension playwright install --with-deps chromium

	# Prebuilt, like cargo-audit above. "cargo install wasm-pack" builds it
	# from source, which only stayed cheap because rust-cache happened to carry
	# ~/.cargo/bin across runs.
	- name: Install wasm-pack
	uses: taiki-e/install-action@wasm-pack

	- name: Build scanner WASM
	run: sh extension/build-wasm.sh

	- name: Test extension
	run: npm run test:all --prefix extension

	- name: Assemble distributable extension
	run: \|
	rm -rf extension-package "$EXTENSION_ZIP"
	mkdir -p "$EXTENSION_PACKAGE_DIR"/{dist,icons}
	cp -R \
	extension/src \
	extension/styles \
	extension/manifest.json \
	extension/popup.html \
	"$EXTENSION_PACKAGE_DIR"/
	cp extension/dist/.js extension/dist/.wasm "$EXTENSION_PACKAGE_DIR"/dist/
	if [ -d extension/dist/snippets ]; then
	cp -R extension/dist/snippets "$EXTENSION_PACKAGE_DIR"/dist/
	fi
	cp extension/icons/icon-{16,32,48,128}.png "$EXTENSION_PACKAGE_DIR"/icons/
	(cd extension-package && zip -r "../$EXTENSION_ZIP" zhtw-mcp-extension)

	# Deliberately NOT named "zhtw-mcp-*": publish-latest downloads that
	# pattern into ./dist, and an unpacked tree there would be checksummed and
	# uploaded file by file.
	- name: Upload unpacked extension artifact
	uses: actions/upload-artifact@v7
	with:
	name: extension-unpacked
	path: ${{ env.EXTENSION_PACKAGE_DIR }}
	if-no-files-found: error

	- name: Upload zipped extension artifact
	uses: actions/upload-artifact@v7
	with:
	name: zhtw-mcp-extension
	path: ${{ env.EXTENSION_ZIP }}
	if-no-files-found: error

	# Rolling binaries for the "latest" tag. This is the project's release: it
	# is not a prerelease, so it carries the "Latest" badge and /releases/latest
	# resolves to it. Nothing here needs a version tag to be pushed.
	#
	# It builds alongside "check" rather than behind it; the release gate is on
	# "publish-latest". Queued behind the matrix these four targets waited on its
	# slowest leg, which on a push to main was most of the pipeline's wall clock.
	# What it costs is four builds thrown away on the pushes where main does fail,
	# which is the rarer case.
	build-latest:
	name: build (${{ matrix.name }})
	if: github.event_name == 'push' && github.ref == 'refs/heads/main'
	strategy:
	fail-fast: false
	matrix:
	include:
	- name: Linux x86_64
	os: ubuntu-24.04
	target: x86_64-unknown-linux-gnu
	- name: Linux arm64
	os: ubuntu-24.04-arm
	target: aarch64-unknown-linux-gnu
	- name: macOS arm64
	os: macos-15
	target: aarch64-apple-darwin
	- name: Windows x86_64
	os: windows-2025
	target: x86_64-pc-windows-msvc

	runs-on: ${{ matrix.os }}
	timeout-minutes: 20

	steps:
	- uses: actions/checkout@v7

	- name: Install Rust toolchain
	run: rustup toolchain install stable --profile minimal --target ${{ matrix.target }}

	# Keyed on the target, not the runner. rust-cache never overwrites an
	# existing entry, so a key shared with the check jobs on the same runner
	# would save whichever tree finished first and then never refresh, and the
	# release tree is not the one that wins that race.
	- name: Rust cache
	uses: Swatinem/rust-cache@v2
	with:
	prefix-key: "v1-rust"
	shared-key: "${{ matrix.target }}-release"

	- name: Set up Python
	uses: actions/setup-python@v7
	with:
	python-version: "3.x"

	# Runs the generator directly rather than through make, which is not
	# dependable on the Windows runner. Unlike the Makefile's $(S2T_STAMP)
	# recipe this skips rustfmt and the stamp: nothing here fmt-checks, and the
	# minimal toolchain has no rustfmt. "python" not "python3" is for Windows;
	# setup-python provides both elsewhere.
	- name: Build and package
	shell: bash
	run: \|
	bin=zhtw-mcp${{ runner.os == 'Windows' && '.exe' \|\| '' }}
	python scripts/gen-s2t-tables.py
	cargo build --locked --release --target ${{ matrix.target }}
	mkdir pkg
	cp "target/${{ matrix.target }}/release/$bin" LICENSE README.md pkg/

	# Prove the binary we are shipping actually starts. Every target here
	# builds on its own architecture, so every archive gets run once; the
	# check matrix is ubuntu + macos, which makes this the only place the
	# Windows and arm64 Linux builds execute at all. 0 is a clean verdict
	# and 1 is a prose gate; both mean it worked.
	printf '\xe6\xb8\xac\xe8\xa9\xa6\xe6\xaa\x94\xe6\xa1\x88\n' > smoke.txt
	rc=0; "pkg/$bin" lint smoke.txt \|\| rc=$?
	[ "$rc" -le 1 ] \|\| { echo "smoke test failed with exit $rc" >&2; exit 1; }

	tar czf "zhtw-mcp-${{ matrix.target }}.tar.gz" -C pkg "$bin" LICENSE README.md

	- uses: actions/upload-artifact@v7
	with:
	name: zhtw-mcp-${{ matrix.target }}
	path: zhtw-mcp-${{ matrix.target }}.tar.gz
	if-no-files-found: error
	retention-days: 1

	# "check" by name rather than by way of another job: "check-size" and
	# "build-latest" run beside it now, so this list is the whole of what stands
	# between a failing test suite and a published binary.
	publish-latest:
	needs: [check, audit, check-size, build-latest, package-extension]
	if: github.event_name == 'push' && github.ref == 'refs/heads/main'
	runs-on: ubuntu-24.04
	# Without this the job inherits the 360 minute default, and since main runs
	# do not cancel each other a wedged gh call would hold the concurrency group
	# (and every later push to main) for six hours.
	timeout-minutes: 10
	permissions:
	contents: write
	env:
	GH_TOKEN: ${{ github.token }}
	GH_REPO: ${{ github.repository }}
	steps:
	- uses: actions/download-artifact@v8
	with:
	pattern: zhtw-mcp-*
	path: dist
	merge-multiple: true

	# This never moves the "latest" git tag. A download URL keys on the
	# release's tag name, not on the commit the ref points at, so force-moving
	# the ref on every build bought nothing. "gh release create" does create
	# the tag when it is missing, and --target applies only in that case; if the
	# tag already exists the release adopts wherever it points, which is why the
	# commit a build came from is recorded in the notes instead.
	#
	# Nothing here deletes the release either. The README hands out these URLs,
	# and a delete-then-create cycle 404s them for as long as the upload takes:
	# the first run of this workflow left a five and a half minute hole.
	- name: Publish the latest release
	shell: bash
	run: \|
	cd dist
	sha256sum -- zhtw-mcp-* > SHA256SUMS

	# The queue already runs pushes in order, so what this actually catches
	# is a manual re-run of an older run, plus main moving while this run is
	# in flight. Best effort: main can advance again right after the check.
	tip=$(gh api "repos/$GH_REPO/commits/main" --jq .sha)
	if [ "$tip" != "$GITHUB_SHA" ]; then
	echo "::warning::latest not refreshed: main moved to ${tip:0:7}, this run built ${GITHUB_SHA:0:7}"
	exit 0
	fi

	# Notes name a commit, so they go up after the archives built from that
	# commit. --clobber deletes every matching asset and re-uploads them
	# concurrently, and gh warns that a failed upload loses the originals,
	# so the whole set is briefly absent and can stay that way. Hence the
	# retry, and the verification below. --draft=false also recovers a
	# release left as a draft by a "create" that died mid-upload.
	notes="Automated build of $GITHUB_SHA on $(date -u +%Y-%m-%d)."
	upload_rc=0
	if gh release view latest >/dev/null 2>&1; then
	gh release upload latest ./* --clobber \|\|
	gh release upload latest ./* --clobber \|\| upload_rc=$?
	# Notes name a commit, so they go up only once the archives from that
	# commit are in place. Letting a failed upload through to here would
	# leave the notes advertising binaries that are not on the release.
	# --prerelease=false is spelled out rather than left off: the flag is
	# sticky, and the release was published as a prerelease originally.
	if [ "$upload_rc" -eq 0 ]; then
	gh release edit latest --notes "$notes" \
	--prerelease=false --latest --draft=false
	fi
	else
	gh release create latest ./* --target "$GITHUB_SHA" \
	--title latest --notes "$notes" --latest
	fi

	# Trust the published set, not the exit status. Scratch files go to
	# RUNNER_TEMP so they cannot land in the glob being compared. Only a
	# missing asset is fatal: an extra one is someone else's upload or a
	# leftover from a renamed target, and failing on that would wedge every
	# later run.
	printf '%s\n' * \| sort > "$RUNNER_TEMP/want"
	gh release view latest --json assets --jq '.assets[].name' \
	\| sort > "$RUNNER_TEMP/got"
	missing=$(comm -23 "$RUNNER_TEMP/want" "$RUNNER_TEMP/got")
	extra=$(comm -13 "$RUNNER_TEMP/want" "$RUNNER_TEMP/got")
	[ -z "$extra" ] \|\| echo "::warning::latest carries assets this build did not produce: $(printf %s "$extra" \| tr '\n' ' ')"
	[ -z "$missing" ] \|\| {
	echo "::error::latest is missing assets after publish: $(printf %s "$missing" \| tr '\n' ' ')"
	exit 1
	}
	[ "$upload_rc" -eq 0 ] \|\|
	{ echo "::error::asset upload failed even after a retry"; exit 1; }

Navigation Menu

Uh oh!

Warn or remove repeated symbols in editable fields #459

Workflow file

Warn or remove repeated symbols in editable fields #459

Uh oh!

Workflow file for this run