Repository navigation
feat(search): tag query lexer + parser → AST - #5
Conversation
The booru query language in `packages/search`, pure and DB-free (`Query → Lexer → Parser → AST`; the AST is the canonical representation everything downstream consumes). Optimizer + SQL builder land with the DB. - `tokenize()` → classified terms (tag | metatag) carrying `-`/`~` prefixes - AST node types (and/or/not/tag/metatag, CompareOp) - `parse()` → AST: implicit AND; `-term` negates; `~term ~term` form one OR group AND'd with the rest; metatag operators `>`,`>=`,`<`,`<=` and `a..b` ranges - 21 tests (lexer + parser), built test-first Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
Important Review skippedNo new commits to review since the last review. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughIntroduces a search query tokenizer, AST types, parser, tests, and public re-exports in ChangesSearch Query Pipeline
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/search/src/parser.ts`:
- Around line 48-57: The range detection in parseMetatag is too broad because
any raw value containing “..” is treated as a range before operator parsing,
which blocks valid literals and mixed comparison inputs. Update parseMetatag to
only classify true range syntax, using a stricter check around the value pattern
rather than a plain includes() test, and keep the existing operator handling for
OPERATORS and the default eq path so inputs like source:https://a..b and
score:>10..20 can parse correctly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: f3dce521-167b-4628-b5a4-6fee7c886d11
📒 Files selected for processing (6)
packages/search/src/ast.tspackages/search/src/index.tspackages/search/src/parser.tspackages/search/src/token.tspackages/search/test/parser.test.tspackages/search/test/token.test.ts
Operators are now matched before ranges, and a range requires the lo..hi pattern with no scheme/path chars — so source:https://a..b stays eq and score:>10..20 is a comparison, not a misparsed range. +2 tests. Addresses CodeRabbit review on PR #5.
|
@coderabbitai review |
✅ Action performedReview finished.
|
Complete the search pipeline started in #5: the parsed query AST compiles to a Drizzle WHERE and filters the assets listing. - core: add the Search Engine (`compileAssetSearch`) — walks the query AST into a Drizzle condition. Metatags map to asset columns (`rating:` enum equality; `width`/`height`/`size` numeric eq/gt/gte/lt/lte and `lo..hi` ranges, with malformed multi-bound ranges like `1..2..3` ignored, not silently truncated); tag terms compile to an `asset_tags` EXISTS subquery (valid SQL now, matching once tagging lands). and/or/not via Drizzle; unknown keys and invalid values are dropped, not errored. - db: `findMany`/`count` accept an optional prebuilt `where: SQL` so the page and its total share one filter (db never parses text). - core: `AssetService.list` accepts `query`; trims then compiles it once. - api: `GET /api/v1/assets?q=` (capped at 1024 chars) forwards to the service. - tests: integration coverage for rating/numeric/range/negation/tag and pagination metadata under a filter; API q-forwarding unit test. Test safety: the destructive integration suites (they TRUNCATE) read a dedicated, opt-in `TEST_DATABASE_URL` instead of the app's `DATABASE_URL`, so a misconfigured dev/prod env can never be wiped — unset → they skip. CI sets it to its ephemeral Postgres; `.env.example` documents it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Summary
The booru tag query language in
packages/search—Query → Lexer → Parser → AST. Pure and DB-free (no HTTP, no Drizzle); the AST is the canonical representation everything downstream consumes (per ARCHITECTURE.md's search rule). Built test-first.What's included
tokenize()(token.ts) — splits a query into classified terms (tag|metatag), each carrying its-(negate) /~(or) prefix; lowercases tag names + metatag keys; drops blanks / bare-/~.ast.ts) —and/or/not/tag/metatagdiscriminated union +CompareOp.parse()(parser.ts) — implicit AND of terms;-term→ NOT;~term ~term→ one OR group AND'd with the rest; metatag operators>,>=,<,<=anda..branges.Syntax (v1)
1girl solo(AND) ·-monochrome(NOT) ·~blue_eyes ~green_eyes(OR group) ·rating:safe,score:>10,width:100..200(metatags)Verification
bun run test✓ (43 total)bun run typecheck✓ ·bun run lint:boundaries✓ (search stays a leaf)Out of scope (lands with the DB)
The optimizer and SQL builder — they consume this AST.
Summary by CodeRabbit
New Features
~-based OR grouping, and metatag filters with comparison operators and range syntax.Tests