Skip to content

bun:sqlite: close() finalizes outstanding prepared statements (drizzle close(true) "database is locked") - #35604

Closed
robobun wants to merge 4 commits into
mainfrom
farm/f0fe7ff6/sqlite-close-finalize-prepared
Closed

robobun wants to merge 4 commits into
mainfrom
farm/f0fe7ff6/sqlite-close-finalize-prepared

Conversation

@robobun

@robobun robobun commented Jul 25, 2026 •

Copy link
Copy Markdown
Collaborator

Repro

import { Database } from "bun:sqlite";

const db = new Database(":memory:");
db.exec("CREATE TABLE t (a INTEGER)");

// What drizzle-orm/bun-sqlite's session does for every query:
(() => { db.prepare("INSERT INTO t VALUES (?)").run(1); })();
(() => { db.prepare("SELECT * FROM t").all(); })();

db.close(true); // throws: database is locked

The statement wrappers fell out of scope but have not been garbage collected, so their sqlite3_stmt* is still live when sqlite3_close() runs and it reports SQLITE_BUSY. The original report (#11418) hit this via drizzle-orm/bun-sqlite/migrator, which prepares a statement per migration query.

Cause

Database#close only finalized the query cache (db.query() results) and the db.transaction() controller statements (#27202). Statements returned from db.prepare() are not tracked anywhere on the JS side, so close(true) routed straight to sqlite3_close() with unfinalized statements outstanding.

Fix

VersionSqlite3 now records every JSSQLStatement prepared against it. close() (both the sqlite3_close and sqlite3_close_v2 paths) walks that set and finalizes each statement before closing the connection, which is what better-sqlite3's Database#close does. Using a statement after its database was closed throws Statement has finalized, the same error stmt.finalize() already produced.

The garbage-collection path is unchanged: a database whose JS object is collected while statements are still reachable keeps the connection open through VersionSqlite3's refcount until the last statement is finalized (can continue to use existing statements after database has been GC'd still covers this).

This supersedes the behavior asserted by the previous close(true) should throw an error if the database is in use and should dispose AND throw an error if the database is in use tests; those now assert the statement is finalized rather than that close throws.

Verification

$ USE_SYSTEM_BUN=1 bun test test/js/bun/sqlite/sqlite.test.js -t "drizzle-orm pattern"
(fail) close(true) works after db.prepare()'d statements were dropped without finalize (drizzle-orm pattern)
  Error message: "database is locked"

$ bun bd test test/js/bun/sqlite/sqlite.test.js
 99 pass
 0 fail

Fixes #11418

Related

#33307 adds the same statement tracking for close(false) but keeps close(true) throwing; this PR finalizes in both paths so the drizzle case works. #34122 routes [Symbol.dispose] to close(false), which this PR also makes unnecessary since close(true) no longer throws for live prepared statements.


no test proof · iteration 1 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/bun/sqlite/sqlite.test.js

db.prepare() returns a statement whose underlying sqlite3_stmt is only
released when the JS wrapper is finalized or garbage collected. A
library such as drizzle-orm that calls db.prepare() per query and drops
the reference leaves those statements alive until GC, so a following
db.close(true) failed with "database is locked" (sqlite3_close()
returning SQLITE_BUSY) even though every query had completed.

VersionSqlite3 now tracks the JSSQLStatement wrappers prepared against
it. close() walks that set and finalizes each statement before closing
the connection, which is what better-sqlite3's Database#close() does.
Using a statement after its database was closed throws
"Statement has finalized".

Fixes #11418
@coderabbitai

coderabbitai Bot commented Jul 25, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Changes

The SQLite binding now tracks live prepared statements per connection and finalizes them when the database closes. Cleanup paths deregister statements and prevent double finalization. Tests cover close behavior, dropped statements, disposal, and concurrent operations.

SQLite statement lifecycle

Layer / File(s) Summary
Statement tracking and close behavior
src/jsc/bindings/sqlite/JSSQLStatement.cpp
VersionSqlite3 tracks prepared statements, closes finalize outstanding statements, and cleanup paths deregister them safely.
SQLite close regression coverage
test/js/bun/sqlite/sqlite.test.js
Tests verify close() and close(true) finalize outstanding statements, handle dropped statements and disposal, and preserve fulfilled concurrent operations.

Suggested reviewers: jarred-sumner

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes address #11418 by finalizing outstanding prepared statements so sqlite.close(true) can complete after migrations.
Out of Scope Changes check ✅ Passed The diff stays focused on sqlite close semantics, statement tracking, and matching tests, with no clear unrelated changes.
Title check ✅ Passed The title clearly matches the main change: finalizing outstanding prepared statements on SQLite close.
Description check ✅ Passed The description covers the repro, cause, fix, and verification, though it doesn’t use the template’s exact headings.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/js/bun/sqlite/sqlite.test.js`:
- Around line 1569-1572: Replace the implementation-narrative comment near the
regression test with only the issue URL comment, preserving the existing test
name and test behavior.
- Around line 1566-1580: The test around db.prepare() and close(true) must prove
the dropped statement remains outstanding before closing. Retain a WeakRef to
one prepared statement, ensure its wrapper becomes unreachable as currently
intended, and assert the WeakRef dereferences immediately before db.close(true);
preserve the existing successful close assertion.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 318e10be-93b8-432f-bd6d-208cae4c1a57

📥 Commits

Reviewing files that changed from the base of the PR and between df6c7ee and 8b90166.

📒 Files selected for processing (2)
  • src/jsc/bindings/sqlite/JSSQLStatement.cpp
  • test/js/bun/sqlite/sqlite.test.js

Comment thread test/js/bun/sqlite/sqlite.test.js Resolved
Comment thread test/js/bun/sqlite/sqlite.test.js Resolved
@robobun

robobun commented Jul 25, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 6:19 AM PT - Jul 25th, 2026

⏳ @robobun, your commit 6f1cd91 is still building in Build #80799, but has 1 failures so far (All Failures):

@github-actions

Copy link
Copy Markdown
Contributor

This PR may be a duplicate of:

  1. bun:sqlite: close() finalizes outstanding statements instead of leaving them live #33307 - Nearly identical goal: makes close() finalize outstanding prepared statements instead of leaving them live, directly addressing the same "database is locked" problem and issue bun:sqlite not closable after running migrations #11418
  2. bun:sqlite: Database[Symbol.dispose] no longer throws over live prepared statements #34122 - Substantial overlap: prevents Database[Symbol.dispose] from throwing when prepared statements are live, addressing the same root cause from the using keyword angle

🤖 Generated with Claude Code

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond the inline findings, I also checked: (1) whether ~JSSQLStatement mutating version_db->statements during GC sweep can race with close() — it can't, JSDestructibleObject destructors run on the mutator thread (same reason the pre-existing non-atomic reference_count is safe); (2) the sqlite3_close() failure path after the swap — the orphaned wrappers have stmt == nullptr, their later removeFirst(this) is a harmless not-found no-op, and release() still balances; (3) the can continue to use existing statements after database has been GC'd test — unaffected because Database is a pure-JS class with no native finalizer, so GC of the wrapper never reaches the new finalize loop.

Extended reasoning...

This PR changes the user-facing contract of Database#close() (statements are now force-finalized rather than allowed to finish or triggering a throw), and the description names two competing open PRs (#33307, #34122) that take different design positions on the same issue — so a human should sign off on the semantics. The inline docs/types nit already flags that sqlite.mdx and sqlite.d.ts still describe the old contract. The note above records the memory-safety concerns I examined and ruled out so they don't need to be re-derived.

@robobun
robobun requested a review from alii as a code owner July 25, 2026 10:32
Comment thread docs/runtime/sqlite.mdx Resolved

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No new issues — the latest push addressed the docs-Note nit and the comment-cop flags. Deferring to a maintainer because this changes close(false)'s user-visible contract (statements now throw instead of continuing via sqlite3_close_v2 zombie mode), and #33307 / #34122 take narrower approaches to the same #11418 symptom; that's an API-design call I shouldn't make.

What was reviewed:

  • VersionSqlite3::statements lifecycle — append in create, swap-and-finalize in close, O(1) swap-remove in finalize()/~JSSQLStatement; no dangling-pointer or double-finalize path found.
  • Post-close() GC of the tracked JSSQLStatements: stmt is nulled so the destructor's sqlite3_finalize is skipped, and untrackStatement on the now-empty vector is a no-op.
  • Updated #13082 test still guards the original segfault while asserting the new throw.
Extended reasoning...

Overview

Native change in src/jsc/bindings/sqlite/JSSQLStatement.cpp: VersionSqlite3 gains a non-owning Vector<JSSQLStatement*> populated in create() and depopulated via a swap-with-last helper in finalize() and ~JSSQLStatement(). jsSQLStatementCloseStatementFunction now swaps the vector out and sqlite3_finalizes every outstanding sqlite3_stmt* before calling sqlite3_close/sqlite3_close_v2. Docs (docs/runtime/sqlite.mdx) and types (packages/bun-types/sqlite.d.ts) are updated to describe the new contract, and test/js/bun/sqlite/sqlite.test.js gains/updates seven tests covering the drizzle repro, using+SuppressedError, WAL file release with a live statement, and the #13082 post-close throw.

All feedback from earlier runs is addressed as of 50da5a1: O(1) untrack (was removeFirst), docs/.d.ts rewritten to match the new semantics, the GC Note reworded to describe the refcount behavior instead of implying close() runs on GC, the WeakRef guard added to the drizzle-pattern test, and both multi-line C++ comments trimmed per comment-cop.

Security risks

None identified. No untrusted input parsing; the new state is a per-connection vector of raw pointers whose add/remove sites are all on the JS thread and gated on the same version_db non-null check, so there's no path that appends without a matching remove. sqlite3_finalize in the close loop doesn't re-enter JS.

Level of scrutiny

Medium-high. The native change is small and mechanically sound, but it flips a documented, tested behavior: close(false) used to defer to sqlite3_close_v2's zombie mode so live statements kept working (the previous #13082 test asserted stmt.all() succeeded after close()); now they throw Statement has finalized. That's the right call per better-sqlite3 parity and fixes real drizzle-orm breakage, but it can break user code that relied on the old zombie semantics, and two other open PRs solve #11418 differently (#33307 keeps close(true) throwing; #34122 only reroutes [Symbol.dispose]). Which contract ships is a maintainer decision.

Other factors

Test coverage is thorough for the new behavior (both close variants, using, WAL file release, dropped-without-finalize + WeakRef liveness proof, #13082 concurrent GC). The GC-keeps-connection-alive test (can continue to use existing statements after database has been GC'd) is unchanged and still passes, confirming the refcount path wasn't touched. CI build #80581 was queued for 53a886c; 50da5a1 is docs/comments-only on top of it.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond the inline nit, I traced the new VersionSqlite3::statements vector for dangling-pointer and refcount hazards: entries are appended in JSSQLStatement::create, removed in both jsSQLStatementFunctionFinalize and ~JSSQLStatement before the object is freed, and close() swaps the vector into a local before iterating so a re-entrant untrack can't invalidate the loop. The finalized statements keep their version_db ref until GC, and release() on the already-nulled db is a no-op, so the refcount stays balanced. All mutation is JS-thread-only. This is still a user-visible behavior change to close() with two competing open PRs (#33307, #34122), so leaving the approach call to a maintainer.

Extended reasoning...

The C++ lifetime tracking looks correct on inspection; the remaining question is whether finalize-on-close (both paths) is the API contract Bun wants, given #33307 keeps close(true) throwing and #34122 only changes [Symbol.dispose].

Comment thread test/js/bun/sqlite/sqlite.test.js Resolved
@robobun

robobun commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator Author

CI status

Build #80799: sqlite.test.js passed on all 39 test lanes that ran. Remaining failures are unrelated to this diff:

  • test/js/bun/util/filesystem_router.test.ts segfault on ubuntu aarch64 (bundler/resolver crash in reload() while Bun.build() resolves the same directory; this PR touches only the sqlite binding)
  • test-fastutf8stream-reopen.js flaky, marked [flaky]
  • build-bun timed out on 3 lanes (freebsd x64, plus cascaded 128 waiting_failed)

Earlier builds #80361 and #80622 were pure build timeouts (no test jobs ran). The diff itself is green where tests ran; needs a maintainer to look past the infra noise.

All review feedback has been addressed through 50da5a1.

Jarred-Sumner added a commit that referenced this pull request Aug 2, 2026
… query() cache LRU (#36793)

Follow-up to #36573, which made both `close()` modes force-finalize
every statement. That was a behavior change for `db.prepare()`
statements held across `close(false)` (reads/iterators used to keep
working via `sqlite3_close_v2`). This restores a coherent graceful mode
and gives `throwOnError` real meaning.

| after… | `close()` / `close(false)` | `close(true)` / `using` |
|---|---|---|
| `db.query()` statements (cached or evicted) | finalized | finalized |
| `db.prepare()` statements | **keep working** (reads *and* writes)
until finalized/GC'd | finalized |
| connection / file handle | released when last `prepare()` stmt is
finalized (`sqlite3_close_v2`) | released immediately (`sqlite3_close`),
throws on failure |
| using a finalized stmt | `Database has closed` | `Database has closed`
|
| `db.query()` / `db.prepare()` new | throws | throws |

Also:

| | before | after |
|---|---|---|
| `db.query()` cache | first 20 distinct strings cached forever;
everything after uncached and non-`PERSISTENT` | LRU of
`MAX_QUERY_CACHE_SIZE`; every `query()` stmt is `PERSISTENT` and tracked
as Database-owned so `close()` finalizes it even after eviction |
| `prepare()` stmt `.run()` after `close(false)` | `Database has closed`
(while `.all()` worked) | works (statement paths use
`sqlite3_db_handle(stmt)`) |
| post-close error | `Statement has finalized` | `Database has closed` |

#36572 / #35604 / #11418 stay fixed (`close(true)` never hits
`SQLITE_BUSY`; `close()` releases the file when only `query()`
statements are outstanding). Docs and types updated.

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bun:sqlite not closable after running migrations

2 participants