fix(memory): use plain OS threads for postgres operations to avoid nested runtime panic#4129
Merged
theonlyhennygod merged 1 commit intomasterfrom Mar 21, 2026
Merged
Conversation
…sted runtime panic Replace `tokio::task::spawn_blocking()` with plain `std::thread::Builder` OS threads in all PostgresMemory trait methods. The sync `postgres` crate (v0.19.x) internally calls `Runtime::block_on()`, which panics when called from Tokio's blocking pool threads in recent Tokio versions. Plain OS threads have no runtime context, so the nested `block_on` succeeds. This matches the pattern already used in `PostgresMemory::initialize_client()`, which correctly used `std::thread::Builder` and never exhibited this bug. A new `run_on_os_thread` helper centralizes the pattern: spawn an OS thread, run the closure, and bridge the result back via a `tokio::sync::oneshot` channel. Fixes #4101
1 task
glamberson
pushed a commit
to lamco-admin/zeroclaw
that referenced
this pull request
Mar 24, 2026
…deadpool Eliminates the run_on_os_thread workaround that spawned a fresh OS thread per database operation. The sync `postgres` crate internally calls Runtime::block_on(), which panics inside tokio. This was the root cause of issues zeroclaw-labs#961, zeroclaw-labs#1055, zeroclaw-labs#4101 and the motivation for the thread-per-query pattern added in zeroclaw-labs#4129. Changes: - Replace `postgres 0.19` with `tokio-postgres 0.7` (native async) - Add `deadpool-postgres 0.14` for connection pooling (default: 4) - Add `tokio-postgres-rustls 0.13` for TLS support - Lazy initialization via OnceCell (matches Qdrant backend pattern) - Add namespace, importance, superseded_by columns (parity with SQLite) - Idempotent schema migration via ADD COLUMN IF NOT EXISTS - Native store_with_metadata() and recall_namespaced() implementations - Filter superseded_by IS NULL in recall/list (matching SQLite) - Add pool_size and tls_mode to StorageProviderConfig Implements Step 2 from upgrade-in-place plan (zeroclaw-labs#4028).
glamberson
pushed a commit
to lamco-admin/zeroclaw
that referenced
this pull request
Mar 24, 2026
…deadpool Eliminates the run_on_os_thread workaround that spawned a fresh OS thread per database operation. The sync `postgres` crate internally calls Runtime::block_on(), which panics inside tokio. This was the root cause of issues zeroclaw-labs#961, zeroclaw-labs#1055, zeroclaw-labs#4101 and the motivation for the thread-per-query pattern added in zeroclaw-labs#4129. Changes: - Replace `postgres 0.19` with `tokio-postgres 0.7` (native async) - Add `deadpool-postgres 0.14` for connection pooling (default: 4) - Add `tokio-postgres-rustls 0.13` for TLS support - Lazy initialization via OnceCell (matches Qdrant backend pattern) - Add namespace, importance, superseded_by columns (parity with SQLite) - Idempotent schema migration via ADD COLUMN IF NOT EXISTS - Native store_with_metadata() and recall_namespaced() implementations - Filter superseded_by IS NULL in recall/list (matching SQLite) - Add pool_size and tls_mode to StorageProviderConfig Implements Step 2 from upgrade-in-place plan (zeroclaw-labs#4028).
webhive
pushed a commit
to webhive/zeroclaw
that referenced
this pull request
Mar 24, 2026
…sted runtime panic (zeroclaw-labs#4129) Replace `tokio::task::spawn_blocking()` with plain `std::thread::Builder` OS threads in all PostgresMemory trait methods. The sync `postgres` crate (v0.19.x) internally calls `Runtime::block_on()`, which panics when called from Tokio's blocking pool threads in recent Tokio versions. Plain OS threads have no runtime context, so the nested `block_on` succeeds. This matches the pattern already used in `PostgresMemory::initialize_client()`, which correctly used `std::thread::Builder` and never exhibited this bug. A new `run_on_os_thread` helper centralizes the pattern: spawn an OS thread, run the closure, and bridge the result back via a `tokio::sync::oneshot` channel. Fixes zeroclaw-labs#4101
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tokio::task::spawn_blocking()with plainstd::thread::BuilderOS threads in allPostgresMemorytrait method implementations (store,recall,get,list,forget,count,health_check)run_on_os_threadhelper that spawns an OS thread and bridges the result back viatokio::sync::oneshotchannelpostgrescrate internally callsRuntime::block_on(), which panics on Tokio blocking pool threads in recent Tokio versions; plain OS threads have no runtime context, avoiding the conflictMotivation
Fixes #4101 -- the daemon panics on the first incoming request when using the
memory-postgresbackend with: "Cannot start a runtime from within a runtime."The initialization in
PostgresMemory::new()already correctly usedstd::thread::Builder(plain OS threads), which is why it worked. But all subsequent operations (store,recall, etc.) usedtokio::task::spawn_blocking, whose threads are still associated with the Tokio runtime's blocking pool, triggering the panic.Risk
Medium -- changes runtime threading for postgres memory operations only. No behavioral changes to query logic or data handling. The pattern matches the existing
initialize_client()approach that was already proven to work.Test plan
valid_identifiers_pass_validation,invalid_identifiers_are_rejected,parse_category_maps_known_and_custom_values,new_does_not_panic_inside_tokio_runtime)memory-postgresbackend