Skip to content

feat(#27,#36): observability floor — PGRST001/PGRST002 error logging + DB-pool telemetry#61

Merged
milmazz merged 2 commits into
mainfrom
observability-floor
Jul 8, 2026
Merged

feat(#27,#36): observability floor — PGRST001/PGRST002 error logging + DB-pool telemetry#61
milmazz merged 2 commits into
mainfrom
observability-floor

Conversation

@milmazz

@milmazz milmazz commented Jul 7, 2026

Copy link
Copy Markdown
Owner

What

The minimal observability floor for a first release: structured JSON error logging for PostgREST's PGRST001/PGRST002-class conditions, plus the [:bier, :pool, …] :telemetry event family. Logging/telemetry is additive and out-of-band — no response body, status, or header changes.

Structured error logging (#27)

  • New Bier.ErrorLogger emits PostgREST's error envelope {code, message, details, hint} as a single JSON line via Logger.error/2 (the issue's "via Logger" option), with :bier_instance / :bier_error_code metadata. The message is built lazily, so nothing is encoded when the :error level is disabled. The moduledoc documents how a host gets PostgREST's exact stderr behavior (config :logger, :default_handler, config: [type: :standard_error]).
  • PGRST002 ("Could not query the database for the schema cache. Retrying.") is wired inside Bier.SchemaCache.load!/3 — the one funnel both the boot introspection (Bier.HttpServerStarter) and every reload (listener or programmatic) go through — so each failing load logs exactly once before re-raising.
  • PGRST001 ("Database client error. Retrying the connection.") fires where pool errors surface in the request path: a DBConnection.ConnectionError reaching Bier.Plugs.FallbackController (new clause rendering the exact opaque 500 the catch-all already produced), or a Postgrex.Error carrying no SQLSTATE (client error; response unchanged).

DB-pool telemetry (#36)

  • New Bier.PoolMonitor (child of each instance's supervisor) samples the Postgrex pool via DBConnection.get_connection_metrics/1 and emits the [:bier, :pool, :status] gauges — :max / :available / :waiting, mirroring pgrst_db_pool_max|available|waiting. One sample at startup, then every 5s; an unreachable pool skips the sample instead of crash-looping alongside it.
  • [:bier, :pool, :checkout_timeout] (mirrors pgrst_db_pool_timeouts_total) fires from the fallback controller when a request's checkout was dropped from the pool queue (reason: :queue_timeout).
  • Both events carry :instance metadata, consistent with [observability] Emit :telemetry events (request / pool / schema-cache) #26, and are fully documented in the Bier.Telemetry moduledoc.
  • db_connection ~> 2.7 is now an explicit dep (already in the lock at 2.10.1): get_connection_metrics/1 is called directly and was added in 2.7.

The [:bier, :jwt_cache, …] family from #36 is not included: as the issue notes, it first requires introducing a JWT verification cache (Bier verifies every JWT directly today) — that remains open.

Verification

  • mix format --check-formatted, mix compile --warnings-as-errors (dev + test env), mix credo --strict, mix docs --warnings-as-errors, mix deps.unlock --check-unused, mix hex.audit — all green locally.
  • New unit tests under test/bier/ (CI runs them): error_logger_test.exs (capture_log envelope shape + the load! PGRST002 funnel via an unreachable pool), plugs/fallback_controller_test.exs (PGRST001 logging, unchanged 500 wire format, checkout-timeout counter), pool_monitor_test.exs (:status gauges on boot; unreachable-pool sample skip).
  • DB-free smoke run exercising every new path end-to-end (envelopes, counter, unchanged response, monitor survival) — output verified.
  • Conformance suite untouched (test/**/spec/** unchanged); CI is the judge.

Closes #27. Part of #36 (pool events delivered; jwt_cache events remain, blocked on a JWT verification cache).

🤖 Generated with Claude Code

milmazz and others added 2 commits July 6, 2026 23:05
… counter

Add Bier.PoolMonitor, a per-instance poller that samples the Postgrex pool
through DBConnection.get_connection_metrics/1 (public since db_connection
2.7, now declared as an explicit dependency floor) and emits the
[:bier, :pool, :status] gauge event — :max / :available / :waiting, mirroring
PostgREST's pgrst_db_pool_max|available|waiting Prometheus gauges. One sample
fires at startup, then every 5s; an unreachable pool skips the sample instead
of crash-looping alongside it.

The counter half (pgrst_db_pool_timeouts_total) is event-driven:
Bier.Telemetry.pool_checkout_timeout/1 emits
[:bier, :pool, :checkout_timeout], wired from the fallback controller in the
follow-up commit. Both events carry the instance name in metadata, consistent
with the #26 events, and are documented in the Bier.Telemetry moduledoc.

The jwt_cache family from #36 remains open: it first requires a JWT
verification cache, which Bier does not have yet.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PostgREST logs database-client and schema-cache failures to stderr as a JSON
object with the error-envelope keys {code, message, details, hint}. Bier now
emits the same envelope through Logger.error/2 (Bier.ErrorLogger):

- PGRST002 — a failing schema-cache introspection. Wired inside
  Bier.SchemaCache.load!/3, the single funnel both the boot introspection
  (Bier.HttpServerStarter) and every reload go through, so each failing load
  logs exactly once before re-raising.
- PGRST001 — a database client/connection error surfacing in the request
  path: a DBConnection.ConnectionError reaching the fallback controller (new
  clause rendering the exact same opaque 500 the catch-all already produced),
  or a Postgrex.Error with no SQLSTATE. A :queue_timeout connection error
  additionally bumps [:bier, :pool, :checkout_timeout].

The message is built lazily (nothing is encoded when :error is disabled) and
carries :bier_instance / :bier_error_code metadata; the moduledoc shows how a
host routes the default handler to stderr for exact PostgREST parity. No
response body, status, or header changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@milmazz
milmazz merged commit 2a3643d into main Jul 8, 2026
3 checks passed
@milmazz
milmazz deleted the observability-floor branch July 8, 2026 06:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[observability] Structured JSON error logging to stderr (PGRST001/PGRST002)

1 participant