Skip to content

Add message broker / emulator fixtures: Kafka (Redpanda), LocalStack, Google Pub/Sub emulator, RabbitMQ #150

Description

@cofin

Summary

Add Docker service fixtures for message brokers and cloud emulators, following the same *_service + Docker-autoskip pattern as the existing fixtures (the Spanner emulator fixture is the closest precedent):

  1. Kafkakafka_service exposing bootstrap servers. Candidate images: redpandadata/redpanda in dev-container mode (Kafka API-compatible, ~2x faster startup, no ZooKeeper/KRaft ceremony) or apache/kafka-native (KRaft, GraalVM native image). Could also ship both (kafka_service / redpanda_service).
  2. LocalStacklocalstack_service exposing endpoint_url, region, and test credentials, so downstream tests can exercise SQS (visibility timeouts, DelaySeconds, FIFO) and other AWS APIs against localstack/localstack.
  3. Google Pub/Sub emulatorpubsub_service mirroring the Spanner emulator fixture shape (project, host/port, PUBSUB_EMULATOR_HOST-style wiring), e.g. gcr.io/google.com/cloudsdktool/google-cloud-cli:emulators running gcloud beta emulators pubsub start.
  4. RabbitMQrabbitmq_service exposing host/port/user/password (and optionally the management API port), backed by the official rabbitmq:4-management image. No emulator needed — it's the real broker; RabbitMQ 4.3+ is preferable so downstream tests can cover quorum-queue strict priority and native delayed messages.

Motivation

litestar-queues is researching broker-style integrations (Kafka / Google Pub/Sub / Amazon SQS / RabbitMQ). Its integration matrix consumes pytest-databases *_service fixtures with importorskip + Docker autoskip gating, and the project policy is to add missing container fixtures upstream here rather than hand-roll session-scoped fixtures downstream. These fixtures would also be broadly useful beyond that use case.

Happy to split this into one issue per fixture if preferred.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions