Skip to content

[chart] Add gateway support for Alloy #5157

@pkarakal-n2g

Description

@pkarakal-n2g

Component(s)

No response

Request

Add an optional gateway deployment, like NGINX, to the Alloy Helm chart, following the same pattern used in the LGTM charts. This would provide a unified HTTP(S) entry point with path-based routing to Alloy's multiple protocol endpoints.

This component should have support for configuring the port, the different protocols/upstreams (eg otlp, otlp http, loki, prometheus etc) and then turn on or off the different nginx configuration for those. It should also have support for authentication (if someone doesn't want to use the different auth methods of Alloy and wants to ensure that Alloy only handles the telemetry data and doesn't concern itself with server authentication).

Use case

I am currently operating a centralized Alloy listens on multiple ports for different telemetry protocols (OTLP on 4317/4318, Loki on 3100, etc.). In production deployments, this creates several challenges:

  • Multiple endpoints to manage: Clients need to know and configure different endpoints for each protocol
  • TLS termination: Need to configure TLS certificates for each port/service separately
  • Authentication: When using the recently added multi-user basic auth feature, auth must be configured per protocol endpoint
  • Firewall/security rules: Opening multiple ports increases attack surface and complexity

Current workaround: Use Ingress annotations for path-based routing
Here we can create a separate rule based on paths and configure the conditions to route the traffic using ingress annotations.
An example for OTLP data using the AWS Load balancer controller we can do the following:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/conditions.alloy-grpc: |
      [{"field":"http-header","httpHeaderConfig":{"httpHeaderName": "Content-Type", "values":["application/grpc"]}}]
spec:
  rules:
  host: alloy.example.com
    http:
      paths:
      - backend:
          service:
            name: alloy-grpc
            port:
              name: nginx
        path: /
        pathType: Prefix
      - backend:
          service:
            name: alloy
            port:
              name: nginx
        path: /
        pathType: Prefix

Some of the cons the ingress approach has are the following:

  • Annotation-heavy configuration (not portable across controllers; we can use Gateway API instead, but not everyone has migrated off of Ingress)
  • Limited control over proxy behavior (buffering, timeouts, connection pooling)
  • Complex rewrite rules needed for some protocols
  • Harder to apply per-path authentication/rate limiting consistently

Some of the pros of adding nginx gateway solution has are that:

  • we have a single entrypoint without having to expose multiple ports
  • we have fine-grained control over many of the connections to Alloy
  • we could delegate server authentication to nginx instead of handling it within Alloy

A con of adding NGINX in front of Alloy is that we introduce another network hop (and yet another component to maintain) which can increase latency but given that this would be an optional component this should not be that big of problem.

Tip

React with 👍 if this issue is important to you.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions