Pull traces from an application

[Pragmateek]

This is a follow-up to this discussion : Pull traces from the application

TL;DR I'd like to get my traces into Jaeger but I can't push to it from my applications as they run in isolated environments.
So the only way to get data from the applications is in response to HTTP requests.
But AFAIK Jaeger (contrary to Prometheus) only work in push mode and does not support pull.

What are my options?

Any input is welcome. 🙂

[dosubot[bot]]

Hi @Pragmateek! I'm Dosu and I'm helping the Jaeger team.

You're correct that Jaeger is entirely push-based — there is no pull/scrape mechanism for traces analogous to how Prometheus scrapes metrics [1]. All of Jaeger's receivers (OTLP, Kafka, Jaeger-native, Zipkin) wait for trace data to be sent to them [2].

Given your constraint (applications only respond to inbound HTTP requests), here are some options to consider:

1. Custom exporter triggered by HTTP request — You could build a lightweight HTTP endpoint in your app that, when called, flushes buffered spans to an OpenTelemetry Collector or directly to Jaeger via OTLP. An external scheduler or orchestrator would periodically hit that endpoint to trigger the export.

2. File-based export + collector pickup — Your application could write spans to a local file (e.g., using the OTLP JSON file exporter), and a separately deployed OpenTelemetry Collector with a filelog receiver or a custom script could pick up those files and push them to Jaeger.

3. Kafka as an intermediary — If you can get a Kafka broker accessible from the isolated environment, your app could produce spans to a Kafka topic, and Jaeger's Kafka receiver would consume them on the other side [3].

4. Reverse proxy / tunnel — If the isolation is network-level (no outbound), but inbound is allowed, you could set up a reverse tunnel or proxy that allows the app to push to a local endpoint that forwards externally.

None of these are "native pull mode" in Jaeger, but option 1 (an HTTP-triggered flush) comes closest to the pull semantics you're describing — you'd just need something external to initiate the request.

Would any of these patterns work within your environment's constraints?

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.

[donaldraph]

hi Pragmateek,
Jaeger's push-only end to end — none of the receivers scrape anything, they just wait for data. So pull is something you have to build, not something Jaeger gives you.
Two ways to fit your "HTTP response is my only egress" constraint: buffer-and-drain, where the app keeps an in-memory exporter and exposes GET /traces that serializes whatever's buffered to OTLP/JSON and clears it, then something outside the isolation polls that endpoint and re-exports to a Collector. The app never opens an outbound connection, something external reaches in. Bound the buffer and make the drain idempotent or a failed poll will double-export.
If you have got a shared volume instead of sockets, skip the live path: write spans via the Collector's fileexporter (JSON-lines), ship the file, ingest with otlpjsonfilereceiver → Jaeger exporter. Zero network needed.
If "isolated" just means no outbound but inbound's fine, a reverse tunnel into a local Collector is way less work and gets you back to normal push.

[Pragmateek]

Hi Donald,
Thanks for your answer.
I have a shared volume, but we can't access it directly from our dev machine, we have to jump on another server via RDP.
But as a way to validate the global setup and workflow and if viable invest more in an automated near real time solution, I could indeed export in JSONL, and manually copy the file back to my dev machine.
Could these files be easily ingested into Jaeger?

[VectorPeak]

Jaeger does not have an ingestion API that will poll applications, and the Jaeger query/storage APIs are not suitable as an ingestion path. The closest “Jaeger-native” answer is still: get the data into an OTLP/Jaeger receiver somehow, usually via an OpenTelemetry Collector.

For an environment where the app can only return data as an HTTP response, the cleanest pattern is to put a small bridge outside the isolated environment:

instrumented app
 -> custom SpanExporter buffers completed spans locally
 -> /traces/drain endpoint returns a batch and marks/acks it
 -> external poller calls /traces/drain
 -> poller exports OTLP to OpenTelemetry Collector
 -> Collector exports to Jaeger

That bridge/poller would be custom code; neither Jaeger nor the Collector currently provide a generic “scrape traces from arbitrary HTTP endpoints” receiver. If you go this route, pay attention to ack semantics, max buffer size, retries, duplicate spans, auth on the drain endpoint, and what happens when the poller is down for longer than the app can buffer.

[Pragmateek]

Thanks for your answer.
This is exactly the path I'm going down.
Hopefully I'll have some time to implement it in the coming weeks.