Issue
Today, OBI's traces are completely auto instrumented, if a user wants to define manual spans he either needs to
- Ditch OBI and go for full auto instrumentation + manual spans
- Run both OBI and a manual instrumentation where the service exports only the manual traces
The second is problematic as well, it means 2 proceses are exporting the telemetry and not only OBI, OBI by default shuts down instrumentation of a service if it sees its already sends telemetry, this needs to be manually disabled.
Another issue is the trace context, the manual spans are not aware of OBI created context, so on best case where both OBI and the manual instrumentation succesfully parse the same w3c context from an incoming requests, the spans will be part of the same trace, but not in the correct hierarchy if server > internal manual span.
worst case is that both OBI and the manual instrumentation create their own trace context and each one ends up in a different trace with no correlation to each other.
This is the situation today, with the exception of golang, where OBI does support for manual instrumentations using uprobes, the application only does the instrumentation, and does not have any trace pipeline / exporter / context manager. OBI manages it for completely, trace context and ordering is preserved.
Goal
Expends OBIs capability to support manual instrumentation in more runtimes / languages.
Proposed solutions
for each language / runtime the implementation might be different, but as an interface goes we should strive to support the normal way users instrument their code manually, using the otel SDK in their specific language.
Some languages might allow us to use a similar solution to the go instrumentation with uprobes / usdts, but some might require us to inject some instrumentation code inside the runtime (OBI already has this type of handling for nodejs context propagation / OBI Java agent, so this is not unheard of)
The spans need to be part of the same OBI trace under the correct hierarchy
Alternatives considered
There is the RFC for custom manual spans from OBI config: #2552
This is a great solution and might be an extension to this issue, but:
- it lacks compatibility with Otel and creates our own standard for manual instrumentation
- very limited in runtime versions / distros which it supports
Issue
Today, OBI's traces are completely auto instrumented, if a user wants to define manual spans he either needs to
The second is problematic as well, it means 2 proceses are exporting the telemetry and not only OBI, OBI by default shuts down instrumentation of a service if it sees its already sends telemetry, this needs to be manually disabled.
Another issue is the trace context, the manual spans are not aware of OBI created context, so on best case where both OBI and the manual instrumentation succesfully parse the same w3c context from an incoming requests, the spans will be part of the same trace, but not in the correct hierarchy if server > internal manual span.
worst case is that both OBI and the manual instrumentation create their own trace context and each one ends up in a different trace with no correlation to each other.
This is the situation today, with the exception of golang, where OBI does support for manual instrumentations using uprobes, the application only does the instrumentation, and does not have any trace pipeline / exporter / context manager. OBI manages it for completely, trace context and ordering is preserved.
Goal
Expends OBIs capability to support manual instrumentation in more runtimes / languages.
Proposed solutions
for each language / runtime the implementation might be different, but as an interface goes we should strive to support the normal way users instrument their code manually, using the otel SDK in their specific language.
Some languages might allow us to use a similar solution to the go instrumentation with uprobes / usdts, but some might require us to inject some instrumentation code inside the runtime (OBI already has this type of handling for nodejs context propagation / OBI Java agent, so this is not unheard of)
The spans need to be part of the same OBI trace under the correct hierarchy
Alternatives considered
There is the RFC for custom manual spans from OBI config: #2552
This is a great solution and might be an extension to this issue, but: