Remove "hexadecimal" restriction

[yurishkuro]

This has been discussed in the original PR (starting from #1 (comment)) but got lost/ignored. In the current form the spec excludes tracing systems that may be using differently formatted strings for trace and/or span IDs.

[bogdandrutu]

Couple of things:

• I would say base16 not "hexadecimal". "hexadecimal" means you encode a number.
• You need to respect HTTP restrictions so you have to encode the "string" or "byte array" or however you call it. So we have to define what is that encoding because otherwise you put a restriction on the traceId to be HTTP compatible.

[yurishkuro]

because otherwise you put a restriction on the traceId to be HTTP compatible.

well, if we're talking about HTTP header spec, isn't that implied?

I would say base16 not "hexadecimal". "hexadecimal" means you encode a number.

https://www.google.com/search?q=define+hexadecimal says "relating to or using a system of numerical notation that has 16 rather than 10 as its base."

The point is, the current spec exclude tracing systems that might choose a different encoding or length of the trace ID. Adrian had an argument about storage implication, which is valid in the sense that yes there are implications, but it's not impossible to resolve. In the best case the two systems use "compatible" trace-ids and can use them directly. In the worse case the receiving instrumentation will record the incoming trace-id as a correlation id.

[codefromthecrypt]

Hi, Yuri. What format does jaeger require that is not hexidecimal? What would happen if it were not? How would you handle it in jaeger? It is important we dont add technical debt in the process of doing this work. Every constraint removed comes at a cost. It would be nice to know you thought this through, especially as it can help others think this through.

[wu-sheng]

+1 about removing the base16 and length things. Skywalking is using three longs for ID(trace segment id and trace id). So length changes for every generated id. If we follow the spec, we must cost more CPU and memory for the ID. This is not good.

Right now, skywalking can only consider to add a new HEAD for this spec if some people want this interop feature. Put two heads, one for skywalking, one for TraceContext Spec. It can work, but it is the best solution. :)

But still, I hope the spec can make the skywalking supporting easier.

[wu-sheng]

@adriancole Right now, skywalking is considering put the ids(traceId and spanId) from outer tracer into tags of the EntrySpan.

[codefromthecrypt]

+1 about removing the base16 and length things. Skywalking is using three longs for ID(trace segment id and trace id). So length changes for every generated id. If we follow the spec, we must cost more CPU and memory for the ID. This is not good.

Hi, sheng. do you need to remove both constraints or just one (length)? On the technical side, I think there's been some benchmarking around encoding, which only affects you as you exit the process (this spec has no concern about in-memory structure). For example, hexadecimal is quite fast to encode, well under microsecond for the entire context, maybe faster than a formatted Date header :)

[codefromthecrypt]

the other thing about encoding, if for memory or perf concerns, I suspect we will have a binary format defined at some point. For example, there's already one under experiment in census <https://github.com/census-instrumentation/opencensus-specs/blob/master/encodings/BinaryEncoding.md>, which is laid on http/2 transport in base64 (platform-specific grpc encoding, but worth mentioning)

[yurishkuro]

Hi Adrian,

What format does jaeger require that is not hexidecimal? What would happen if it were not? How would you handle it in jaeger?

I am not saying that general strings are required or even supported by Jaeger. Jaeger is using 128bit byte array as a trace ID, and cannot support arbitrary string IDs without substantial changes to the backend and client libs. The way I would handle incoming string ID is by recording it as a correlation ID tag.

But it's not about Jaeger, it's about inclusiveness of the spec. Maybe we should start with the goals of this spec, what exactly it is trying to achieve. There are different possible interop modes.

[codefromthecrypt]

correlation by saving a string to a database allows offline correlation, but breaks propagation interop, which is the primary goal of the project. I think there certainly is space to define what we are defining by "context" when we say this, but we have defined it! So, my suggestion is that when we invalidate things, we provide for how to retain the charter while doing this. If the charter is passing trace context, and we accept incompatible data, we can't punt the responsibility to keep interop working in some fashion. IOTW hope is not a specification! For example, a lot of this tension is about a single header. We can in fact define means by which you can pass incompatible format while still continuing the trace. Otherwise, the spec grows to a correlation algorithm, right? A trace context header is used to pass trace context information across systems for a HTTP request. Our goal is to share this with the community so that various tracing and diagnostics products can operate together, and so that services can pass context through them, even if they're not being traced (useful for load balancers, etc.)

[yurishkuro]

can we define what the charter is? it's not stated anywhere in the repository, hence the confusion.

[codefromthecrypt]

The goal is mentioned here, we can top-level it, good point https://github.com/TraceContext/tracecontext-spec/blob/master/HTTP_HEADER_FORMAT.md#trace-context-http-header-format

[yurishkuro]

Let's discuss the charter in #17

[codefromthecrypt]

copied missing data in #18

[wu-sheng]

Hi, sheng. do you need to remove both constraints or just one (length)?

Skywalking id is long1.long2.long3. The dot is not part of base16 and long is not the certain length literally(After encoding, sure, it can be). So if the both constraints removed, skywalking will be no need to add new head, just need to separate the current sw3-head into two parts. This is my best choice.

And as you known, skywalking has a way to resolve this: support this by using a new head for interop and do not change the sw3-head. In that way, cost of encoding and size of http package will change, right? Maybe not big, but for a very high throughputs app, like some 10K tps application in Chinese e-commerce and telecom systems (large population -> large system...), everything changed, and I have to treat all perf related risks very carefully. Btw, these users have such high throughputs, but can't accept sampling... So you can image what is the situation skywalking facing......