b3 single header format

[codefromthecrypt]

Design moved here https://cwiki.apache.org/confluence/display/ZIPKIN/b3+single+header+format

In designing the Trace Context format, we made a section called tracestate which holds the authoritative propagation data.

This issue defines a value that could be used as a separate "b3" header, and would be the same value used in the w3c tracestate field. Specific to the w3c format, this holds data not in the "traceparent" format, such as parent ID and the debug flag. It would be a completely non-lossy way to allocate our current headers into one value.

In simplest terms it is a mapping:

b3={x-b3-traceid}-{x-b3-spanid}-{if x-b3-flags 'd' else x-b3-sampled}-{x-b3-parentspanid}, where the last two fields are optional.

For example, the following headers:

X-B3-TraceId: 80f198ee56343ba864fe8b2a57d3eff7
X-B3-ParentSpanId: 05e3ac9a4f6e3b90
X-B3-SpanId: e457b5a2e4d86bd1
X-B3-Sampled: 1

Become one header or state field. For example, if a header:

b3: 80f198ee56343ba864fe8b2a57d3eff7-05e3ac9a4f6e3b90-1-e457b5a2e4d86bd1

Or if using w3c trace context format

tracestate: b3=80f198ee56343ba864fe8b2a57d3eff7-05e3ac9a4f6e3b90-1-e457b5a2e4d86bd1

Here are some more examples:

A sampled root span would look like:
b3: 4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-1

A not yet sampled root span would look like:
b3: 4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7

A debug RPC child span would look like:
b3: 4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-d-5b4185666d50f68b

Like normal B3, it is valid to omit trace identifiers in order to only propagate a sampling decision. For example, the following are valid downstream hints:

• don't sample - b3: 0
• sampled - b3: 1
• debug - b3: d

NOTE this does not match the prefix of traceparent, so we must define ours independently and consider that the w3c may change in different ways. This is ok as the "tracestate" entries in w3c format are required to be treated opaque. In other words we can be different on purpose or by accident of drift in their spec.

On positional encoding vs nested key/values

Positional encoding is more space efficient and less complicated to parse vs key/value encoding. For example, the AWS format code in brave is complex due to splitting, dealing with white space etc. Positional is simple to parse and straight-forward to map. Rationale is same as w3c traceparent for the most part.

Different than new specs, we expect no additional fields. B3 is a very stable spec and we are not defining anything new except how to encode it. For this reason, positional should be fine.

On putting mandatory fields up front

The trace ID and span ID fields are the only mandatory fields. This would allow fixed-length parsing for those just correlating on these values. Usually parentid is not used for correlation, rather scraping. Moreover, this is easier for existing proxies who only create trace identifiers.

Ex: you can control trace identifiers without making a sampling decision like so:

# root trace and span IDs
b3: 80f198ee56343ba864fe8b2a57d3eff7-05e3ac9a4f6e3b90

On sampled before parent span ID

When name-values aren't used, it could be confusing which of the equal length fields are the parent. By placing the sampled flag in-between, we make this more clear. Also, it matches the prefix of the current traceparent encoding.

Encoding "not yet sampled"

Leaving out the single-character sampled field is how we encoded the "no decision" state. This matches the way we used to address this (by leaving out X-B3-Sampled).

Encoding debug

We encode the debug flag (previously X-B3-Flags: 1), as the letter 'd' in the same place as sampled ('1'). This is because debug is a boosted sampled signal. Most implementations record it out-of-band as Span.debug=true to ensure it reaches the collector tier.

# force trace on a root span
b3: 80f198ee56343ba864fe8b2a57d3eff7-05e3ac9a4f6e3b90-d

One alternative considered was adding another field just to hold debug (ex a trailing -1). Not only was this less intuitive, it made parsing harder especially as parentId is also optional. This was reverted in openzipkin/brave#773

W3C drift alert

While we should watch out for changes in the TraceContext spec, for example, if they add a "priority flag", we should keep our impl independent. B3 fields haven't changed in years and we can lock in something far safer knowing that.

Why also define as a separate header

We have had continual problems with b3 with technology like JMS. In addition to declaring this format for w3c Trace Context, we could use it right away as the header "b3". This would solve all the problems we have like JMS hating hyphens in names, and allow those who opt into it a consistent format for when they transition to w3c.

openzipkin/brave#584

In other words, in messaging propagation and even normal http, some libraries could choose to read the "b3" header for the exact same format instead of "X-B3-X"

Should we use flags instead of two fields for sampled and debug?

We could encode the three sampled states and debug as a single 8-bit field encoded as hex. If we used flags, an example sampled span would be:

b3: 80f198ee56343ba864fe8b2a57d3eff7-05e3ac9a4f6e3b90-3-e457b5a2e4d86bd1

Notice this is 3, not 1. That's because if using bit field we need to tell the difference between unsampled and no sampling decision. This could be confusing to people.

On flags, in java, we already encode sampled and debug state internally flags like this internally

  static final int FLAG_SAMPLED = 1 << 1;
  static final int FLAG_SAMPLED_SET = 1 << 2;
  static final int FLAG_DEBUG = 1 << 3;

  static Boolean sampled(int flags) {
    return (flags & FLAG_SAMPLED_SET) == FLAG_SAMPLED_SET
        ? (flags & FLAG_SAMPLED) == FLAG_SAMPLED
        : null;
  }

  static boolean debug(int flags) {
    return (flags & FLAG_DEBUG) == FLAG_DEBUG;
  }
}

This might be better off than having two fields, although it is less simple as people often make mistakes coding bit fields, and X-B3-Flags caused confusion many times here including #20.

What about finagle's flags?

If we used flags, we could also do it the same way as finagle does, except I think it would be confusing as the length they allocated (64 bits or 16 characters in hex) was never used in practice.

In practice we could use a single hex character to encode all the flags in our format (that supports 8 flags). Also, using Finagle's flag encoding could further confusion about the "X-B3-Flags" header, which in http encoding never has a value besides "1" (#20). At any rate, we can consider using the first 8 bits of their format as prior art regardless of if we use it.

  /*
   * The debug flag is used to ensure this the current trace passes
   * all of the sampling stages.
   */
  val Debug = 1L << 0 // 1

  /**
   * Reserved for future use to encode sampling behavior, currently
   * encoded explicitly in TraceId.sampled (Option[Boolean]).
   */
  val SamplingKnown = 1L << 1
  val Sampled = 1L << 2

[codefromthecrypt]

cc @openzipkin/core @openzipkin/instrumentation-owners for feedback

[codefromthecrypt]

rewrote the description with more examples

[codefromthecrypt]

Notes on implementation once we settle on things, assuming we decide to introduce a "b3" header in addition to the same value in tracestate format.

• If you receive a "b3" header, ignore any "X-B3-*" headers
• If you want to write in brown-field, write both "b3" and "X-B3-*" headers on the way out. In cases where you can't anyway (like JMS), just write "b3"

At least looking at brave, this would be easy because there is one codebase parsing headers. A quick check for a "b3" header would be easy to to do, and be contained in B3 code, which would be independent of any other propagation code anyway. Libraries who have hard-coding around b3 might have more impact. For this reason a tracking issue would need to occur, and similar to our other things such as 128-bit trace ID, we'd expect a long time before all libraries will support this.

[codefromthecrypt]

posted an email to alert those not watching this repo https://groups.google.com/forum/#!topic/zipkin-dev/EdZjqHXuXsg

[jcchavezs]

+1

[wu-sheng]

+1 definitely

I am positive and agree with putting the X-B3-TraceId, X-B3-ParentSpanId, X-B3-SpanId and X-B3-Sampled into a single fields. SkyWalking has the similar sw3 format

These will make much easier for me to start a SkyWalking new feature, which could generate a new ref in EntrySpan, maybe named OTHER-TRACER-REF, and system=Zipkin.

[basvanbeek]

+1

on the separate sampled, debug vs flags I'm indifferent. I see pros and cons on each

[jsw]

This is great.

Would this new header eventually be considered the preferred propagation format over the existing headers? Would it be reasonable and possible to just use this header in an ecosystem where all applications are using updated tracing libraries?

[codefromthecrypt]

glad you like it. this has a two-role purpose: 1. answering the single-field problem that's plagued us for years, most notably in JMS where hyphens don't work. "b3" or "B3" is far simpler to get working than multiplied by several headers and several case formats. 2. having a state definition for the eventual w3c format, when/if gets to that. implementation wise: I would code this up in brave for example, to show how it works.. basically look for "b3" first, then fallback to legacy headers. Have a choice of double-encode on the way out or choose one or the other. In a green field, yeah "b3" would make more sense once we have tracked it through a quorum of libraries.

[bplotnick]

+1

I'm for this, but I have two concerns:

1. I'd like to make sure that we have clarifying examples of when some fields can be missing and when they can't. e.g. "Optional fields MUST appear in the correct position. e.g. If x-b3-parentspanid is present, then x-b3-sampled must also be present"

2. By supporting a new header, we'll eventually be supporting three header formats instead of two: X-B3-*, b3, and trace-state/trace-parent. This imposes a slightly larger burden for brown fields. i don't think that it is much, but it's more than just frameworks. Proxies, load balancers, logging frameworks, etc. may need to handle them as well.

[codefromthecrypt]

valid points and good feedback. thanks!

…

On Mon, 30 Apr 2018, 20:31 Ben Plotnick, ***@***.***> wrote: +1 I'm for this, but I have two concerns: 1. I'd like to make sure that we have clarifying examples of when some fields can be missing and when they can't. e.g. "Optional fields MUST appear in the correct position. e.g. If x-b3-parentspanid is present, then x-b3-sampled must also be present" 2. By supporting a new header, we'll eventually be supporting three header formats instead of two: X-B3-*, b3, and trace-state/trace-parent. This imposes a slightly larger burden for brown fields. i don't think that it is much, but it's more than just frameworks. Proxies, load balancers, logging frameworks, etc. may need to handle them as well. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#21 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAD61_87LOUGrlKk0RLEKmxaSyJjgeyMks5tt1hmgaJpZM4TerHT> .

[nicmunroe]

This looks good, although I'm unsure about how to handle some corner case scenarios.

For example, what would I do if I want to specify "no decision" on sampling, but also want to specify parent ID?

Or your force trace on a root span example: 80f198ee56343ba864fe8b2a57d3eff7-05e3ac9a4f6e3b90-1-1. It appears that in this case the fourth slot is the debug flag, but in all other cases it's parent ID. Should it maybe be 80f198ee56343ba864fe8b2a57d3eff7-05e3ac9a4f6e3b90-1--1 (double dash - before the final 1) so that implementers can split on dash - and always look in the fourth slot for parent ID with the caveat that it might be empty since it's optional?

Maybe that's a potential solution for the other issue as well...to specify "no decision" sampling but include a parent ID: 80f198ee56343ba864fe8b2a57d3eff7-05e3ac9a4f6e3b90--e457b5a2e4d86bd1?

[nicmunroe]

I guess my concern is very close to @bplotnick's concern #1, but I think it would be a valid use case to want "no decision" sampling but still want to specify parent ID, and there are probably other mix & match scenarios of the optional fields that should be supported. I think the double-dash to omit an optional field while still allowing you to specify one of the later optional fields could work and wouldn't be too confusing.

I'm also happy if someone thinks of another solution that allows these use cases.