Skip to content

Commit aafc575

Browse files
author
Liudmila Molkova
committed
Provide more details on Correlaiton-Context overhead and misuse and on parent id logging
Commit migrated from dotnet/corefx@b5202ab
1 parent 0c65ac7 commit aafc575

File tree

2 files changed

+18
-17
lines changed

2 files changed

+18
-17
lines changed

src/libraries/System.Diagnostics.DiagnosticSource/src/FlatRequestId.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,16 @@ We strongly recommend every implementation to support [Hierarchical Request-Id](
55
* `Request-Id` uniquely identifies every HTTP request involved in operation processing and MUST be generated for every incoming and outgoing request
66
* `Correlation-Context` has `Id` property serving as single unique identifier of the whole operation and implementation MUST generate one if it is missing.
77

8-
[Root Request Id](HierarchicalRequestId.md#root-request-id-generation) requirements and generation considerations must be used for flat Request-Id
8+
It is important to log `Request-Id` received from the upstream service along with the incoming request. It ensures that parent-child relationships between requests are retained and the whole tree of the requests could be restored.
9+
Therefore implementations MUST provide access to the 'parent' Request-Id for logging system.
910

10-
## Correlation Id
11-
Many applications and tracing systems use single correlation id to identify whole operation through all services and client applications.
11+
[Root Request Id](HierarchicalRequestId.md#root-request-id-generation) requirements and generation considerations must be used for flat Request-Id
1212

13+
## Correlation Id (Trace Id)
14+
Many applications and tracing systems use single correlation/trace id to identify whole operation through all services and client applications.
1315
In case of heterogeneous environment (where some services generate hierarchical Request-Ids and others generate flat Ids) having single identifier, common for all requests, helps to make telemetry query simple and efficient.
1416

15-
Implementations MUST use `Id` property in `Correlation-Context` if they need propagate correlation id across the cluster.
16-
Implementation it MUST ensure `Id` is present in `Correlation-Context` or [generate](#correlation-id-generation) new one and add to the `Correlation-Context`.
17+
If implementation generates flat Request-Id, it MUST ensure `Id` is present in `Correlation-Context` or [generate](#correlation-id-generation) new one and add to the `Correlation-Context`.
1718

1819
### Correlation Id generation
1920
If implementation needs to add `Id` property to `Correlation-Context`:

src/libraries/System.Diagnostics.DiagnosticSource/src/HttpCorrelationProtocol.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ These scenarios require every request to carry additional context and services t
1212
Tracing an operation involves an overhead on application performance and should always be considered as optional, so application may not trace anything, trace only particular operations or some percent of all operations.
1313
Tracing should be consistent: operation should be either fully traced, or not traced at all.
1414

15-
This document provides guidance on the context needed for telemetry correlation and describes its format in HTTP communication. The context is not specific to HTTP protocol, it represents set of identifiers that are needed or helpful for end-to-end tracing. Application widely use distributed queues for asynchronous processing so operation may start (or continue) from a queue message; applications should propagate the context through the queues and restore (create) it when they start processing received task.
15+
This document provides guidance on the context needed for telemetry correlation and describes its format in HTTP communication. The context is not specific to HTTP protocol, it represents set of identifiers that is needed or helpful for end-to-end tracing. Applications widely use distributed queues for asynchronous processing so operation may start (or continue) from a queue message; applications should propagate the context through the queues and restore (create) it when they start processing received task.
1616

1717
# HTTP Protocol proposal
1818
| Header name | Format | Description |
@@ -26,34 +26,34 @@ This document provides guidance on the context needed for telemetry correlation
2626
Request-Id is generated on the caller side and passed to callee.
2727

2828
Implementation of this protocol should expect to receive `Request-Id` in header of incoming request.
29-
Absence of Request-Id indicates that it is either first instrumented service in the system or this request was not traced by upstream service and therefore does not have any context associated with it.
29+
Absence of Request-Id indicates that it is either the first instrumented service in the system or this request was not traced by upstream service and therefore does not have any context associated with it.
3030
To start tracing the request, implementation MUST generate new `Request-Id` (see [Root Request Id Generation](#root-request-id-generation)) for the incoming request.
3131

3232
When Request-Id is provided by upstream service, there is no guarantee that it is unique within the entire system.
3333
Implementation SHOULD make it unique by adding small suffix to incoming Request-Id to represent internal activity and use it for outgoing requests, see more details in [Hierarchical Request-Id document](HierarchicalRequestId.md).
3434

35-
`Request-Id` is required field, which means that every instrumented request MUST have it. If implementation does not find `Request-Id` in the incoming request headers, it should consider it as non-traced and MAY not look for `Correlation-Context`.
35+
`Request-Id` is required field, i.e., every instrumented request MUST have it. If implementation does not find `Request-Id` in the incoming request headers, it should consider it as non-traced and MAY not look for `Correlation-Context`.
3636

3737
It is essential that 'incoming' and 'outgoing' Request-Ids are included in the telemetry events, so implementation of this protocol MUST provide read access to Request-Id for logging systems.
3838

3939
### Request-Id Format
4040
`Request-Id` is a string up to 1024 bytes length. It contains only [Base64](https://en.wikipedia.org/wiki/Base64) and "-" (hyphen), "|" (vertical bar), "." (dot), and "_" (underscore) characters.
4141

42-
Vertical bar, dot and underscore are reserved characters that used to mark and delimit hierarchical Request-Id, and must not be present in the nodes. Hyphen may be used in the nodes.
42+
Vertical bar, dot and underscore are reserved characters that are used to mark and delimit hierarchical Request-Id, and must not be present in the nodes. Hyphen may be used in the nodes.
4343

4444
Implementations SHOULD support hierarchical structure for the Request-Id, described in [Hierarchical Request-Id document](HierarchicalRequestId.md).
4545
See [Flat Request-Id](FlatRequestId.md) for non-hierarchical Request-Id requirements.
4646

4747
## Correlation-Context
48-
Root service can add state (key value pairs) that will automatically propagate to all other services including intermediary services (that support this protocol). A typical scenarios are sampling or A/B testing (feature flags) so that the root service has a way to pass this kind of information down to all services (including intermediary). All services other than the root SHOULD consider Correlation-Context as read-only.
48+
First service MAY add state (key value pairs) that will automatically propagate to all other services including intermediary services (that support this protocol). A typical scenarios for the Correlation-Context include logging control and sampling or A/B testing (feature flags) so that the first service has a way to pass this kind of information down to all services (including intermediary). All services other than the first one SHOULD consider Correlation-Context as read-only.
49+
It is important to keep the size of any property small because these get serialized into HTTP headers which have significant size restrictions; Correlation-Context parsing, storing and propagation involves performance overhead on all downstream services.
4950

50-
We anticipate that there will be common well-known Correlation-Context keys. If you wish to use this for you own custom (not well-known) context key, prefix it with "@".
51-
52-
It is important to keep the size of any property because they get serialized in HTTP headers and headers have significant size restrictions. The Correlation-Context MUST NOT be used as generic data passing mechanism (between services or within service).
51+
Correlation-Context MUST NOT be used as generic data passing mechanism between services or within one service.
5352

54-
`Correlation-Context` is optional, which means that it may or may not be provided by upstream service.
53+
We anticipate that there will be common well-known Correlation-Context keys. If you wish to use this for you own custom (not well-known) context key, prefix it with "@".
5554

56-
If `Correlation-Context` is provided by upstream service, implementation MUST propagate it further to downstream services.
55+
`Correlation-Context` is optional, it may or may not be provided by upstream (instrumented) service.
56+
If `Correlation-Context` is provided by upstream service, implementation MUST propagate it further to downstream services. It MUST NOT change or remove properties and SHOULD NOT add new properties.
5757

5858
Implementation MUST provide read access to `Correlation-Context` for logging systems and MUST support adding properties to Correlation-Context.
5959

@@ -62,11 +62,11 @@ Implementation MUST provide read access to `Correlation-Context` for logging sys
6262

6363
`Correlation-Context: key1=value1, key2=value2`
6464

65-
Neither keys nor values MUST NOT contain "=" (equals) or "," (comma) characters.
65+
Keys and values MUST NOT contain "=" (equals) or "," (comma) characters.
6666

6767
Overall Correlation-Context length MUST NOT exceed 1024 bytes, key and value length should stay well under the combined limit of 1024 bytes.
6868

69-
Note that uniqueness of the key within the Correlation-Context is not guaranteed. Context received from upstream service is read-only and implementation MUST not remove or aggregate duplicated keys.
69+
Note that uniqueness of the key within the Correlation-Context is not guaranteed. Context received from upstream service is read-only and implementation MUST NOT remove or aggregate duplicated keys.
7070

7171
# HTTP Guidelines and Limitations
7272
- [HTTP 1.1 RFC2616](https://tools.ietf.org/html/rfc2616)

0 commit comments

Comments
 (0)