Skip to content

Metric Exemplars SDK Specification#1828

Merged
jmacd merged 14 commits intoopen-telemetry:mainfrom
jsuereth:wip-exemplar
Aug 30, 2021
Merged

Metric Exemplars SDK Specification#1828
jmacd merged 14 commits intoopen-telemetry:mainfrom
jsuereth:wip-exemplar

Conversation

@jsuereth
Copy link
Contributor

@jsuereth jsuereth commented Jul 27, 2021

Fixes #1797

Changes

Add Exemplar specification and customization to the metrics specification.

Related issues #1260 #617

Related oteps #113

Notes

  • This exposes two extensible interfaces. It's fine to drop these if they are contentious. The key important piece we must keep is the ability to turn OFF exemplars and the ability to limit exemplars to only measurements with associated sampled spans on context.
  • This diverges from the Exemplar OTEP in that it is less restrictive on behavior of the exemplar reservoir (currently). Additionally the aggregators available when the exemplar OTEP was written are not yet specified. We can refine this in a follow on PR as necessary. We should align any sampling decisions w/ the future of tracing sampling.
  • There are still some TODOs for discussion.

weyert
weyert reviewed Jul 29, 2021
View reviewed changes
jmacd
jmacd reviewed Jul 29, 2021
View reviewed changes
Copy link
Contributor

@jmacd jmacd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

aabmass
aabmass reviewed Aug 5, 2021
View reviewed changes
@jmacd
Copy link
Contributor

jmacd commented Aug 12, 2021

FYI I have been working on trace sampling lately, and there is a lot of alignment between that work and this work. I have made claims that we can use weighted sampling to implement a hard rate limit on spans inside a tail sampler, and by extension I claim to know how to do the same for histogram exemplars. For a histogram aggregator outputting high-resolution histogram buckets you might not want one exemplar per bucket, and with weighted sampling we can produce a fixed number (i.e., using a reservoir) of exemplars with an equal expected number of spans per bucket.

This simple approach uses two passes, documented in the README of the https://github.com/lightstep/varopt repository. The more complicated approach can do this approximately in a single pass, which I also discussed briefly in today's Sampling SIG. See this draft. I will elaborate on this in a prototype exemplar reservoir tail sampler based on Varopt.

@jsuereth jsuereth marked this pull request as ready for review August 17, 2021 13:40
@jsuereth jsuereth requested review from a team August 17, 2021 13:40
@jsuereth
Copy link
Contributor Author

jsuereth commented Aug 17, 2021

@jmacd I agree I think we can do way better than status quo, especially for high-fidelity histograms. I re-tweaked the wording to leave room for an exemplar reservoir sampling specification when those land.

@github-actions
Copy link

github-actions bot commented Aug 25, 2021

This PR was marked stale due to lack of activity. It will be closed in 7 days.

@github-actions github-actions bot added the Stale label Aug 25, 2021
@jsuereth jsuereth added area:sdk Related to the SDK spec:metrics Related to the specification/metrics directory and removed Stale labels Aug 25, 2021
jmacd
jmacd approved these changes Aug 25, 2021
View reviewed changes
@jsuereth jsuereth changed the title First cut at exemplar spec. Metric Exemplars SDK Specification Aug 25, 2021
reyang
reyang reviewed Aug 26, 2021
View reviewed changes
reyang
reyang reviewed Aug 26, 2021
View reviewed changes
reyang
reyang reviewed Aug 26, 2021
View reviewed changes
reyang
reyang reviewed Aug 26, 2021
View reviewed changes
reyang
reyang reviewed Aug 26, 2021
View reviewed changes
reyang
reyang approved these changes Aug 26, 2021
View reviewed changes
Copy link
Member

@reyang reyang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I've left some minor comments.

aabmass
aabmass approved these changes Aug 26, 2021
View reviewed changes
specification/metrics/sdk.md Outdated
The "offer" method SHOULD accept measurements, including:

- value
- `Attributes`
Copy link
Member

@aabmass aabmass Aug 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above

specification/metrics/sdk.md Outdated
applies to [synchronous Instruments](./api.md#synchronous-instrument).
* The `aggregation` (optional) to be used. If not provided, a default
aggregation will be applied by the SDK. The default aggregation is a TODO.
* The `exemplar_reservoir` to use for storing exemplars.
Copy link
Member

@aabmass aabmass Aug 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be marked optional right?

jsuereth and others added 9 commits August 26, 2021 13:39
@jsuereth @reyang
Remove the word parent
2e6db06 
Co-authored-by: Reiley Yang <reyang@microsoft.com>
@jsuereth @reyang
Update specification/metrics/sdk.md
8a9b6a9 
Co-authored-by: Reiley Yang <reyang@microsoft.com>
@jsuereth @reyang
Update specification/metrics/sdk.md
26b5957 
Co-authored-by: Reiley Yang <reyang@microsoft.com>
@jsuereth @reyang
Update specification/metrics/sdk.md
09dd010 
Co-authored-by: Reiley Yang <reyang@microsoft.com>
@jsuereth @reyang
Update specification/metrics/sdk.md
ff106f4 
Co-authored-by: Reiley Yang <reyang@microsoft.com>
@jsuereth @aabmass
Update specification/metrics/sdk.md
4c7e082 
Co-authored-by: Aaron Abbott <aaronabbott@google.com>
@jsuereth
Merge remote-tracking branch 'origin/main' into wip-exemplar
67e4ff6
@jsuereth
Merge branch 'wip-exemplar' of https://github.com/jsuereth/openteleme…
b451a7b 
…try-specification into wip-exemplar
@jsuereth
Fixes from review.
530859b
jmacd
jmacd reviewed Aug 27, 2021
View reviewed changes
@jmacd jmacd merged commit 307f20b into open-telemetry:main Aug 30, 2021
@jsuereth jsuereth deleted the wip-exemplar branch August 31, 2021 13:30
@reyang reyang mentioned this pull request Sep 10, 2021
Merged
@jsuereth jsuereth restored the wip-exemplar branch December 6, 2021 13:59
carlosalberto pushed a commit to carlosalberto/opentelemetry-specification that referenced this pull request Oct 31, 2024
@jsuereth @reyang @aabmass
Metric Exemplars SDK Specification (open-telemetry#1828)
485e0bb 
* First cut at exemplar spec.

* Update exemplar specification.

* Remove the word parent

Co-authored-by: Reiley Yang <reyang@microsoft.com>

* Update specification/metrics/sdk.md

Co-authored-by: Reiley Yang <reyang@microsoft.com>

* Update specification/metrics/sdk.md

Co-authored-by: Reiley Yang <reyang@microsoft.com>

* Update specification/metrics/sdk.md

Co-authored-by: Reiley Yang <reyang@microsoft.com>

* Update specification/metrics/sdk.md

Co-authored-by: Reiley Yang <reyang@microsoft.com>

* Update specification/metrics/sdk.md

Co-authored-by: Aaron Abbott <aaronabbott@google.com>

* Fixes from review.

* Clarify a confusing paragraph.

Co-authored-by: Reiley Yang <reyang@microsoft.com>
Co-authored-by: Aaron Abbott <aaronabbott@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jmacd jmacd jmacd approved these changes

@reyang reyang reyang approved these changes

+2 more reviewers

@weyert weyert weyert left review comments

@aabmass aabmass aabmass approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@bogdandrutu bogdandrutu

Labels

area:sdk Related to the SDK spec:metrics Related to the specification/metrics directory

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Define Exemplar requirements in the Metrics SDK spec

6 participants

@jsuereth @jmacd @weyert @aabmass @reyang @bogdandrutu