Skip to content

Allow primitive OTEL delta ingestion to be enabled #11631

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jul 7, 2025

Conversation

fionaliao
Copy link
Contributor

@fionaliao fionaliao commented Jun 4, 2025

What this PR does

This PR introduces the -distributor.otel-native-delta-ingestion flag (and corresponding per-tenant setting), which enables primitive OTEL delta metrics ingestion via the OTLP endpoint. This feature was implemented in Prometheus in prometheus/prometheus#16360. This PR allows Mimir users to enable this feature too.

As per the Prometheus PR:

This allows otlp metrics with delta temporality to be ingested and stored as-is, with metric type unknown. To get "increase" or "rate", sum_over_time(metric[<interval>]) (/ <interval>) can be used.

This is the first step towards implementing prometheus/proposals#48. That proposal has additional suggestions around type-aware functions and making the rate() and increase() functions work for deltas too. However, there are some questions around the best way to do querying over deltas, so having this simple implementation without changing any PromQL functions allow us to get some form of delta ingestion out there gather some feedback to decide the best way to go further.

Which issue(s) this PR fixes or relates to

An initial PR for #10439

Checklist

  • Tests updated.
  • Documentation added.
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
  • about-versioning.md updated with experimental features.

@fionaliao fionaliao changed the title Allow primitive OTEL delta ingestion to be enabled Allow primitive OTEL delta ingestion to be enabled on a per-tenant basis Jun 4, 2025
@fionaliao fionaliao force-pushed the fl/primitive-delta-support branch from c24d3f4 to 26fca3e Compare June 4, 2025 16:55
Copy link
Contributor

github-actions bot commented Jun 4, 2025

💻 Deploy preview deleted.

@fionaliao fionaliao changed the title Allow primitive OTEL delta ingestion to be enabled on a per-tenant basis Allow primitive OTEL delta ingestion to be enabled Jun 4, 2025
@fionaliao fionaliao marked this pull request as ready for review June 16, 2025 17:43
@fionaliao fionaliao requested review from tacole02 and a team as code owners June 16, 2025 17:43
Copy link
Contributor

@tacole02 tacole02 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @fionaliao !

@fionaliao fionaliao force-pushed the fl/primitive-delta-support branch from c4a1e7f to b4f3f64 Compare July 4, 2025 15:23
@aknuds1 aknuds1 requested a review from Copilot July 7, 2025 12:43
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds an experimental flag to ingest raw OTLP delta metrics without conversion.

  • Introduce OTelNativeDeltaIngestion in limits, overrides, handler, and converter
  • Wire the new flag through tests, CLI help, JSON descriptor, and documentation
  • Add TestOTelDeltaIngestion to validate delta behavior

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.

Show a summary per file
File Description
pkg/util/validation/limits.go Add OTelNativeDeltaIngestion field and register the CLI flag
pkg/distributor/push_test.go Add mock implementation for OTelNativeDeltaIngestion
pkg/distributor/otel.go Extend OTLPHandlerLimits and conversionOptions to include delta flag
pkg/distributor/otel_test.go New TestOTelDeltaIngestion covering allowed and disallowed cases
docs/sources/mimir/configure/configuration-parameters/index.md Document the otel_native_delta_ingestion parameter
docs/sources/mimir/configure/about-versioning.md Add experimental feature entry for delta ingestion
cmd/mimir/help-all.txt.tmpl Add CLI help entry for -distributor.otel-native-delta-ingestion
cmd/mimir/config-descriptor.json Add config descriptor for otel_native_delta_ingestion
CHANGELOG.md Log the new experimental feature under [FEATURE]
Comments suppressed due to low confidence (2)

docs/sources/mimir/configure/about-versioning.md:119

  • The backticked flag is missing its leading hyphen. It should be -distributor.otel-native-delta-ingestion to match other entries.
    - `distributor.otel-native-delta-ingestion`

cmd/mimir/help-all.txt.tmpl:1428

  • Align this flag's indentation with other entries (use a leading tab) so the CLI help output remains consistently formatted.
  -distributor.otel-native-delta-ingestion

Copy link
Contributor

@aknuds1 aknuds1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@fionaliao fionaliao merged commit 5e033a8 into main Jul 7, 2025
33 checks passed
@fionaliao fionaliao deleted the fl/primitive-delta-support branch July 7, 2025 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants