Skip to content

Remove opentelemetry as the default metrics library #97

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
May 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
b255f8c
[WIP]
emschwartz Apr 21, 2023
7cbe053
Update prometheus-client version
emschwartz May 16, 2023
ff161ea
Merge branch 'main' into prometheus-client-2
emschwartz May 16, 2023
9e402eb
Finish support for prometheus-client
emschwartz May 16, 2023
f29aae1
Fix doc links
emschwartz May 16, 2023
11d052f
Ensure that prometheus-client is actually being used
emschwartz May 16, 2023
9817601
Work on fixing tests, remove const_format dep
emschwartz May 16, 2023
8635ad7
Don't use regexes in tests (because different libraries order labels …
emschwartz May 17, 2023
8130657
Make once_cell a required dependency
emschwartz May 17, 2023
49410dc
Rename concurrency tracker to gauge
emschwartz May 17, 2023
537553e
Merge branch 'main' into prometheus-client-2
emschwartz May 17, 2023
ecfe52f
Move PROMETHEUS_CLIENT_REGISTRY to integrations module
emschwartz May 19, 2023
e311510
Run all the other tests first
emschwartz May 19, 2023
ad830c3
Run the tests in the same order
emschwartz May 19, 2023
4406f08
The prometheus-exporter feature needs to depend on the prometheus fea…
emschwartz May 19, 2023
43bea12
Rename export to just REGISTRY
emschwartz May 19, 2023
36f65e2
Mention prometheus-client in readme and changelog
emschwartz May 19, 2023
e3e2a9c
Remove opentelemetry as the default metrics library
emschwartz May 23, 2023
140c7ce
Disable opentelemetry tests
emschwartz May 23, 2023
30f531f
continue-on-error
emschwartz May 23, 2023
5c67249
previously...
emschwartz May 23, 2023
7cd5e69
Merge branch 'main' into no-default-metrics-library
emschwartz May 23, 2023
ce948b5
Remove opentelemetry memory api call
emschwartz May 23, 2023
31ac39f
Fix error type
emschwartz May 23, 2023
ba0849f
Remove duplicate code
emschwartz May 23, 2023
80cec89
Make metrics library features mutually exclusive
emschwartz May 23, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 11 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,15 +24,18 @@ jobs:
- run: cargo clippy --features=prometheus-client
- run: cargo clippy --features=opentelemetry

# Build the packages
- run: cargo build
- run: cargo build --features=custom-objective-percentile,custom-objective-latency

# Run the tests with each of the different metrics libraries
- run: cargo test --features=prometheus-exporter
- run: cargo test --no-default-features --features=prometheus-exporter,metrics --tests
- run: cargo test --no-default-features --features=prometheus-exporter,prometheus --tests
- run: cargo test --no-default-features --features=prometheus-exporter,prometheus-client --tests
- run: cargo test --features=prometheus-exporter,metrics
- run: cargo test --features=prometheus-exporter,prometheus --tests
- run: cargo test --features=prometheus-exporter,prometheus-client --tests
# The tests using opentelemetry are currently failing because of this issue:
# https://github.com/open-telemetry/opentelemetry-rust/issues/1070
# We should remove the continue-on-error once that is fixed
- run: cargo test --features=prometheus-exporter,opentelemetry
continue-on-error: true

# Build the crate using the other optional features
- run: cargo build --features=metrics,custom-objective-percentile,custom-objective-latency

# Compile the examples
- run: cargo build --package example-axum
Expand Down
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- Users must configure the metrics library they want to use autometrics with
- Metrics library feature flags are now mutually exclusive (previously, `autometrics` would only
produce metrics using a single metrics library if multiple feature flags were enabled, using
a prioritization order defined internally)
- `GetLabels` trait (publicly exported but meant for internal use) changed the signature
of its function to accomodate the new `ResultLabels` macro. This change is not significant
if you never imported `autometrics::__private` manually (#61)
- When using the `opentelemetry` together with the `prometheus-exporter`, it will no longer
use the default registry provided by the `prometheus` crate. It will instead use a new registry
- The `prometheus-exporter`'s `encode_global_metrics` feature now returns an error enum
defined by `autometrics` as opposed to directly returning the `prometheus::Error` type
- Updated `opentelemetry` dependencies to v0.19. This means that users using autometrics
with `opentelemetry` but not using the `prometheus-exporter` must update the `opentelemetry`
to use v0.19.
Expand All @@ -29,6 +37,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Removed

- `opentelemetry` is no longer used by default
- `GetLabelsForResult` trait (publicly exported but meant for internal use) was removed
to accomodate the new `ResultLabels` macro. This change is not significant
if you never imported `autometrics::__private` manually (#61)
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ See [Why Autometrics?](https://github.com/autometrics-dev#4-why-autometrics) for

1. Add `autometrics` to your project:
```sh
cargo add autometrics --features=prometheus-exporter
cargo add autometrics --features=prometheus,prometheus-exporter
```
2. Instrument your functions with the [`#[autometrics]`](https://docs.rs/autometrics/latest/autometrics/attr.autometrics.html) macro

Expand Down
5 changes: 3 additions & 2 deletions autometrics/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ categories = { workspace = true }
readme = "README.md"

[features]
default = ["opentelemetry"]
metrics = ["dep:metrics"]
opentelemetry = ["opentelemetry_api"]
prometheus = ["dep:prometheus"]
Expand All @@ -22,7 +21,8 @@ prometheus-exporter = [
"metrics-exporter-prometheus",
"opentelemetry-prometheus",
"opentelemetry_sdk",
"prometheus"
"dep:prometheus",
"thiserror"
]
custom-objective-percentile = []
custom-objective-latency = []
Expand All @@ -43,6 +43,7 @@ metrics-exporter-prometheus = { version = "0.12", default-features = false, opti
opentelemetry-prometheus = { version = "0.12.0", optional = true }
opentelemetry_sdk = { version = "0.19", default-features = false, features = ["metrics"], optional = true }
prometheus = { version = "0.13", default-features = false, optional = true }
thiserror = { version = "1", optional = true }

# Used for prometheus-client feature
prometheus-client = { version = "0.21.1", optional = true }
Expand Down
5 changes: 3 additions & 2 deletions autometrics/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,12 +115,13 @@ fn main() {

#### Metrics Libraries

Configure the crate that autometrics will use to produce metrics by using one of the following feature flags:
**Required:** Configure the crate that autometrics will use to produce metrics by using one of the following feature flags:

> **Note**
>
> If you are **not** using the `prometheus-exporter`, you must ensure that you are using the exact same version of the metrics library as `autometrics` (and it must come from `crates.io` rather than git or another source). If not, the autometrics metrics will not appear in your exported metrics.

- `opentelemetry` (enabled by default) - use the [opentelemetry](https://crates.io/crates/opentelemetry) crate for producing metrics.
- `opentelemetry` - use the [opentelemetry](https://crates.io/crates/opentelemetry) crate for producing metrics.
- `metrics` - use the [metrics](https://crates.io/crates/metrics) crate for producing metrics
- `prometheus` - use the [prometheus](https://crates.io/crates/prometheus) crate for producing metrics
- `prometheus-client` - use the official [prometheus-client](https://crates.io/crates/prometheus-client) crate for producing metrics
110 changes: 66 additions & 44 deletions autometrics/src/prometheus_exporter.rs
Original file line number Diff line number Diff line change
@@ -1,48 +1,94 @@
use crate::HISTOGRAM_BUCKETS;
#[cfg(feature = "metrics")]
use metrics_exporter_prometheus::{PrometheusBuilder, PrometheusHandle};
use once_cell::sync::Lazy;
#[cfg(feature = "opentelemetry")]
use opentelemetry_prometheus::{exporter, PrometheusExporter};
#[cfg(feature = "opentelemetry")]
use opentelemetry_sdk::export::metrics::aggregation;
#[cfg(feature = "opentelemetry")]
use opentelemetry_sdk::metrics::{controllers, processors, selectors};
use prometheus::{default_registry, Error, TextEncoder};
#[cfg(any(feature = "opentelemetry", feature = "prometheus"))]
use prometheus::TextEncoder;
use thiserror::Error;

static GLOBAL_EXPORTER: Lazy<GlobalPrometheus> = Lazy::new(initialize_metrics_exporter);
#[cfg(not(any(
feature = "metrics",
feature = "opentelemetry",
feature = "prometheus",
feature = "prometheus-client"
)))]
compile_error!("At least one of the metrics, opentelemetry, prometheus, or prometheus-client features must be enabled in order to use the prometheus-exporter");

#[derive(Debug, Error)]
pub enum EncodingError {
#[cfg(any(feature = "prometheus", feature = "opentelemetry"))]
#[error(transparent)]
Prometheus(#[from] prometheus::Error),
#[cfg(feature = "prometheus-client")]
#[error(transparent)]
Format(#[from] std::fmt::Error),
}

static GLOBAL_EXPORTER: Lazy<GlobalPrometheus> = Lazy::new(|| GlobalPrometheus {
#[cfg(feature = "metrics")]
metrics_exporter: PrometheusBuilder::new()
.set_buckets(&crate::HISTOGRAM_BUCKETS)
.expect("Failed to set histogram buckets")
.install_recorder()
.expect("Failed to install recorder"),

#[cfg(feature = "opentelemetry")]
opentelemetry_exporter: exporter(
controllers::basic(processors::factory(
selectors::simple::histogram(crate::HISTOGRAM_BUCKETS),
aggregation::cumulative_temporality_selector(),
))
.build(),
)
.init(),
});

#[derive(Clone)]
#[doc(hidden)]
pub struct GlobalPrometheus {
exporter: PrometheusExporter,
#[cfg(feature = "opentelemetry")]
opentelemetry_exporter: PrometheusExporter,
#[cfg(feature = "metrics")]
handle: PrometheusHandle,
metrics_exporter: PrometheusHandle,
}

impl GlobalPrometheus {
fn encode_metrics(&self) -> Result<String, Error> {
let metric_families = self.exporter.registry().gather();
let encoder = TextEncoder::new();
#[allow(unused_mut)]
let mut output = encoder.encode_to_string(&metric_families)?;
fn encode_metrics(&self) -> Result<String, EncodingError> {
let mut output = String::new();

#[cfg(feature = "metrics")]
{
output.push_str(&self.metrics_exporter.render());
output.push('\n');
output.push_str(&self.handle.render());
}

#[cfg(feature = "prometheus-client")]
#[cfg(feature = "opentelemetry")]
{
let metric_families = self.opentelemetry_exporter.registry().gather();
let encoder = TextEncoder::new();
encoder.encode_utf8(&metric_families, &mut output)?;
output.push('\n');
}

#[cfg(feature = "prometheus")]
{
let metric_families = prometheus::default_registry().gather();
let encoder = TextEncoder::new();
encoder.encode_utf8(&metric_families, &mut output)?;
output.push('\n');
}

#[cfg(feature = "prometheus-client")]
{
prometheus_client::encoding::text::encode(
&mut output,
&crate::tracker::prometheus_client::REGISTRY,
)
.map_err(|err| {
Error::Msg(format!(
"Failed to encode prometheus-client metrics: {}",
err
))
})?;
)?;
}

Ok(output)
Expand Down Expand Up @@ -84,30 +130,6 @@ pub fn global_metrics_exporter() -> GlobalPrometheus {
/// }
/// }
/// ```
pub fn encode_global_metrics() -> Result<String, Error> {
pub fn encode_global_metrics() -> Result<String, EncodingError> {
GLOBAL_EXPORTER.encode_metrics()
}

fn initialize_metrics_exporter() -> GlobalPrometheus {
let controller = controllers::basic(processors::factory(
selectors::simple::histogram(HISTOGRAM_BUCKETS),
aggregation::cumulative_temporality_selector(),
))
.build();

// Use the prometheus crate's default registry so it still works with custom
// metrics defined through the prometheus crate
let registry = default_registry().clone();
let prometheus_exporter = exporter(controller).with_registry(registry).init();

GlobalPrometheus {
exporter: prometheus_exporter,

#[cfg(feature = "metrics")]
handle: PrometheusBuilder::new()
.set_buckets(&HISTOGRAM_BUCKETS)
.expect("Failed to set histogram buckets")
.install_recorder()
.expect("Failed to install recorder"),
}
}
103 changes: 78 additions & 25 deletions autometrics/src/tracker/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,36 +9,89 @@ mod prometheus;
#[cfg(feature = "prometheus-client")]
pub(crate) mod prometheus_client;

// Priority if multiple features are enabled:
// 1. prometheus
// 2. prometheus-client
// 3. metrics
// 4. opentelemetry (default)

// By default, use the opentelemetry crate
#[cfg(all(
feature = "opentelemetry",
not(any(
feature = "metrics",
feature = "prometheus",
feature = "prometheus-client"
))
))]
pub use self::opentelemetry::OpenTelemetryTracker as AutometricsTracker;
#[cfg(feature = "metrics")]
pub use self::metrics::MetricsTracker;
#[cfg(feature = "opentelemetry")]
pub use self::opentelemetry::OpenTelemetryTracker;
#[cfg(feature = "prometheus")]
pub use self::prometheus::PrometheusTracker;
#[cfg(feature = "prometheus-client")]
pub use self::prometheus_client::PrometheusClientTracker;

// But use one of the other crates if any of those features are enabled
#[cfg(all(
feature = "metrics",
not(any(feature = "prometheus", feature = "prometheus-client"))
#[cfg(any(
all(
feature = "metrics",
any(
feature = "opentelemetry",
feature = "prometheus",
feature = "prometheus-client"
)
),
all(
feature = "opentelemetry",
any(feature = "prometheus", feature = "prometheus-client")
),
all(feature = "prometheus", feature = "prometheus-client")
))]
pub use self::metrics::MetricsTracker as AutometricsTracker;
#[cfg(feature = "prometheus")]
pub use self::prometheus::PrometheusTracker as AutometricsTracker;
#[cfg(all(feature = "prometheus-client", not(feature = "prometheus")))]
pub use self::prometheus_client::PrometheusClientTracker as AutometricsTracker;
compile_error!("Only one of the metrics, opentelemetry, prometheus, or prometheus-client features can be enabled at a time");

pub trait TrackMetrics {
fn set_build_info(build_info_labels: &BuildInfoLabels);
fn start(gauge_labels: Option<&GaugeLabels>) -> Self;
fn finish(self, counter_labels: &CounterLabels, histogram_labels: &HistogramLabels);
}

pub struct AutometricsTracker {
#[cfg(feature = "metrics")]
metrics_tracker: MetricsTracker,
#[cfg(feature = "opentelemetry")]
opentelemetry_tracker: OpenTelemetryTracker,
#[cfg(feature = "prometheus")]
prometheus_tracker: PrometheusTracker,
#[cfg(feature = "prometheus-client")]
prometheus_client_tracker: PrometheusClientTracker,
}

impl TrackMetrics for AutometricsTracker {
#[allow(unused_variables)]
fn set_build_info(build_info_labels: &BuildInfoLabels) {
#[cfg(feature = "metrics")]
MetricsTracker::set_build_info(build_info_labels);
#[cfg(feature = "opentelemetry")]
OpenTelemetryTracker::set_build_info(build_info_labels);
#[cfg(feature = "prometheus")]
PrometheusTracker::set_build_info(build_info_labels);
#[cfg(feature = "prometheus-client")]
PrometheusClientTracker::set_build_info(build_info_labels);
}

#[allow(unused_variables)]
fn start(gauge_labels: Option<&GaugeLabels>) -> Self {
Self {
#[cfg(feature = "metrics")]
metrics_tracker: MetricsTracker::start(gauge_labels),
#[cfg(feature = "opentelemetry")]
opentelemetry_tracker: OpenTelemetryTracker::start(gauge_labels),
#[cfg(feature = "prometheus")]
prometheus_tracker: PrometheusTracker::start(gauge_labels),
#[cfg(feature = "prometheus-client")]
prometheus_client_tracker: PrometheusClientTracker::start(gauge_labels),
}
}

#[allow(unused_variables)]
fn finish(self, counter_labels: &CounterLabels, histogram_labels: &HistogramLabels) {
#[cfg(feature = "metrics")]
self.metrics_tracker
.finish(counter_labels, histogram_labels);
#[cfg(feature = "opentelemetry")]
self.opentelemetry_tracker
.finish(counter_labels, histogram_labels);
#[cfg(feature = "prometheus")]
self.prometheus_tracker
.finish(counter_labels, histogram_labels);
#[cfg(feature = "prometheus-client")]
self.prometheus_client_tracker
.finish(counter_labels, histogram_labels);
}
}
2 changes: 1 addition & 1 deletion examples/axum/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ publish = false
edition = "2021"

[dependencies]
autometrics = { path = "../../autometrics", features = ["prometheus-exporter"] }
autometrics = { path = "../../autometrics", features = ["prometheus", "prometheus-exporter"] }
autometrics-example-util = { path = "../util" }
axum = { version = "0.6", features = ["json"] }
rand = "0.8"
Expand Down
2 changes: 1 addition & 1 deletion examples/full-api/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ publish = false
edition = "2021"

[dependencies]
autometrics = { path = "../../autometrics", features = ["prometheus-exporter"] }
autometrics = { path = "../../autometrics", features = ["prometheus", "prometheus-exporter"] }
autometrics-example-util = { path = "../util" }
axum = { version = "0.6", features = ["json"] }
rand = "0.8"
Expand Down