Alignment on duration metric units

[drewrelmas]

Duration metric units: should we standardize on seconds?

Our internal flow/duration metrics are emitted in nanoseconds (unit = "ns"), while the Collector standardizes its duration telemetry on seconds (unit = "s", Double histograms). We should decide whether to align with the OpenTelemetry convention before more duration metrics are added in the repo.

Background

The recently added flow duration metric is the trigger for this discussion:

otel-arrow/rust/otap-dataflow/crates/engine/src/flow_metrics.rs

Lines 40 to 47 in 3159ecb

	/// Duration metric set emitted by the end node of a flow range.
	#[metric_set(name = "flow")]
	#[derive(Debug, Default, Clone)]
	pub struct FlowDurationMetrics {
	/// Sum of per-node compute durations (nanoseconds) for messages traversing the flow range.
	#[metric(name = "compute.duration", unit = "ns")]
	pub compute_duration: Mmsc,
	}

This is consistent with the rest of our codebase, where ns is the dominant convention for internal durations.

The Collector emits processor duration as the following:

### otelcol_processor_internal_duration

Duration of time taken to process a batch of telemetry data through the processor.

| Unit | Metric Type | Value Type | Stability |
| ---- | ----------- | ---------- | --------- |
| s | Histogram | Double | Alpha |

The Metrics semantic convention guide also suggests using seconds when talking about duration.

When instruments are measuring durations, seconds (i.e. s) SHOULD be used.

Options

1. Keep ns integer sums (status quo). Maximum precision, matches the internal convention, but diverges from OpenTelemetry's s/Double convention.
2. Adopt s to match OpenTelemetry. Best interop with Collector dashboards and semconv.
3. Keep ns internally, convert to s at the export boundary. Preserve integer precision in-process and only rescale when exporting externally.

[cijothomas]

OTel semantic conventions recommend "seconds", but almost all critical ones like HTTP metrics has a line below that saying "must use custom Histogram buckets", and override the default buckets to "[ 0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10 ]".. (We don't support histogram yet, but once we do, the unit and buckets should align)