From 7fe63e4301a2a1d07fa78d79523a281242d4a68e Mon Sep 17 00:00:00 2001 From: Pavel Sirotkin <285394121+pvlsirotkin@users.noreply.github.com> Date: Wed, 27 May 2026 15:18:43 +0000 Subject: [PATCH 1/4] Add gen_ai.workflow.steps metric Adds a new GenAI semantic convention metric that measures the number of discrete steps observed within a single GenAI workflow: * gen_ai.workflow.steps (histogram, {step}): count of observable events within one workflow execution. A 'step' is intentionally framework-defined - it typically corresponds to one event (a tool call, a tool result, a model response in ADK; a node execution in LangGraph, etc.). Dimensioned by gen_ai.workflow.name; when set, gen_ai.agent.name and gen_ai.agent.version may be used to slice steps per-agent within a workflow. In that case the count is restricted to events attributed to that agent. This metric is colocated with gen_ai.workflow.duration (added in #126) in the model and docs so the two workflow-level metrics live together. NOTE: docs/registry/ and schema-snapshot/ regeneration via 'make generate-all' has NOT been run in this commit (no Docker available in the authoring environment). Run it locally before pushing for review. --- CHANGELOG.md | 3 + docs/gen-ai/gen-ai-metrics.md | 67 +++++++++++++++++ model/gen-ai/metrics.yaml | 34 +++++++++ schema-snapshot/registry.yaml | 137 ++++++++++++++++++++++++++++++++++ 4 files changed, 241 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9cd505f12..029268bbe 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,6 +31,9 @@ ([#126](https://github.com/open-telemetry/semantic-conventions-genai/pull/126)) - Add `moonshot_ai` to `gen_ai.provider.name` well-known values. ([#99](https://github.com/open-telemetry/semantic-conventions-genai/pull/99)) +- Add `gen_ai.workflow.steps` metric to track the number of discrete steps + observed within a single GenAI workflow. + ([#203](https://github.com/open-telemetry/semantic-conventions-genai/pull/203)) ### 🧰 Bug fixes 🧰 diff --git a/docs/gen-ai/gen-ai-metrics.md b/docs/gen-ai/gen-ai-metrics.md index f479c9d9e..9f91b2396 100644 --- a/docs/gen-ai/gen-ai-metrics.md +++ b/docs/gen-ai/gen-ai-metrics.md @@ -19,6 +19,7 @@ linkTitle: Metrics - [Metric: `gen_ai.server.time_to_first_token`](#metric-gen_aiservertime_to_first_token) - [Generative AI workflow metrics](#generative-ai-workflow-metrics) - [Metric: `gen_ai.workflow.duration`](#metric-gen_aiworkflowduration) + - [Metric: `gen_ai.workflow.steps`](#metric-gen_aiworkflowsteps) @@ -881,6 +882,12 @@ If instrumentation measures only a single provider-facing client operation (for example, one model API call), `gen_ai.client.operation.duration` SHOULD be used instead. Instrumentation MAY emit both metrics for the same request path when both boundaries are available. +`gen_ai.workflow.steps` complements `gen_ai.workflow.duration` by counting the +number of discrete steps observed within a single workflow execution. It is +intended for instrumentations of agent frameworks (for example, ADK, +LangChain agents, CrewAI agents) that can attribute observable events to a +specific workflow or agent. + ### Metric: `gen_ai.workflow.duration` This metric is [required][MetricRequired] when instrumented component implements workflow operations. @@ -931,6 +938,66 @@ If there is no low-cardinality workflow name available for a given framework, th +### Metric: `gen_ai.workflow.steps` + +This metric is [recommended][MetricRecommended] for instrumentations that +can reliably attribute steps (events) to a specific workflow or agent. + +A "step" is intentionally framework-defined; it typically corresponds to a +single observable event emitted within the workflow (for example, a tool +call, a tool result, or a model response in ADK; a node execution in +LangGraph). When the `gen_ai.agent.name` dimension is set, instrumentations +SHOULD only count steps attributed to that agent; events attributed to +sub-agents or to a parent workflow SHOULD NOT be counted. + +This metric SHOULD be specified with [ExplicitBucketBoundaries] of +[1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]. + + + + + + +| Name | Instrument Type | Unit (UCUM) | Description | Stability | Entity Associations | +| -------- | --------------- | ----------- | -------------- | --------- | ------ | +| `gen_ai.workflow.steps` | Histogram | `{step}` | Number of steps in a single GenAI workflow. [1] | ![Development](https://img.shields.io/badge/-development-blue) | | + +**[1]:** This metric measures the number of discrete steps observed within a +single GenAI workflow. A "step" is intentionally framework-defined: it +typically corresponds to a single observable event emitted within the +workflow (for example, a tool call, a tool result, or a model response +in ADK; a node execution in LangGraph). + +When the `gen_ai.agent.name` dimension is set, instrumentations SHOULD +only count steps attributed to that agent (for example, events +authored by that agent); events attributed to sub-agents or to a +parent workflow SHOULD NOT be counted. + +This metric complements `gen_ai.workflow.duration` by giving operators +visibility into how "chatty" or compute-intensive a workflow is, and +is useful for capacity planning and regression detection across +versions. + +**Attributes:** + +| Key | Stability | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Value Type | Description | Example Values | +| --- | --- | --- | --- | --- | --- | +| [`gen_ai.agent.name`](/docs/registry/attributes/gen-ai.md) | ![Development](https://img.shields.io/badge/-development-blue) | `Conditionally Required` when available | string | Human-readable name of the GenAI agent provided by the application. | `Math Tutor`; `Fiction Writer` | +| [`gen_ai.agent.version`](/docs/registry/attributes/gen-ai.md) | ![Development](https://img.shields.io/badge/-development-blue) | `Conditionally Required` when available | string | The version of the GenAI agent. | `1.0.0`; `2025-05-01` | +| [`gen_ai.workflow.name`](/docs/registry/attributes/gen-ai.md) | ![Development](https://img.shields.io/badge/-development-blue) | `Conditionally Required` If available. | string | Human-readable name of the GenAI workflow provided by the application. [1] | `multi_agent_rag`; `customer_support_pipeline` | + +**[1] `gen_ai.workflow.name`:** This attribute can be populated in different frameworks; for example, as the name of the first chain in LangChain or the name of the crew in CrewAI. +The workflow name is usually provided by the application in a way that is specific to the generative AI framework or library that orchestrates the workflow. +It is usually a static name that is expected to be unique within an application. + +`gen_ai.workflow.name` MUST have low cardinality. +Semantic conventions for individual Generative AI frameworks SHOULD document what `gen_ai.workflow.name` means in the context of that framework. +If there is no low-cardinality workflow name available for a given framework, this attribute MUST NOT be captured by default. + + + + + [DocumentStatus]: https://opentelemetry.io/docs/specs/otel/document-status [MetricRequired]: https://github.com/open-telemetry/semantic-conventions/blob/v1.40.0/docs/general/metric-requirement-level.md#required [MetricRecommended]: https://github.com/open-telemetry/semantic-conventions/blob/v1.40.0/docs/general/metric-requirement-level.md#recommended diff --git a/model/gen-ai/metrics.yaml b/model/gen-ai/metrics.yaml index 24b5cb59b..202dbb001 100644 --- a/model/gen-ai/metrics.yaml +++ b/model/gen-ai/metrics.yaml @@ -116,6 +116,40 @@ metrics: - ref: gen_ai.workflow.name requirement_level: conditionally_required: If available. + - name: gen_ai.workflow.steps + annotations: + code_generation: + metric_value_type: int + brief: 'Number of steps in a single GenAI workflow.' + note: | + This metric measures the number of discrete steps observed within a + single GenAI workflow. A "step" is intentionally framework-defined: it + typically corresponds to a single observable event emitted within the + workflow (for example, a tool call, a tool result, or a model response + in ADK; a node execution in LangGraph). + + When the `gen_ai.agent.name` dimension is set, instrumentations SHOULD + only count steps attributed to that agent (for example, events + authored by that agent); events attributed to sub-agents or to a + parent workflow SHOULD NOT be counted. + + This metric complements `gen_ai.workflow.duration` by giving operators + visibility into how "chatty" or compute-intensive a workflow is, and + is useful for capacity planning and regression detection across + versions. + instrument: histogram + unit: "{step}" + stability: development + attributes: + - ref: gen_ai.workflow.name + requirement_level: + conditionally_required: If available. + - ref: gen_ai.agent.name + requirement_level: + conditionally_required: when available + - ref: gen_ai.agent.version + requirement_level: + conditionally_required: when available metric_refinements: - id: openai.client.token.usage diff --git a/schema-snapshot/registry.yaml b/schema-snapshot/registry.yaml index acb1db978..f57048b08 100644 --- a/schema-snapshot/registry.yaml +++ b/schema-snapshot/registry.yaml @@ -2562,6 +2562,75 @@ refinements: path: ./model/gen-ai/metrics.yaml stability: development unit: s + - annotations: + code_generation: + metric_value_type: int + attributes: + - brief: Human-readable name of the GenAI agent provided by the application. + examples: + - Math Tutor + - Fiction Writer + key: gen_ai.agent.name + provenance: + path: ./model/gen-ai/registry.yaml + requirement_level: + conditionally_required: when available + stability: development + type: string + - brief: The version of the GenAI agent. + examples: + - 1.0.0 + - 2025-05-01 + key: gen_ai.agent.version + provenance: + path: ./model/gen-ai/registry.yaml + requirement_level: + conditionally_required: when available + stability: development + type: string + - brief: Human-readable name of the GenAI workflow provided by the application. + examples: + - multi_agent_rag + - customer_support_pipeline + key: gen_ai.workflow.name + note: | + This attribute can be populated in different frameworks; for example, as the name of the first chain in LangChain or the name of the crew in CrewAI. + The workflow name is usually provided by the application in a way that is specific to the generative AI framework or library that orchestrates the workflow. + It is usually a static name that is expected to be unique within an application. + + `gen_ai.workflow.name` MUST have low cardinality. + Semantic conventions for individual Generative AI frameworks SHOULD document what `gen_ai.workflow.name` means in the context of that framework. + If there is no low-cardinality workflow name available for a given framework, this attribute MUST NOT be captured by default. + provenance: + path: ./model/gen-ai/registry.yaml + requirement_level: + conditionally_required: If available. + stability: development + type: string + brief: Number of steps in a single GenAI workflow. + id: gen_ai.workflow.steps + instrument: histogram + name: gen_ai.workflow.steps + note: | + This metric measures the number of discrete steps observed within a + single GenAI workflow. A "step" is intentionally framework-defined: it + typically corresponds to a single observable event emitted within the + workflow (for example, a tool call, a tool result, or a model response + in ADK; a node execution in LangGraph). + + When the `gen_ai.agent.name` dimension is set, instrumentations SHOULD + only count steps attributed to that agent (for example, events + authored by that agent); events attributed to sub-agents or to a + parent workflow SHOULD NOT be counted. + + This metric complements `gen_ai.workflow.duration` by giving operators + visibility into how "chatty" or compute-intensive a workflow is, and + is useful for capacity planning and regression detection across + versions. + provenance: + path: ./model/gen-ai/metrics.yaml + stability: development + unit: \u0027{step}\u0027 - attributes: - brief: | Describes a class of error the operation ended with. @@ -15692,6 +15761,74 @@ registry: path: ./model/gen-ai/metrics.yaml stability: development unit: s + - annotations: + code_generation: + metric_value_type: int + attributes: + - brief: Human-readable name of the GenAI agent provided by the application. + examples: + - Math Tutor + - Fiction Writer + key: gen_ai.agent.name + provenance: + path: ./model/gen-ai/registry.yaml + requirement_level: + conditionally_required: when available + stability: development + type: string + - brief: The version of the GenAI agent. + examples: + - 1.0.0 + - 2025-05-01 + key: gen_ai.agent.version + provenance: + path: ./model/gen-ai/registry.yaml + requirement_level: + conditionally_required: when available + stability: development + type: string + - brief: Human-readable name of the GenAI workflow provided by the application. + examples: + - multi_agent_rag + - customer_support_pipeline + key: gen_ai.workflow.name + note: | + This attribute can be populated in different frameworks; for example, as the name of the first chain in LangChain or the name of the crew in CrewAI. + The workflow name is usually provided by the application in a way that is specific to the generative AI framework or library that orchestrates the workflow. + It is usually a static name that is expected to be unique within an application. + + `gen_ai.workflow.name` MUST have low cardinality. + Semantic conventions for individual Generative AI frameworks SHOULD document what `gen_ai.workflow.name` means in the context of that framework. + If there is no low-cardinality workflow name available for a given framework, this attribute MUST NOT be captured by default. + provenance: + path: ./model/gen-ai/registry.yaml + requirement_level: + conditionally_required: If available. + stability: development + type: string + brief: Number of steps in a single GenAI workflow. + instrument: histogram + name: gen_ai.workflow.steps + note: | + This metric measures the number of discrete steps observed within a + single GenAI workflow. A "step" is intentionally framework-defined: it + typically corresponds to a single observable event emitted within the + workflow (for example, a tool call, a tool result, or a model response + in ADK; a node execution in LangGraph). + + When the `gen_ai.agent.name` dimension is set, instrumentations SHOULD + only count steps attributed to that agent (for example, events + authored by that agent); events attributed to sub-agents or to a + parent workflow SHOULD NOT be counted. + + This metric complements `gen_ai.workflow.duration` by giving operators + visibility into how "chatty" or compute-intensive a workflow is, and + is useful for capacity planning and regression detection across + versions. + provenance: + path: ./model/gen-ai/metrics.yaml + stability: development + unit: \u0027{step}\u0027 - attributes: - brief: | Describes a class of error the operation ended with. From f0a82a3bf9d0b1ddf9ddf96f6706279a487e50d4 Mon Sep 17 00:00:00 2001 From: Pavel Sirotkin <285394121+pvlsirotkin@users.noreply.github.com> Date: Tue, 2 Jun 2026 22:16:45 +0000 Subject: [PATCH 2/4] Address review comments on PR #203 * Expand the `gen_ai.workflow.steps` 'step' definition with a list of what counts as a step in the most common frameworks (ADK events, LangGraph nodes, LangChain AgentAction/AgentFinish, CrewAI agent turns) and add a MUST-document requirement for instrumentations. Addresses trask's feedback that the original definition was too vague. * Standardize the conditionally_required phrasing for all three attributes on this metric (gen_ai.workflow.name, gen_ai.agent.name, gen_ai.agent.version) to 'If available.', matching the adjacent gen_ai.workflow.duration entry. Addresses Copilot's within-metric inconsistency comment. --- docs/gen-ai/gen-ai-metrics.md | 20 ++++++++++++------ model/gen-ai/metrics.yaml | 20 ++++++++++++------ schema-snapshot/registry.yaml | 40 ++++++++++++++++++++++++----------- 3 files changed, 56 insertions(+), 24 deletions(-) diff --git a/docs/gen-ai/gen-ai-metrics.md b/docs/gen-ai/gen-ai-metrics.md index 9f91b2396..eae51ae17 100644 --- a/docs/gen-ai/gen-ai-metrics.md +++ b/docs/gen-ai/gen-ai-metrics.md @@ -963,10 +963,18 @@ This metric SHOULD be specified with [ExplicitBucketBoundaries] of | `gen_ai.workflow.steps` | Histogram | `{step}` | Number of steps in a single GenAI workflow. [1] | ![Development](https://img.shields.io/badge/-development-blue) | | **[1]:** This metric measures the number of discrete steps observed within a -single GenAI workflow. A "step" is intentionally framework-defined: it -typically corresponds to a single observable event emitted within the -workflow (for example, a tool call, a tool result, or a model response -in ADK; a node execution in LangGraph). +single GenAI workflow. A "step" is a framework-defined unit of +execution that corresponds to a single observable event within the +workflow. Common framework-specific definitions: + +- **ADK**: each event in the agent's invocation context (model + response, tool call, tool result, agent transfer). +- **LangGraph**: each node execution in the workflow graph. +- **LangChain agents**: each `AgentAction` or `AgentFinish` step. +- **CrewAI**: each agent turn within a task. + +Instrumentations MUST document what counts as a step for their +framework. When the `gen_ai.agent.name` dimension is set, instrumentations SHOULD only count steps attributed to that agent (for example, events @@ -982,8 +990,8 @@ versions. | Key | Stability | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Value Type | Description | Example Values | | --- | --- | --- | --- | --- | --- | -| [`gen_ai.agent.name`](/docs/registry/attributes/gen-ai.md) | ![Development](https://img.shields.io/badge/-development-blue) | `Conditionally Required` when available | string | Human-readable name of the GenAI agent provided by the application. | `Math Tutor`; `Fiction Writer` | -| [`gen_ai.agent.version`](/docs/registry/attributes/gen-ai.md) | ![Development](https://img.shields.io/badge/-development-blue) | `Conditionally Required` when available | string | The version of the GenAI agent. | `1.0.0`; `2025-05-01` | +| [`gen_ai.agent.name`](/docs/registry/attributes/gen-ai.md) | ![Development](https://img.shields.io/badge/-development-blue) | `Conditionally Required` If available. | string | Human-readable name of the GenAI agent provided by the application. | `Math Tutor`; `Fiction Writer` | +| [`gen_ai.agent.version`](/docs/registry/attributes/gen-ai.md) | ![Development](https://img.shields.io/badge/-development-blue) | `Conditionally Required` If available. | string | The version of the GenAI agent. | `1.0.0`; `2025-05-01` | | [`gen_ai.workflow.name`](/docs/registry/attributes/gen-ai.md) | ![Development](https://img.shields.io/badge/-development-blue) | `Conditionally Required` If available. | string | Human-readable name of the GenAI workflow provided by the application. [1] | `multi_agent_rag`; `customer_support_pipeline` | **[1] `gen_ai.workflow.name`:** This attribute can be populated in different frameworks; for example, as the name of the first chain in LangChain or the name of the crew in CrewAI. diff --git a/model/gen-ai/metrics.yaml b/model/gen-ai/metrics.yaml index 202dbb001..a2e8286d4 100644 --- a/model/gen-ai/metrics.yaml +++ b/model/gen-ai/metrics.yaml @@ -123,10 +123,18 @@ metrics: brief: 'Number of steps in a single GenAI workflow.' note: | This metric measures the number of discrete steps observed within a - single GenAI workflow. A "step" is intentionally framework-defined: it - typically corresponds to a single observable event emitted within the - workflow (for example, a tool call, a tool result, or a model response - in ADK; a node execution in LangGraph). + single GenAI workflow. A "step" is a framework-defined unit of + execution that corresponds to a single observable event within the + workflow. Common framework-specific definitions: + + - **ADK**: each event in the agent's invocation context (model + response, tool call, tool result, agent transfer). + - **LangGraph**: each node execution in the workflow graph. + - **LangChain agents**: each `AgentAction` or `AgentFinish` step. + - **CrewAI**: each agent turn within a task. + + Instrumentations MUST document what counts as a step for their + framework. When the `gen_ai.agent.name` dimension is set, instrumentations SHOULD only count steps attributed to that agent (for example, events @@ -146,10 +154,10 @@ metrics: conditionally_required: If available. - ref: gen_ai.agent.name requirement_level: - conditionally_required: when available + conditionally_required: If available. - ref: gen_ai.agent.version requirement_level: - conditionally_required: when available + conditionally_required: If available. metric_refinements: - id: openai.client.token.usage diff --git a/schema-snapshot/registry.yaml b/schema-snapshot/registry.yaml index f57048b08..bd20ce4a9 100644 --- a/schema-snapshot/registry.yaml +++ b/schema-snapshot/registry.yaml @@ -2574,7 +2574,7 @@ refinements: provenance: path: ./model/gen-ai/registry.yaml requirement_level: - conditionally_required: when available + conditionally_required: If available. stability: development type: string - brief: The version of the GenAI agent. @@ -2585,7 +2585,7 @@ refinements: provenance: path: ./model/gen-ai/registry.yaml requirement_level: - conditionally_required: when available + conditionally_required: If available. stability: development type: string - brief: Human-readable name of the GenAI workflow provided by the application. @@ -2613,10 +2613,18 @@ refinements: name: gen_ai.workflow.steps note: | This metric measures the number of discrete steps observed within a - single GenAI workflow. A "step" is intentionally framework-defined: it - typically corresponds to a single observable event emitted within the - workflow (for example, a tool call, a tool result, or a model response - in ADK; a node execution in LangGraph). + single GenAI workflow. A "step" is a framework-defined unit of + execution that corresponds to a single observable event within the + workflow. Common framework-specific definitions: + + - **ADK**: each event in the agent\u0027s invocation context (model + response, tool call, tool result, agent transfer). + - **LangGraph**: each node execution in the workflow graph. + - **LangChain agents**: each `AgentAction` or `AgentFinish` step. + - **CrewAI**: each agent turn within a task. + + Instrumentations MUST document what counts as a step for their + framework. When the `gen_ai.agent.name` dimension is set, instrumentations SHOULD only count steps attributed to that agent (for example, events @@ -15773,7 +15781,7 @@ registry: provenance: path: ./model/gen-ai/registry.yaml requirement_level: - conditionally_required: when available + conditionally_required: If available. stability: development type: string - brief: The version of the GenAI agent. @@ -15784,7 +15792,7 @@ registry: provenance: path: ./model/gen-ai/registry.yaml requirement_level: - conditionally_required: when available + conditionally_required: If available. stability: development type: string - brief: Human-readable name of the GenAI workflow provided by the application. @@ -15811,10 +15819,18 @@ registry: name: gen_ai.workflow.steps note: | This metric measures the number of discrete steps observed within a - single GenAI workflow. A "step" is intentionally framework-defined: it - typically corresponds to a single observable event emitted within the - workflow (for example, a tool call, a tool result, or a model response - in ADK; a node execution in LangGraph). + single GenAI workflow. A "step" is a framework-defined unit of + execution that corresponds to a single observable event within the + workflow. Common framework-specific definitions: + + - **ADK**: each event in the agent\u0027s invocation context (model + response, tool call, tool result, agent transfer). + - **LangGraph**: each node execution in the workflow graph. + - **LangChain agents**: each `AgentAction` or `AgentFinish` step. + - **CrewAI**: each agent turn within a task. + + Instrumentations MUST document what counts as a step for their + framework. When the `gen_ai.agent.name` dimension is set, instrumentations SHOULD only count steps attributed to that agent (for example, events From f4f0a4cfeb6dbf5f715b44037f262fbb85634120 Mon Sep 17 00:00:00 2001 From: Pavel Sirotkin <285394121+pvlsirotkin@users.noreply.github.com> Date: Wed, 10 Jun 2026 15:17:42 +0000 Subject: [PATCH 3/4] Refocus on per-agent step counting, rename to gen_ai.agent.steps The original RFC and the production ADK implementation always intended this metric to count steps a *specific agent* takes during a single invocation - not steps in some larger workflow. A previous revision of this PR moved the metric into the gen_ai.workflow.* namespace, which caused most of the confusion in the latest review round (multiple reviewers asked whether the metric is a per-invocation count or a cumulative workflow-level aggregation; whether nested sub-agent steps should be counted; whether workflow.name should be required given the name implies workflow scope). This commit refocuses the metric back on its intended scope: * Rename gen_ai.workflow.steps -> gen_ai.agent.steps. Lives in the gen_ai.agent.* namespace alongside gen_ai.agent.invocation.duration (#201). * Drop gen_ai.workflow.name and gen_ai.agent.version from the attribute list. The metric is now dimensioned only by gen_ai.agent.name (at 'recommended' level, matching #201/#202). * Add attributes.gen_ai.error ref_group (Mike's feedback). * Rewrite the metric note to spell out per-agent semantics: - Counts only events the agent itself authored (e.g. in ADK, events whose event.author equals the agent name). - Tool results are NOT counted (authored by tool, not agent). - Sub-agent steps are NOT counted (each step recorded once across the call tree - addresses aabmass' 'recorded exactly once' invariant). - Failed and partial steps still count (they consumed execution time - addresses Mike's question). - Step counts MUST NOT be compared across frameworks (addresses Mike's stronger-wording ask). - Concrete framework-specific definitions for ADK, LangGraph, LangChain agents, CrewAI, with a link to the adk.dev event-loop documentation (aabmass' suggestion). - Histogram bucket recommendation included in the YAML note (Mike's #M3 ask). * Restructure the docs/gen-ai/gen-ai-metrics.md page: remove gen_ai.agent.steps from the 'Generative AI workflow metrics' section and create a new 'Generative AI agent metrics' section. MD body kept thin; the metric note in YAML carries the full semantics and is rendered via the autogen block. --- CHANGELOG.md | 4 +- docs/gen-ai/gen-ai-metrics.md | 107 ++++++------ model/gen-ai/metrics.yaml | 62 +++---- schema-snapshot/registry.yaml | 308 +++++++++++++++++----------------- 4 files changed, 246 insertions(+), 235 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 029268bbe..564a5058c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,8 +31,8 @@ ([#126](https://github.com/open-telemetry/semantic-conventions-genai/pull/126)) - Add `moonshot_ai` to `gen_ai.provider.name` well-known values. ([#99](https://github.com/open-telemetry/semantic-conventions-genai/pull/99)) -- Add `gen_ai.workflow.steps` metric to track the number of discrete steps - observed within a single GenAI workflow. +- Add `gen_ai.agent.steps` metric to track the number of steps a GenAI agent + takes during a single invocation. ([#203](https://github.com/open-telemetry/semantic-conventions-genai/pull/203)) ### 🧰 Bug fixes 🧰 diff --git a/docs/gen-ai/gen-ai-metrics.md b/docs/gen-ai/gen-ai-metrics.md index eae51ae17..b3141ddde 100644 --- a/docs/gen-ai/gen-ai-metrics.md +++ b/docs/gen-ai/gen-ai-metrics.md @@ -19,7 +19,8 @@ linkTitle: Metrics - [Metric: `gen_ai.server.time_to_first_token`](#metric-gen_aiservertime_to_first_token) - [Generative AI workflow metrics](#generative-ai-workflow-metrics) - [Metric: `gen_ai.workflow.duration`](#metric-gen_aiworkflowduration) - - [Metric: `gen_ai.workflow.steps`](#metric-gen_aiworkflowsteps) +- [Generative AI agent metrics](#generative-ai-agent-metrics) + - [Metric: `gen_ai.agent.steps`](#metric-gen_aiagentsteps) @@ -882,12 +883,6 @@ If instrumentation measures only a single provider-facing client operation (for example, one model API call), `gen_ai.client.operation.duration` SHOULD be used instead. Instrumentation MAY emit both metrics for the same request path when both boundaries are available. -`gen_ai.workflow.steps` complements `gen_ai.workflow.duration` by counting the -number of discrete steps observed within a single workflow execution. It is -intended for instrumentations of agent frameworks (for example, ADK, -LangChain agents, CrewAI agents) that can attribute observable events to a -specific workflow or agent. - ### Metric: `gen_ai.workflow.duration` This metric is [required][MetricRequired] when instrumented component implements workflow operations. @@ -938,69 +933,79 @@ If there is no low-cardinality workflow name available for a given framework, th -### Metric: `gen_ai.workflow.steps` +## Generative AI agent metrics -This metric is [recommended][MetricRecommended] for instrumentations that -can reliably attribute steps (events) to a specific workflow or agent. +Individual systems may include additional system-specific attributes. +It is recommended to check system-specific documentation, if available. -A "step" is intentionally framework-defined; it typically corresponds to a -single observable event emitted within the workflow (for example, a tool -call, a tool result, or a model response in ADK; a node execution in -LangGraph). When the `gen_ai.agent.name` dimension is set, instrumentations -SHOULD only count steps attributed to that agent; events attributed to -sub-agents or to a parent workflow SHOULD NOT be counted. +### Metric: `gen_ai.agent.steps` + +This metric is [recommended][MetricRecommended] for instrumentations that +can attribute steps to a specific agent within a single invocation. This metric SHOULD be specified with [ExplicitBucketBoundaries] of [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]. - + | Name | Instrument Type | Unit (UCUM) | Description | Stability | Entity Associations | | -------- | --------------- | ----------- | -------------- | --------- | ------ | -| `gen_ai.workflow.steps` | Histogram | `{step}` | Number of steps in a single GenAI workflow. [1] | ![Development](https://img.shields.io/badge/-development-blue) | | - -**[1]:** This metric measures the number of discrete steps observed within a -single GenAI workflow. A "step" is a framework-defined unit of -execution that corresponds to a single observable event within the -workflow. Common framework-specific definitions: - -- **ADK**: each event in the agent's invocation context (model - response, tool call, tool result, agent transfer). -- **LangGraph**: each node execution in the workflow graph. -- **LangChain agents**: each `AgentAction` or `AgentFinish` step. -- **CrewAI**: each agent turn within a task. - -Instrumentations MUST document what counts as a step for their -framework. - -When the `gen_ai.agent.name` dimension is set, instrumentations SHOULD -only count steps attributed to that agent (for example, events -authored by that agent); events attributed to sub-agents or to a -parent workflow SHOULD NOT be counted. - -This metric complements `gen_ai.workflow.duration` by giving operators -visibility into how "chatty" or compute-intensive a workflow is, and -is useful for capacity planning and regression detection across -versions. +| `gen_ai.agent.steps` | Histogram | `{step}` | Number of steps a GenAI agent takes during a single invocation. [1] | ![Development](https://img.shields.io/badge/-development-blue) | | + +**[1]:** Recorded once per agent invocation. A "step" is a framework-defined +unit of work that the agent itself performs during the invocation; +instrumentations MUST count only steps the agent itself authored. + +Common framework-specific definitions: + +- **ADK**: each [event](https://adk.dev/runtime/event-loop/#event) in + the invocation context whose `event.author` equals the agent name + (typically a model response, a tool call the agent issued, or an + agent transfer the agent initiated). Tool results, which are + authored by the tool rather than the agent, are not counted. +- **LangGraph**: each node execution the agent itself performed in + the workflow graph. +- **LangChain agents**: each `AgentAction` or `AgentFinish` step the + agent itself emitted. +- **CrewAI**: each agent turn the agent itself took within a task. + +Steps emitted by sub-agents or by the surrounding workflow SHOULD +NOT be counted; each step SHOULD be recorded exactly once across +the full call tree. + +Steps are counted regardless of outcome: a step that errored or +partially completed still consumed the agent's execution and SHOULD +be counted. + +Step counts MUST NOT be compared across different frameworks. They +are only meaningful within the context of a single framework's +instrumentation, which is why instrumentations MUST document what +counts as a step for their framework. + +This metric SHOULD be specified with `ExplicitBucketBoundaries` of +[1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]. **Attributes:** | Key | Stability | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Value Type | Description | Example Values | | --- | --- | --- | --- | --- | --- | -| [`gen_ai.agent.name`](/docs/registry/attributes/gen-ai.md) | ![Development](https://img.shields.io/badge/-development-blue) | `Conditionally Required` If available. | string | Human-readable name of the GenAI agent provided by the application. | `Math Tutor`; `Fiction Writer` | -| [`gen_ai.agent.version`](/docs/registry/attributes/gen-ai.md) | ![Development](https://img.shields.io/badge/-development-blue) | `Conditionally Required` If available. | string | The version of the GenAI agent. | `1.0.0`; `2025-05-01` | -| [`gen_ai.workflow.name`](/docs/registry/attributes/gen-ai.md) | ![Development](https://img.shields.io/badge/-development-blue) | `Conditionally Required` If available. | string | Human-readable name of the GenAI workflow provided by the application. [1] | `multi_agent_rag`; `customer_support_pipeline` | +| [`error.type`](https://github.com/open-telemetry/semantic-conventions/blob/v1.41.1/docs/registry/attributes/error.md) | ![Stable](https://img.shields.io/badge/-stable-lightgreen) | `Conditionally Required` if the operation ended in an error | string | Describes a class of error the operation ended with. [1] | `timeout`; `java.net.UnknownHostException`; `server_certificate_invalid`; `500` | +| [`gen_ai.agent.name`](/docs/registry/attributes/gen-ai.md) | ![Development](https://img.shields.io/badge/-development-blue) | `Recommended` | string | Human-readable name of the GenAI agent provided by the application. | `Math Tutor`; `Fiction Writer` | -**[1] `gen_ai.workflow.name`:** This attribute can be populated in different frameworks; for example, as the name of the first chain in LangChain or the name of the crew in CrewAI. -The workflow name is usually provided by the application in a way that is specific to the generative AI framework or library that orchestrates the workflow. -It is usually a static name that is expected to be unique within an application. +**[1] `error.type`:** The `error.type` SHOULD match the error code returned by the Generative AI provider or the client library, +the canonical name of exception that occurred, or another low-cardinality error identifier. +Instrumentations SHOULD document the list of errors they report. -`gen_ai.workflow.name` MUST have low cardinality. -Semantic conventions for individual Generative AI frameworks SHOULD document what `gen_ai.workflow.name` means in the context of that framework. -If there is no low-cardinality workflow name available for a given framework, this attribute MUST NOT be captured by default. +--- + +`error.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used. + +| Value | Description | Stability | +| --- | --- | --- | +| `_OTHER` | A fallback error value to be used when the instrumentation doesn't define a custom value. | ![Stable](https://img.shields.io/badge/-stable-lightgreen) | diff --git a/model/gen-ai/metrics.yaml b/model/gen-ai/metrics.yaml index a2e8286d4..53faa42a8 100644 --- a/model/gen-ai/metrics.yaml +++ b/model/gen-ai/metrics.yaml @@ -116,48 +116,52 @@ metrics: - ref: gen_ai.workflow.name requirement_level: conditionally_required: If available. - - name: gen_ai.workflow.steps + - name: gen_ai.agent.steps annotations: code_generation: metric_value_type: int - brief: 'Number of steps in a single GenAI workflow.' + brief: > + Number of steps a GenAI agent takes during a single invocation. note: | - This metric measures the number of discrete steps observed within a - single GenAI workflow. A "step" is a framework-defined unit of - execution that corresponds to a single observable event within the - workflow. Common framework-specific definitions: + Recorded once per agent invocation. A "step" is a framework-defined + unit of work that the agent itself performs during the invocation; + instrumentations MUST count only steps the agent itself authored. + + Common framework-specific definitions: + + - **ADK**: each [event](https://adk.dev/runtime/event-loop/#event) in + the invocation context whose `event.author` equals the agent name + (typically a model response, a tool call the agent issued, or an + agent transfer the agent initiated). Tool results, which are + authored by the tool rather than the agent, are not counted. + - **LangGraph**: each node execution the agent itself performed in + the workflow graph. + - **LangChain agents**: each `AgentAction` or `AgentFinish` step the + agent itself emitted. + - **CrewAI**: each agent turn the agent itself took within a task. - - **ADK**: each event in the agent's invocation context (model - response, tool call, tool result, agent transfer). - - **LangGraph**: each node execution in the workflow graph. - - **LangChain agents**: each `AgentAction` or `AgentFinish` step. - - **CrewAI**: each agent turn within a task. + Steps emitted by sub-agents or by the surrounding workflow SHOULD + NOT be counted; each step SHOULD be recorded exactly once across + the full call tree. - Instrumentations MUST document what counts as a step for their - framework. + Steps are counted regardless of outcome: a step that errored or + partially completed still consumed the agent's execution and SHOULD + be counted. - When the `gen_ai.agent.name` dimension is set, instrumentations SHOULD - only count steps attributed to that agent (for example, events - authored by that agent); events attributed to sub-agents or to a - parent workflow SHOULD NOT be counted. + Step counts MUST NOT be compared across different frameworks. They + are only meaningful within the context of a single framework's + instrumentation, which is why instrumentations MUST document what + counts as a step for their framework. - This metric complements `gen_ai.workflow.duration` by giving operators - visibility into how "chatty" or compute-intensive a workflow is, and - is useful for capacity planning and regression detection across - versions. + This metric SHOULD be specified with `ExplicitBucketBoundaries` of + [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]. instrument: histogram unit: "{step}" stability: development attributes: - - ref: gen_ai.workflow.name - requirement_level: - conditionally_required: If available. + - ref_group: attributes.gen_ai.error - ref: gen_ai.agent.name - requirement_level: - conditionally_required: If available. - - ref: gen_ai.agent.version - requirement_level: - conditionally_required: If available. + requirement_level: recommended metric_refinements: - id: openai.client.token.usage diff --git a/schema-snapshot/registry.yaml b/schema-snapshot/registry.yaml index bd20ce4a9..2b97a4e19 100644 --- a/schema-snapshot/registry.yaml +++ b/schema-snapshot/registry.yaml @@ -835,6 +835,84 @@ refinements: path: ./model/gen-ai/events.yaml stability: development metrics: + - annotations: + code_generation: + metric_value_type: int + attributes: + - brief: | + Describes a class of error the operation ended with. + examples: + - timeout + - java.net.UnknownHostException + - server_certificate_invalid + - \u0027500\u0027 + key: error.type + note: | + The `error.type` SHOULD match the error code returned by the Generative AI provider or the client library, + the canonical name of exception that occurred, or another low-cardinality error identifier. + Instrumentations SHOULD document the list of errors they report. + requirement_level: + conditionally_required: if the operation ended in an error + stability: stable + type: + members: + - brief: | + A fallback error value to be used when the instrumentation doesn\u0027t define a custom value. + id: other + stability: stable + value: _OTHER + - brief: Human-readable name of the GenAI agent provided by the application. + examples: + - Math Tutor + - Fiction Writer + key: gen_ai.agent.name + provenance: + path: ./model/gen-ai/registry.yaml + requirement_level: recommended + stability: development + type: string + brief: | + Number of steps a GenAI agent takes during a single invocation. + id: gen_ai.agent.steps + instrument: histogram + name: gen_ai.agent.steps + note: | + Recorded once per agent invocation. A "step" is a framework-defined + unit of work that the agent itself performs during the invocation; + instrumentations MUST count only steps the agent itself authored. + + Common framework-specific definitions: + + - **ADK**: each [event](https://adk.dev/runtime/event-loop/#event) in + the invocation context whose `event.author` equals the agent name + (typically a model response, a tool call the agent issued, or an + agent transfer the agent initiated). Tool results, which are + authored by the tool rather than the agent, are not counted. + - **LangGraph**: each node execution the agent itself performed in + the workflow graph. + - **LangChain agents**: each `AgentAction` or `AgentFinish` step the + agent itself emitted. + - **CrewAI**: each agent turn the agent itself took within a task. + + Steps emitted by sub-agents or by the surrounding workflow SHOULD + NOT be counted; each step SHOULD be recorded exactly once across + the full call tree. + + Steps are counted regardless of outcome: a step that errored or + partially completed still consumed the agent\u0027s execution and SHOULD + be counted. + + Step counts MUST NOT be compared across different frameworks. They + are only meaningful within the context of a single framework\u0027s + instrumentation, which is why instrumentations MUST document what + counts as a step for their framework. + + This metric SHOULD be specified with `ExplicitBucketBoundaries` of + [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]. + provenance: + path: ./model/gen-ai/metrics.yaml + stability: development + unit: \u0027{step}\u0027 - annotations: code_generation: metric_value_type: double @@ -2562,83 +2640,6 @@ refinements: path: ./model/gen-ai/metrics.yaml stability: development unit: s - - annotations: - code_generation: - metric_value_type: int - attributes: - - brief: Human-readable name of the GenAI agent provided by the application. - examples: - - Math Tutor - - Fiction Writer - key: gen_ai.agent.name - provenance: - path: ./model/gen-ai/registry.yaml - requirement_level: - conditionally_required: If available. - stability: development - type: string - - brief: The version of the GenAI agent. - examples: - - 1.0.0 - - 2025-05-01 - key: gen_ai.agent.version - provenance: - path: ./model/gen-ai/registry.yaml - requirement_level: - conditionally_required: If available. - stability: development - type: string - - brief: Human-readable name of the GenAI workflow provided by the application. - examples: - - multi_agent_rag - - customer_support_pipeline - key: gen_ai.workflow.name - note: | - This attribute can be populated in different frameworks; for example, as the name of the first chain in LangChain or the name of the crew in CrewAI. - The workflow name is usually provided by the application in a way that is specific to the generative AI framework or library that orchestrates the workflow. - It is usually a static name that is expected to be unique within an application. - - `gen_ai.workflow.name` MUST have low cardinality. - Semantic conventions for individual Generative AI frameworks SHOULD document what `gen_ai.workflow.name` means in the context of that framework. - If there is no low-cardinality workflow name available for a given framework, this attribute MUST NOT be captured by default. - provenance: - path: ./model/gen-ai/registry.yaml - requirement_level: - conditionally_required: If available. - stability: development - type: string - brief: Number of steps in a single GenAI workflow. - id: gen_ai.workflow.steps - instrument: histogram - name: gen_ai.workflow.steps - note: | - This metric measures the number of discrete steps observed within a - single GenAI workflow. A "step" is a framework-defined unit of - execution that corresponds to a single observable event within the - workflow. Common framework-specific definitions: - - - **ADK**: each event in the agent\u0027s invocation context (model - response, tool call, tool result, agent transfer). - - **LangGraph**: each node execution in the workflow graph. - - **LangChain agents**: each `AgentAction` or `AgentFinish` step. - - **CrewAI**: each agent turn within a task. - - Instrumentations MUST document what counts as a step for their - framework. - - When the `gen_ai.agent.name` dimension is set, instrumentations SHOULD - only count steps attributed to that agent (for example, events - authored by that agent); events attributed to sub-agents or to a - parent workflow SHOULD NOT be counted. - - This metric complements `gen_ai.workflow.duration` by giving operators - visibility into how "chatty" or compute-intensive a workflow is, and - is useful for capacity planning and regression detection across - versions. - provenance: - path: ./model/gen-ai/metrics.yaml - stability: development - unit: \u0027{step}\u0027 - attributes: - brief: | Describes a class of error the operation ended with. @@ -14050,6 +14051,83 @@ registry: path: ./model/gen-ai/events.yaml stability: development metrics: + - annotations: + code_generation: + metric_value_type: int + attributes: + - brief: | + Describes a class of error the operation ended with. + examples: + - timeout + - java.net.UnknownHostException + - server_certificate_invalid + - \u0027500\u0027 + key: error.type + note: | + The `error.type` SHOULD match the error code returned by the Generative AI provider or the client library, + the canonical name of exception that occurred, or another low-cardinality error identifier. + Instrumentations SHOULD document the list of errors they report. + requirement_level: + conditionally_required: if the operation ended in an error + stability: stable + type: + members: + - brief: | + A fallback error value to be used when the instrumentation doesn\u0027t define a custom value. + id: other + stability: stable + value: _OTHER + - brief: Human-readable name of the GenAI agent provided by the application. + examples: + - Math Tutor + - Fiction Writer + key: gen_ai.agent.name + provenance: + path: ./model/gen-ai/registry.yaml + requirement_level: recommended + stability: development + type: string + brief: | + Number of steps a GenAI agent takes during a single invocation. + instrument: histogram + name: gen_ai.agent.steps + note: | + Recorded once per agent invocation. A "step" is a framework-defined + unit of work that the agent itself performs during the invocation; + instrumentations MUST count only steps the agent itself authored. + + Common framework-specific definitions: + + - **ADK**: each [event](https://adk.dev/runtime/event-loop/#event) in + the invocation context whose `event.author` equals the agent name + (typically a model response, a tool call the agent issued, or an + agent transfer the agent initiated). Tool results, which are + authored by the tool rather than the agent, are not counted. + - **LangGraph**: each node execution the agent itself performed in + the workflow graph. + - **LangChain agents**: each `AgentAction` or `AgentFinish` step the + agent itself emitted. + - **CrewAI**: each agent turn the agent itself took within a task. + + Steps emitted by sub-agents or by the surrounding workflow SHOULD + NOT be counted; each step SHOULD be recorded exactly once across + the full call tree. + + Steps are counted regardless of outcome: a step that errored or + partially completed still consumed the agent\u0027s execution and SHOULD + be counted. + + Step counts MUST NOT be compared across different frameworks. They + are only meaningful within the context of a single framework\u0027s + instrumentation, which is why instrumentations MUST document what + counts as a step for their framework. + + This metric SHOULD be specified with `ExplicitBucketBoundaries` of + [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]. + provenance: + path: ./model/gen-ai/metrics.yaml + stability: development + unit: \u0027{step}\u0027 - annotations: code_generation: metric_value_type: double @@ -15769,82 +15847,6 @@ registry: path: ./model/gen-ai/metrics.yaml stability: development unit: s - - annotations: - code_generation: - metric_value_type: int - attributes: - - brief: Human-readable name of the GenAI agent provided by the application. - examples: - - Math Tutor - - Fiction Writer - key: gen_ai.agent.name - provenance: - path: ./model/gen-ai/registry.yaml - requirement_level: - conditionally_required: If available. - stability: development - type: string - - brief: The version of the GenAI agent. - examples: - - 1.0.0 - - 2025-05-01 - key: gen_ai.agent.version - provenance: - path: ./model/gen-ai/registry.yaml - requirement_level: - conditionally_required: If available. - stability: development - type: string - - brief: Human-readable name of the GenAI workflow provided by the application. - examples: - - multi_agent_rag - - customer_support_pipeline - key: gen_ai.workflow.name - note: | - This attribute can be populated in different frameworks; for example, as the name of the first chain in LangChain or the name of the crew in CrewAI. - The workflow name is usually provided by the application in a way that is specific to the generative AI framework or library that orchestrates the workflow. - It is usually a static name that is expected to be unique within an application. - - `gen_ai.workflow.name` MUST have low cardinality. - Semantic conventions for individual Generative AI frameworks SHOULD document what `gen_ai.workflow.name` means in the context of that framework. - If there is no low-cardinality workflow name available for a given framework, this attribute MUST NOT be captured by default. - provenance: - path: ./model/gen-ai/registry.yaml - requirement_level: - conditionally_required: If available. - stability: development - type: string - brief: Number of steps in a single GenAI workflow. - instrument: histogram - name: gen_ai.workflow.steps - note: | - This metric measures the number of discrete steps observed within a - single GenAI workflow. A "step" is a framework-defined unit of - execution that corresponds to a single observable event within the - workflow. Common framework-specific definitions: - - - **ADK**: each event in the agent\u0027s invocation context (model - response, tool call, tool result, agent transfer). - - **LangGraph**: each node execution in the workflow graph. - - **LangChain agents**: each `AgentAction` or `AgentFinish` step. - - **CrewAI**: each agent turn within a task. - - Instrumentations MUST document what counts as a step for their - framework. - - When the `gen_ai.agent.name` dimension is set, instrumentations SHOULD - only count steps attributed to that agent (for example, events - authored by that agent); events attributed to sub-agents or to a - parent workflow SHOULD NOT be counted. - - This metric complements `gen_ai.workflow.duration` by giving operators - visibility into how "chatty" or compute-intensive a workflow is, and - is useful for capacity planning and regression detection across - versions. - provenance: - path: ./model/gen-ai/metrics.yaml - stability: development - unit: \u0027{step}\u0027 - attributes: - brief: | Describes a class of error the operation ended with. From 26131bc21d0cc446a4cad6e476aaced8db6d8211 Mon Sep 17 00:00:00 2001 From: Pavel Sirotkin <285394121+pvlsirotkin@users.noreply.github.com> Date: Wed, 10 Jun 2026 15:31:38 +0000 Subject: [PATCH 4/4] Remove duplicate ExplicitBucketBoundaries paragraph The bucket-boundaries recommendation was being rendered in two places: in the MD prose above the autogen block, and inside the YAML `note` (which gets rendered into the autogen block too). Other duration/usage metrics in this file keep ExplicitBucketBoundaries only in the MD prose by convention; aligned with that pattern. Net effect: the recommendation appears once, in the MD prose, like `gen_ai.workflow.duration`, `gen_ai.client.token.usage`, and the other existing metrics. --- docs/gen-ai/gen-ai-metrics.md | 5 +---- model/gen-ai/metrics.yaml | 3 --- schema-snapshot/registry.yaml | 10 ++-------- 3 files changed, 3 insertions(+), 15 deletions(-) diff --git a/docs/gen-ai/gen-ai-metrics.md b/docs/gen-ai/gen-ai-metrics.md index b3141ddde..0b4c29183 100644 --- a/docs/gen-ai/gen-ai-metrics.md +++ b/docs/gen-ai/gen-ai-metrics.md @@ -985,14 +985,11 @@ are only meaningful within the context of a single framework's instrumentation, which is why instrumentations MUST document what counts as a step for their framework. -This metric SHOULD be specified with `ExplicitBucketBoundaries` of -[1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]. - **Attributes:** | Key | Stability | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Value Type | Description | Example Values | | --- | --- | --- | --- | --- | --- | -| [`error.type`](https://github.com/open-telemetry/semantic-conventions/blob/v1.41.1/docs/registry/attributes/error.md) | ![Stable](https://img.shields.io/badge/-stable-lightgreen) | `Conditionally Required` if the operation ended in an error | string | Describes a class of error the operation ended with. [1] | `timeout`; `java.net.UnknownHostException`; `server_certificate_invalid`; `500` | +| [`error.type`](https://github.com/open-telemetry/semantic-conventions/blob/v1.41.1/docs/registry/attributes/error.md) | ![Stable](https://img.shields.io/badge/-stable-lightgreen) | `Conditionally Required` If the operation ended in an error. | string | Describes a class of error the operation ended with. [1] | `timeout`; `java.net.UnknownHostException`; `server_certificate_invalid`; `500` | | [`gen_ai.agent.name`](/docs/registry/attributes/gen-ai.md) | ![Development](https://img.shields.io/badge/-development-blue) | `Recommended` | string | Human-readable name of the GenAI agent provided by the application. | `Math Tutor`; `Fiction Writer` | **[1] `error.type`:** The `error.type` SHOULD match the error code returned by the Generative AI provider or the client library, diff --git a/model/gen-ai/metrics.yaml b/model/gen-ai/metrics.yaml index 53faa42a8..d91c29a14 100644 --- a/model/gen-ai/metrics.yaml +++ b/model/gen-ai/metrics.yaml @@ -152,9 +152,6 @@ metrics: are only meaningful within the context of a single framework's instrumentation, which is why instrumentations MUST document what counts as a step for their framework. - - This metric SHOULD be specified with `ExplicitBucketBoundaries` of - [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]. instrument: histogram unit: "{step}" stability: development diff --git a/schema-snapshot/registry.yaml b/schema-snapshot/registry.yaml index 2b97a4e19..1d0ef04c7 100644 --- a/schema-snapshot/registry.yaml +++ b/schema-snapshot/registry.yaml @@ -852,7 +852,7 @@ refinements: the canonical name of exception that occurred, or another low-cardinality error identifier. Instrumentations SHOULD document the list of errors they report. requirement_level: - conditionally_required: if the operation ended in an error + conditionally_required: If the operation ended in an error. stability: stable type: members: @@ -906,9 +906,6 @@ refinements: are only meaningful within the context of a single framework\u0027s instrumentation, which is why instrumentations MUST document what counts as a step for their framework. - - This metric SHOULD be specified with `ExplicitBucketBoundaries` of - [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]. provenance: path: ./model/gen-ai/metrics.yaml stability: development @@ -14068,7 +14065,7 @@ registry: the canonical name of exception that occurred, or another low-cardinality error identifier. Instrumentations SHOULD document the list of errors they report. requirement_level: - conditionally_required: if the operation ended in an error + conditionally_required: If the operation ended in an error. stability: stable type: members: @@ -14121,9 +14118,6 @@ registry: are only meaningful within the context of a single framework\u0027s instrumentation, which is why instrumentations MUST document what counts as a step for their framework. - - This metric SHOULD be specified with `ExplicitBucketBoundaries` of - [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]. provenance: path: ./model/gen-ai/metrics.yaml stability: development