Skip to content

Add async/long-running agent lifecycle events: gen_ai.agent.paused / .resumed / .checkpointed #159

Open

Description

@meshailabs

TL;DR

Propose adding standard span events for long-running async agent lifecycles: gen_ai.agent.paused, .resumed, .checkpointed, plus supporting .started, .completed, .failed. These complement #55's gen_ai.task.status attribute (which captures final state) by capturing timing of mid-execution state transitions — critical for HITL, scheduled, and durable agents.

Filed separately from #55 per @Cirilla-zmh's suggestion that this is beyond #55's scope. Full background in the original gist.

Why it's distinct from gen_ai.task.status

#55's gen_ai.task.status (values: pending / in_progress / completed / failed) is useful as a final-state marker on a task span. It doesn't capture when or why an agent transitioned mid-execution: when it paused for human input, when it persisted a checkpoint, when it resumed.

For short-lived workflow execution, the status attribute is enough. For long-running, human-in-the-loop, scheduled, or durable agents — increasingly common as agents move to async execution patterns (see Zak Knill, "All Your Agents Are Going Async", 2026) — timing matters too. An agent that paused for 3 hours waiting on human approval needs observability on when and why it paused, not just whether it ultimately completed.

Adjacent issues that don't cover this

Proposed events

Span events that long-running agents MAY emit, nested inside a gen_ai.workflow, gen_ai.agent, or gen_ai.task span:

Event Description Stability
gen_ai.agent.started Agent execution has begun; resources loaded Development
gen_ai.agent.paused Agent has paused execution awaiting external input Development
gen_ai.agent.resumed Agent has resumed execution after a pause Development
gen_ai.agent.checkpointed Agent state has been persisted for durability/recovery Development
gen_ai.agent.completed Agent execution ended successfully Development
gen_ai.agent.failed Agent execution ended in failure requiring attention Development

Suggested event attributes

Attribute Description Type
gen_ai.agent.pause.reason Why execution paused (human_input, external_system, rate_limit, scheduled_delay) string
gen_ai.agent.checkpoint.id Identifier for the persisted checkpoint, for recovery correlation string
gen_ai.agent.execution.id Identifier for a single end-to-end execution run (can differ from gen_ai.agent.id when the same agent runs repeatedly) string

Example: nightly reconciliation agent

span: gen_ai.agent invoke_agent reconciliation-agent [4h duration]
  event: gen_ai.agent.started                       (t+0s)
  span: execute_tool fetch_invoices                 (t+5s)
  span: chat claude-sonnet-4-7                      (t+12s)
  event: gen_ai.agent.checkpointed (checkpoint.id=ck_001)  (t+900s)
  event: gen_ai.agent.paused (reason=human_input)   (t+1800s)
  event: gen_ai.agent.resumed                       (t+14000s, 3h later)
  span: execute_tool post_journal_entry             (t+14010s)
  event: gen_ai.agent.completed                     (t+14200s)

Without these events, reconstructing "what was the agent doing from t+900s to t+14200s" requires custom instrumentation.

Open question for the SIG

Should these be events on the existing invoke_agent span (simpler), or should long-running async agents get a new span kind entirely (more honest about the different execution model)?

Implementation status

MeshAI Labs has implemented these as custom attributes in our platform. Happy to:

  1. Refine into a formal spec PR once the SIG indicates direction
  2. Contribute a reference implementation showing the attributes across CrewAI / AutoGen / LangGraph

cc @Cirilla-zmh per #55 redirect

Activity

  1. Rul1an commented on May 18, 2026

    @Rul1an

    This distinction from final gen_ai.task.status makes sense to me. Final state and mid-execution lifecycle transitions answer different questions.

    One thing worth making explicit before the event shape settles: the pause -> checkpoint -> resume transitions need to be correlatable across process and time boundaries, not just visible inside one trace. A span event is attached to whatever span is open when it fires, so the correlation has to live in the attributes the events carry, not in span structure alone.

    For short in-process runs, events on an existing invoke_agent span may be enough. But the cases this proposal targets are the hard ones: human approval after hours, scheduled delay, durable recovery, or a process restart. In those cases the original span may not still be open, and resumed work may land in a new span or even a new trace.

    So I’d treat gen_ai.agent.execution.id as the logical run id, and make the transition edges explicit:

    • checkpointed carries checkpoint.id
    • paused carries a stable pause or interruption id when external resolution is expected
    • resumed carries the checkpoint id or pause id it resumed from

    That gives downstream consumers enough to reconstruct: this execution paused here, persisted this state, and later resumed from that exact boundary.

    Without that edge, resumed is observable but weaker. A consumer can tell that something resumed, but not always which suspended state it continued from.

    This also affects the events-vs-span-kind question. I wouldn’t require one long span to cover the whole async lifecycle. A segment-per-active-execution model with stable lifecycle ids or span links may be more honest for durable agents, while the same event names still work for the simple in-process case.

  2. meshailabs commented on May 18, 2026

    @meshailabs
    Author

    +1 on gen_ai.agent.execution.id as the logical run id — agree the correlation has to live in attributes, not span structure. When we instrumented agentic tool hooks via OpenTelemetry, we hit the same gap. The checkpoint.id / pause.id carry-through on the transition events is the right shape: a consumer can reconstruct the suspended → resumed boundary even when spans aren't continuous.

    The segment-per-execution model belongs in the spec text explicitly. Treating invoke_agent as one continuous span breaks the moment you have a multi-hour pause.

    Happy to draft YAML for the attribute additions if useful.

  3. Keesan12 commented on May 23, 2026

    @Keesan12

    The segment-per-active-execution model feels right for the durable cases this proposal targets. I would make the edge fields required whenever an event crosses a suspend/resume boundary.

    A small invariant could be: a resumed segment MUST identify the suspended boundary it resumes from, either by checkpoint.id, pause.id, or an equivalent lifecycle transition id. Then an observability backend can reconstruct the chain even if the original span is closed, sampled out, or exported in a different trace.

    That also gives implementations a clean test: pause -> checkpoint -> process restart -> resume should produce two active execution segments with a shared logical execution id and an explicit resumed-from edge. Without that, the event names are useful for timelines but weaker for audit and recovery.

  4. Rul1an commented on May 23, 2026

    @Rul1an

    I agree with this invariant. One nuance I’d keep in the wording: the required edge should be a resumed-from relationship, not necessarily a checkpoint-specific relationship.

    For durable recovery, checkpoint.id is the natural edge. For human approval or scheduled delay, the suspended boundary may be a pause.id, interruption.id, or more generic lifecycle transition id. The important rule is that resumed names the boundary it continues from, and the same logical gen_ai.agent.execution.id ties the active segments together.

    That gives the spec a clean conformance test without forcing every implementation to manufacture a checkpoint when it only paused. Something like:

    • if resumed work starts a new active execution segment, gen_ai.agent.resumed MUST identify the prior suspended boundary it resumes from
    • the prior and resumed segments SHOULD share the same logical execution id
    • span links MAY connect the physical spans, but the edge must still be present in attributes because traces may be split, sampled, or exported separately

    That feels like the durable-agent minimum: enough structure to rebuild the chain, without requiring one long span or one storage model.

  5. added
    area:agentSingle agent invocation: identity, configuration, reasoning/planning loop
    on May 28, 2026
  6. meshailabs commented on May 28, 2026

    @meshailabs
    Author

    I've drafted the full semconv model encoding the convergence from this thread: gen_ai.agent.execution.id as the logical run id, plus @Keesan12's conformance invariant (a resumed segment must name the boundary it continues from) generalized per @Rul1an into a gen_ai.agent.resumed_from edge that takes a checkpoint.id, a pause.id, or any equivalent transition id, so human-approval / scheduled-delay pauses that never checkpoint still satisfy the rule.

    The load-bearing decision is the requirement level: resumed_from is conditionally_required only when the resumed work starts a new active execution segment (process restart, durable recovery, different trace), and MAY be omitted when resumption continues in the same still-open span. That keeps the short-lived in-process case zero-overhead while making the durable case auditable.

    Rather than drop the whole model inline, two open questions for the SIG:

    1. Naming: gen_ai.agent.execution.id vs gen_ai.agent.run.id? Want to align with any existing run/execution convention in gen_ai.
    2. resumed_from as a single string ref, or a structured {type, id} pair?

    I can paste the full YAML here or open it as a draft PR, whichever the SIG prefers. cc @Cirilla-zmh

  7. StevenMih commented on Aug 7, 2026

    @StevenMih

    +1 on this shape and on where the thread converged (gen_ai.agent.execution.id + a resumed_from edge — the correlation has to survive process boundaries, since durable interrupts routinely resume in a different worker, or not at all).

    Two additions from implementing exactly this:

    1. paused/resumed assumes every pause eventually resumes. The two other endings are the ones oversight consumers care about most: refused (the human said no) and expired (nobody answered before the deadline). Both are terminal resolutions of the pause, not absences of a resume, and both need first-class representation — a refusal that's indistinguishable from an error, or an expiry indistinguishable from a still-pending pause, is wrong in both directions for anyone consuming this for governance. Concretely: a resolution event (or final state) with approved / refused / expired, linked to the pause it resolves.

    2. An interrupted execution is a dispatch record, not a failure. Whatever status vocabulary eventually lands (here, or in Add Workflows/Agents/Tasks to gen_ai #55's gen_ai.task.status, or Add semantic conventions for A2A (Agent-to-Agent) protocol #70's A2A states — they should agree), interrupted must be distinct from failed: the semantics are "dispatched, awaiting a resolution that arrives elsewhere," and vendor extensions are already shipping interrupted/cancelled values ahead of the spec, so a vocabulary that can't express this gets forked in practice on day one.

    This is framework-neutral now — durable approval mechanics with exactly these outcomes (approve/refuse/expire) have shipped in LangGraph, the OpenAI Agents SDK, Google ADK, and Microsoft Agent Framework within the last year, so the conventions can bind to the common shape rather than any single framework's.

  8. meshailabs commented on Aug 7, 2026

    @meshailabs
    Author

    +1 to both additions. From the oversight-consumer side, refused and expired are not edge cases, they are the outcomes a reviewer of the telemetry most needs to tell apart. The two failure modes you name are real and asymmetric: a refusal recorded as an error corrupts error-rate metrics and understates governance activity, while an expiry indistinguishable from a pending pause hides that the approval channel itself is broken. Those are different problems with different owners, and we compute oversight metrics from exactly this distinction in production.

    The resolution event also closes the state machine this thread converged on: every pause ends in exactly one of resumed, refused, or expired. resumed_from then only ever exists on the approved path, and refused/expired terminate the logical execution at the suspended boundary without starting a new segment. That keeps the segment model intact and gives conformance a clean invariant: a pause with no resolution is pending, full stop.

    Agree the status vocabulary should be settled once and shared with #55 and #70, with interrupted distinct from failed.

  9. Rul1an commented on Aug 7, 2026

    @Rul1an

    +1 on @StevenMih's refused/expired split, and on @meshailabs' framing of the two as asymmetric in what they corrupt. Agreement, nothing to add there.

    On the two open questions.

    2, the resumed_from shape. Carry the type, but as two flat attributes rather than a nested pair: gen_ai.agent.resumed_from.type alongside gen_ai.agent.resumed_from.id.

    Carrying the type at all is the same reason the edge stopped being checkpoint-specific. The suspended boundary can be a durable checkpoint, a pause awaiting human approval, or a scheduled delay, and those are not interchangeable to whoever reads the record later. With a bare id a consumer learns which kind it is holding only by resolving it against the event that emitted it, and this thread's whole premise is that the counterpart may be closed, sampled out, or in a different trace. Requiring resolution to learn the type gives back exactly what the edge was introduced to protect.

    Flat rather than a complex value because the authoring guidance is to represent complex values as a set of flat attributes whenever possible, and to assume backends do not index their individual properties. The eight any-typed attributes in model/gen-ai/registry.yaml today are all bulk content payloads with a JSON schema behind each, which a two-field reference does not need. gen_ai.tool.type sitting next to gen_ai.tool.call.id is already this shape. It also keeps "group resumptions by boundary type" an ordinary query.

    1, naming. I went looking for the convention you wanted to align with and there is no run or execution attribute in the gen_ai registry at all, so nothing constrains the choice. The nearest pattern is <noun>.id: gen_ai.conversation.id, gen_ai.agent.id, gen_ai.memory.store.id. Both candidates fit it. execution.id reads more precisely to me, though only weakly, and matching whatever the SIG settles on matters more than the choice.

    A draft PR would be easier to review than inline YAML, and I am happy to review either. One case I would rather work through against concrete YAML than in the abstract: expired covers the deadline passing, but a producer that dies while holding the pause emits nothing at all, so "no resolution recorded" ends up with more than one cause. Whether that belongs in the conventions or is left to the producer is the part worth arguing against a real requirement level.

  10. StevenMih commented on Aug 9, 2026

    @StevenMih

    @meshailabs , agreed on all of it, and "a pause with no resolution is pending, full stop" is the invariant I'd want in the conventions text verbatim. The asymmetry you name (refusal corrupts error-rate metrics; expiry hides a broken approval channel — different problems, different owners) is the best one-paragraph case for the split I've seen; better than mine.

    @Rul1an , with you on the flat pair (gen_ai.agent.resumed_from.type + .id), for your the reasons given: the counterpart event may be closed, sampled out, or in another trace, and an edge you must resolve to interpret gives back what the edge exists to protect. No strong view on execution.id vs alternatives — matching whatever the SIG settles is worth more than the choice.

    On the producer that dies holding the pause: I'd keep expired strictly as a deadline passed and a resolution event says so — a record of absence — and never let consumers infer it from silence, which is absence of record. They're different facts with different owners: an expiry is a governance outcome; silence is an availability problem, and collapsing them recreates the exact ambiguity the resolution event removes. Concretely: a producer SHOULD emit the expiry resolution when a deadline exists, MUST NOT emit expired without one, and what unresolved silence means stays consumer policy rather than vocabulary. Worth working through against real YAML, agreed.

    On the draft PR: @meshailabs, it's your issue — if you'd like to author it, I'll review same-day. If it's more useful the other way, I'm happy to draft the YAML plus a reference scenario (the three events, the resolution event with approved / refused / expired, and the flat resumed_from pair as discussed) and you both review. Either way this looks ready for concrete YAML.

  11. meshailabs commented on Aug 9, 2026

    @meshailabs
    Author

    Honored to take it, and thank you both for the design work that got this thread to convergence. The draft PR is up: #445.

    It encodes what we settled on: the three lifecycle events, gen_ai.agent.pause.resolved with approved / refused / expired, the flat resumed_from.type + resumed_from.id pair recorded together or not at all, and the deadline rule for expired exactly as stated above (a record of absence: SHOULD with a configured deadline, MUST NOT without one, and what silence means stays consumer policy). The pending invariant sits in the gen_ai.agent.pause.resolution definition itself. A framework-neutral reference scenario models a durable approval gate and exercises all three outcomes, including an approved resume in a new execution segment after the original span closed.

    @StevenMih the same-day review offer is gratefully accepted. @Rul1an your flat-pair reasoning is close to verbatim in the attribute note, so please check I did it justice. Scope notes in the PR: started/completed/failed stay out (the invoke_agent span already expresses them), and the status vocabulary question stays with #55 and #70 as discussed.

  12. meshailabs commented on Aug 13, 2026

    @meshailabs
    Author

    Cross-referencing a downstream consumer that has appeared since the PR went up: OpenSecureAIAlliance/wg-safe#13, part of the SAFE incident-evidence work, cites these lifecycle events as the natural carrier for the approval moment in machine-verifiable approval-to-execution binding. Its four-outcome problem statement (no authority existed, execution stayed in scope, execution left scope, evidence insufficient to tell) depends on being able to join an approval to the pause it resolved, which is exactly the pause.id linkage #445 proposes. Noting it here as concrete external demand for the convention while the PR is in review.

  13. SpaceBum9 commented on Aug 23, 2026

    @SpaceBum9

    Implementation note from MCT-2600027: while drafting a W3C/OpenTelemetry adapter, we hit a distinction that may be useful for this lifecycle proposal.

    A durable checkpoint has at least two identifiers with different semantics:

    1. a checkpoint ID/ref used to correlate pause → persisted state → resume; and
    2. a content digest used to verify the exact canonical checkpoint artifact.

    Neither should be reused as a W3C trace-id or parent-id. In our current implementation the checkpoint digest is 64-hex SHA-256, while W3C trace-id is 32 hex and parent-id is a 16-hex span ID. Beyond the size mismatch, the semantics differ: physical span causality is not artifact lineage.

    The interop shape we plan to test is therefore:

    • gen_ai.agent.execution.id remains stable across active execution segments;
    • a checkpoint event carries a stable checkpoint ID/ref plus an optional full content hash;
    • a resumed segment names the boundary it resumed from;
    • a new physical span may link to the prior span rather than keeping one span open across a long pause;
    • checkpoint existence, signature/integrity, and verification verdict remain separate states—received or hashed does not automatically mean verified.

    A minimal negative fixture would cover a missing resume edge, a changed checkpoint digest, an unknown canonicalization profile, and a checkpoint that exists but has not been externally verified.

    We documented the proposed MCT mapping and non-claims in MCT issue #2. PR #1 contains the deterministic offline checkpoint/evaluation foundation; it is not an OpenTelemetry conformance claim.

    Would the SIG prefer a standard checkpoint content-hash attribute beside the correlation ID, or should integrity metadata remain implementation-specific with only the checkpoint ID standardized?

  14. meshailabs commented on Aug 24, 2026

    @meshailabs
    Author

    Thank you, the two-identifier distinction is worth making explicit.

    On your question: I would standardize the correlation id only, for now. gen_ai.agent.checkpoint.id in #445 names the boundary a resume continued from, and it is deliberately not trace context, so your size and semantics point holds there. A resumed segment links to the prior span rather than reusing either value as trace-id or parent-id.

    A checkpoint content hash reads as a separate proposal rather than an addition to this one. The guidance here is that an attribute needs a signal that carries it and a reference scenario in a library that owns the value, and integrity metadata has neither today. Your three states, existence, signature, and verified verdict, also point to more than one attribute, which is easier to settle on its own thread than folded into this one.

    If a library exposes a canonical digest at its checkpoint API, that would be the natural place to start.

  15. moved this from Post-stability to In Progress in Stable GenAI Semantic Conventionson Aug 28, 2026
  16. moved this from In Progress to Post-stability in Stable GenAI Semantic Conventionson Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:agentSingle agent invocation: identity, configuration, reasoning/planning loop

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions