Record various activity metrics

[simvlad]

What changed?

Changes activity metrics

Deprecated Metrics

• activity_e2e_latency → Deprecated, as it measures individual attempts rather than true end-to-end latency, and replaced with clearer activity_start_to_close_latency

New & Replacement Metrics

• activity_start_to_close_latency: Measures the latency from activity start to close (per attempt).
• activity_schedule_to_close_latency: Measures true end-to-end duration, including retries and backoff.
• activity_success: Counts the number of succeeded activities. Aligned with the existing workflow_success counter.
• activity_fail: Counts final failures for activities. Similar to workflow_failed, although it doesn’t include retries.
• activity_timeout: Incremented on the final activity timeout (including ScheduleToStartTimeout), tagged by timeout_type. Aligned with the existing workflow_timeout counter.
• activity_task_fail: Counts failures for activities including retries. Note that we don’t need to capture the number of retries, as this metric represents this number well.
• activity_task_timeout: Incremented on the activity attempt timeout (including ScheduleToStartTimeout), tagged by timeout_type.
• activity_cancel: Incremented when an activity is cancelled ****

Why?

ActivityE2ELatency is inaccurate, since it measures only individual attempts. The other metrics are not recorded yet.

How did you test it?

• built
• run locally and tested manually
• covered by existing tests
• added new unit test(s)
• added new functional test(s)

Ran the server locally and checked the metrics in Prometheus:

Test 1: Activity with 5 attempts (1s each, 1s gap, all time out)

• activity_task_timeout = 5
• activity_timeout = 1
• activity_success = 0
• activity_fail = 0 (counted as timeout instead)
• activity_task_fail = 0 (counted as timeout instead)
• activity_schedule_to_close_latency_sum = 9s
• activity_end_to_end_latency_sum = 1s (deprecated)
• activity_start_to_close_latency_sum = 1s

Test 2: Basic helloworld activity

• activity_success = 1

Test 3: Activity with 5 attempts (1s each, 1s gap, all fail except for the last one)

• activity_task_fail = 4
• activity_success = 1