Skip to content

[SDCI-1360] Add critical path and exec time feature support #28922

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,6 @@ The exclusive time of a job on the critical path represents the amount of time t

If a CI job `job1` is on the critical path with a duration of 100ms and runs in parallel with a CI job `job2`, which has a duration of 80ms, the exclusive time of `job1` on the critical path is 20ms. This means that reducing the duration of the `job1` by more than 20ms would still only decrease the overall pipeline duration by 20ms.

### Supported CI providers

Filtering and computing the Exclusive Time of CI jobs on the critical path is available for the following CI providers:
* [GitLab][3]

<div class="alert alert-info">If you are interested in critical path but your CI provider is not supported yet, fill out <a href="https://forms.gle/zDgiDSGQxA1HgjXh6" target="_blank">this form</a>.</div>

Even without Filtering and computing Exclusive Time, you can still [highlight which CI jobs are on the critical path][4] using the Pipeline Execution detail view across all CI providers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we now support every CI provider I thought it was better to just remove this instead of saying "we support all CI providers"

## Identify the key CI jobs to improve your CI pipeline

### Using the facet
Expand All @@ -50,7 +41,7 @@ Notice that these facets are only available using the `ci_level:job` in your que
### Using the dashboard template

You can also import the [CI Visibility - Critical Path][1] dashboard template:
- Open the [civisibility-critical-path-gitlab-dashboard.json][1] dashboard template and copy the content in the clipboard.
- Open the [civisibility-critical-path-dashboard.json][1] dashboard template and copy the content in the clipboard.
- Create a [New Dashboard][2] in Datadog.
- Paste the copied content in the new dashboard.
- Save the dashboard.
Expand All @@ -77,7 +68,7 @@ Since we know this CI job is on the critical path 43.5% of the time, we could po

{{< partial name="whats-next/whats-next.html" >}}

[1]: /resources/json/civisibility-critical-path-gitlab-dashboard.json
[1]: /resources/json/civisibility-critical-path-dashboard.json
[2]: /dashboards/
[3]: /continuous_integration/pipelines/gitlab/?tab=gitlabcom
[4]: /continuous_integration/search/#highlight-critical-path
4 changes: 2 additions & 2 deletions content/en/continuous_integration/pipelines/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,13 +145,13 @@ If your CI provider is not supported, you can try setting up Pipeline Visibility
| {{< ci-details title="Pipeline trace visualization" >}}Visualization of pipeline executions with associated tracing.{{< /ci-details >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} |
| {{< ci-details title="Job failure analysis" >}}Analysis and categorization of job failures using LLM models based on relevant logs. <a href="https://docs.datadoghq.com/continuous_integration/guides/use_ci_jobs_failure_analysis/">More info</a>.{{< /ci-details >}} | | {{< X >}} | | | {{< X >}} | | | | | |
| {{< ci-details title="Running pipelines" >}}Identification of pipelines executions that are running with associated tracing.{{< /ci-details >}} | {{< X >}} | {{< X >}} | | | {{< X >}} | | | | {{< X >}} | {{< X >}} |
| {{< ci-details title="Filter CI Jobs on the critical path" >}}Identification of CI jobs that are on the critical path of the pipeline.{{< /ci-details >}} | | {{< X >}} | | | | | | | | |
| {{< ci-details title="Filter CI Jobs on the critical path" >}}Identification of CI jobs that are on the critical path of the pipeline.{{< /ci-details >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} |
| {{< ci-details title="Partial retries" >}}Identification of partial retries (for example, when only a subset of jobs were retried).{{< /ci-details >}} | | {{< X >}} | | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} |
| {{< ci-details title="Step spans" >}}Step level spans are available for more granular visibility.{{< /ci-details >}} | {{< X >}} (_But are presented as job spans_) | | | | {{< X >}} | | {{< X >}} | | | {{< X >}} |
| {{< ci-details title="Manual steps" >}}Identification of when there is a job with a manual approval phase in the overall pipeline.{{< /ci-details >}} | {{< X >}} | {{< X >}} | | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | | {{< X >}} | {{< X >}} |
| {{< ci-details title="Approval wait time">}}Time for which a pipeline or job has been waiting for a manual approval.{{< /ci-details >}} | | {{< X >}} | | | {{< X >}} | {{< X >}} | | | {{< X >}} | |
| {{< ci-details title="Queue time" >}}Time for which a pipeline or job was in the queue before execution.{{< /ci-details >}} | {{< X >}} | {{< X >}} | | {{< X >}} | {{< X >}} | | {{< X >}} | {{< X >}} | | {{< X >}} |
| {{< ci-details title="Execution time" >}}Time for which a pipeline has been actively running jobs.{{< /ci-details >}} | | {{< X >}} | | | | | | | | |
| {{< ci-details title="Execution time" >}}Time for which a pipeline has been actively running jobs.{{< /ci-details >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} |
| {{< ci-details title="Logs correlation" >}}Retrieval of pipeline or job logs from the CI provider. Logs are displayed on the <strong>Logs</strong> tab in the Pipeline Execution view.{{< /ci-details >}} | {{< X >}} | {{< X >}} | {{< X >}} | | {{< X >}} | {{< X >}} | | | {{< X >}} | |
| {{< ci-details title="Infrastructure metric correlation" >}}Correlation of host-level information for the Datadog Agent, CI pipelines, or job runners to CI pipeline execution data.{{< /ci-details >}} | {{< X >}} | {{< X >}} | | {{< X >}} | {{< X >}} | | | | | |
| {{< ci-details title="Custom spans for traced commands using datadog-ci" >}}Support for sending command-level events to CI Visibility to be incorporated into pipeline flame graph visualization. You can then query and analyze <a href="https://docs.datadoghq.com/continuous_integration/pipelines/custom_commands/">these events</a>. {{< /ci-details >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | | | {{< X >}} | |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ Set up tracing on AWS CodePipeline to collect data about pipeline executions, an
| **Logs correlation | Logs correlation | Correlate pipeline and job spans to logs and enable [job log correlation](#collect-job-logs). |
| [Approval wait time][17] | Approval wait time | View the amount of time jobs and pipelines wait for manual approvals. |
| [Custom spans][18] | Custom spans | Configure custom spans for your pipelines. |
| [Filter CI Jobs on the critical path][19] | Filter CI Jobs on the critical path | Filter by jobs on the critical path. |
| [Execution time][20] | Execution time | View the amount of time pipelines have been running jobs. |

*AWS CodePipeline running pipelines don't have Git information until they have finished.\
**AWS CodePipeline logs correlation is only available for AWS CodeBuild actions.
Expand Down Expand Up @@ -160,3 +162,5 @@ The **CI Pipeline List** page shows data for only the default branch of each rep
[16]: /logs/guide/send-aws-services-logs-with-the-datadog-lambda-function
[17]: /glossary/#approval-wait-time
[18]: /glossary/#custom-span
[19]: /continuous_integration/guides/identify_highest_impact_jobs_with_critical_path/
[20]: /glossary/#pipeline-execution-time
4 changes: 4 additions & 0 deletions content/en/continuous_integration/pipelines/azure.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ Set up tracing on Azure Pipelines to gain real time insights into your CI/CD wor
|---|---|---|
| [Custom tags][10] [and measures at runtime][11] | Custom tags and measures at runtime | Configure [custom tags and measures][6] at runtime. |
| [Custom spans][15] | Custom spans | Configure custom spans for your pipelines. |
| [Filter CI Jobs on the critical path][19] | Filter CI Jobs on the critical path | Filter by jobs on the critical path. |
| [Execution time][20] | Execution time | View the amount of time pipelines have been running jobs. |

## Configure the Datadog integration

Expand Down Expand Up @@ -157,3 +159,5 @@ The **CI Pipeline List** page shows data for only the default branch of each rep
[16]: /integrations/azure_devops/
[17]: /continuous_integration/search/#search-for-pipelines
[18]: /logs/guide/best-practices-for-log-management/
[19]: /continuous_integration/guides/identify_highest_impact_jobs_with_critical_path/
[20]: /glossary/#pipeline-execution-time
8 changes: 6 additions & 2 deletions content/en/continuous_integration/pipelines/buildkite.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ Set up tracing on Buildkite to optimize your resource usage, reduce overhead, an
| [Queue time][13] | Queue time | View the amount of time pipeline jobs sit in the queue before processing. |
| [Custom tags][10] [and measures at runtime][11] | Custom tags and measures at runtime | Configure [custom tags and measures][6] at runtime. |
| [Custom spans][14] | Custom spans | Configure custom spans for your pipelines. |
| [Filter CI Jobs on the critical path][17] | Filter CI Jobs on the critical path | Filter by jobs on the critical path. |
| [Execution time][18] | Execution time | View the amount of time pipelines have been running jobs. |

## Configure the Datadog integration

Expand Down Expand Up @@ -78,9 +80,9 @@ The resulting pipeline looks like the following:
{{< img src="ci/buildkite-custom-tags.png" alt="Buildkite pipeline trace with custom tags" style="width:100%;">}}

Any metadata with a key starting with `dd-measures.` and containing a numerical value will be set as
a metric tag that can be used to create numerical measures.
a metric tag that can be used to create numerical measures.

You can use the `buildkite-agent meta-data set` command to create these tags.
You can use the `buildkite-agent meta-data set` command to create these tags.

For example, you can measure the binary size in a pipeline with this command:

Expand Down Expand Up @@ -145,3 +147,5 @@ The **CI Pipeline List** page shows data for only the default branch of each rep
[14]: /glossary/#custom-span
[15]: /continuous_integration/explorer
[16]: /continuous_integration/search/#search-for-pipelines
[17]: /continuous_integration/guides/identify_highest_impact_jobs_with_critical_path/
[18]: /glossary/#pipeline-execution-time
4 changes: 4 additions & 0 deletions content/en/continuous_integration/pipelines/circleci.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ Set up tracing on CircleCI to optimize the performance of your pipelines, improv
| [Custom spans][13] | Custom spans | Configure custom spans for your pipelines. |
| Custom pre-defined tags | Custom pre-defined tags | Set [custom tags][6] to all generated pipeline and job spans. |
| [Custom tags][14] [and measures at runtime][15] | Custom tags and measures at runtime | Configure [custom tags and measures][7] at runtime. |
| [Filter CI Jobs on the critical path][18] | Filter CI Jobs on the critical path | Filter by jobs on the critical path. |
| [Execution time][19] | Execution time | View the amount of time pipelines have been running jobs. |

## Configure the Datadog integration

Expand Down Expand Up @@ -128,3 +130,5 @@ The **CI Pipeline List** page shows data for only the default branch of each rep
[15]: /glossary/#custom-measure
[16]: /logs/guide/best-practices-for-log-management/
[17]: /continuous_integration/search/#search-for-pipelines
[18]: /continuous_integration/guides/identify_highest_impact_jobs_with_critical_path/
[19]: /glossary/#pipeline-execution-time
8 changes: 6 additions & 2 deletions content/en/continuous_integration/pipelines/codefresh.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ further_reading:

## Overview

[Codefresh][1] is a continuous integration and delivery platform built for Kubernetes which offers automation features that streamline the building, testing, and deploying of your applications.
[Codefresh][1] is a continuous integration and delivery platform built for Kubernetes which offers automation features that streamline the building, testing, and deploying of your applications.

Set up tracing in Codefresh to collect data on each step of your pipelines, analyze performance bottlenecks, troubleshoot operational challenges, and monitor your deployment workflows.

Expand All @@ -29,6 +29,8 @@ Set up tracing in Codefresh to collect data on each step of your pipelines, anal
| [Manual steps][8] | Manual steps | View manually triggered pipelines. |
| [Parameters][9] | Parameters | Set custom parameters (for example, [Codefresh variables][6]) when a pipeline is triggered. |
| [Pipeline failure reasons][10] | Pipeline failure reasons | Identify pipeline failure reasons from error messages. |
| [Filter CI Jobs on the critical path][12] | Filter CI Jobs on the critical path | Filter by jobs on the critical path. |
| [Execution time][13] | Execution time | View the amount of time pipelines have been running jobs. |

## Configure the Datadog integration

Expand Down Expand Up @@ -61,4 +63,6 @@ The **CI Pipeline List** page shows data for only the default branch of each rep
[8]: /glossary/#manual-step
[9]: /glossary/#parameter
[10]: /glossary/#pipeline-failure
[11]: /continuous_integration/search/#search-for-pipelines
[11]: /continuous_integration/search/#search-for-pipelines
[12]: /continuous_integration/guides/identify_highest_impact_jobs_with_critical_path/
[13]: /glossary/#pipeline-execution-time
5 changes: 5 additions & 0 deletions content/en/continuous_integration/pipelines/custom.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,9 @@ You can send custom pipelines through HTTP using the [public API endpoint][1]. F
| [Partial retries][10] | Partial pipelines | View partially retried pipeline executions. |
| [Pipeline failure reasons][11] | Pipeline failure reasons | Identify pipeline failure reasons from error messages. |
| [Queue time][12] | Queue time | View the amount of time pipeline jobs sit in the queue before processing. |
| [Filter CI Jobs on the critical path][16] | Filter CI Jobs on the critical path | Filter by jobs on the critical path. |
| [Execution time][17] | Execution time | View the amount of time pipelines have been running jobs. |


## Configure CI Visibility

Expand Down Expand Up @@ -128,3 +131,5 @@ The **CI Pipeline List** page shows data for only the default branch of each rep
[13]: /continuous_integration/search/#search-for-pipelines
[14]: https://app.datadoghq.com/organization-settings/api-keys
[15]: /glossary/#running-pipeline
[16]: /continuous_integration/guides/identify_highest_impact_jobs_with_critical_path/
[17]: /glossary/#pipeline-execution-time
4 changes: 4 additions & 0 deletions content/en/continuous_integration/pipelines/github.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ Set up CI Visibility in GitHub Actions to track the execution of your workflows,
| [Queue time][8] | Queue time | View the amount of time pipeline jobs sit in the queue before processing. |
| [Approval wait time][9] | Approval wait time | View the amount of time workflow runs and workflow jobs wait for manual approvals. |
| [Custom spans][10] | Custom spans | Configure custom spans for your pipelines. |
| [Filter CI Jobs on the critical path][24] | Filter CI Jobs on the critical path | Filter by jobs on the critical path. |
| [Execution time][25] | Execution time | View the amount of time pipelines have been running jobs. |


The following GitHub versions are supported:
Expand Down Expand Up @@ -145,3 +147,5 @@ The **CI Pipeline List** page shows data for only the default branch of each rep
[21]: https://github.com/actions/actions-runner-controller
[22]: https://github.com/DataDog/datadog-agent/blob/main/pkg/config/config_template.yaml
[23]: /continuous_integration/guides/use_ci_jobs_failure_analysis/
[24]: /continuous_integration/guides/identify_highest_impact_jobs_with_critical_path/
[25]: /glossary/#pipeline-execution-time
4 changes: 3 additions & 1 deletion content/en/continuous_integration/pipelines/gitlab.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ Set up tracing in GitLab to collect data on your pipeline executions, analyze pe
|---|---|---|
| [Running pipelines][24] | Running pipelines | View pipeline executions that are running. Queued or waiting pipelines show with status "Running" on Datadog. |
| [CI jobs failure analysis][28] | CI jobs failure analysis | Analysis of the root causes of failed CI jobs based on relevant logs using LLM models. |
| [Filter CI Jobs on the critical path][29] | Filter CI Jobs on the critical path | Filter by jobs on the critical path. |
| [Partial retries][19] | Partial pipelines | View partially retried pipeline executions. |
| [Manual steps][20] | Manual steps | View manually triggered pipelines. |
| [Queue time][21] | Queue time | View the amount of time pipeline jobs sit in the queue before processing. |
Expand Down Expand Up @@ -449,4 +450,5 @@ The **CI Pipeline List** page shows data for only the default branch of each rep
[25]: /glossary/#custom-span
[26]: /continuous_integration/explorer
[27]: /continuous_integration/search/#search-for-pipelines
[28]: /continuous_integration/guides/use_ci_jobs_failure_analysis/
[28]: /continuous_integration/guides/use_ci_jobs_failure_analysis/
[29]: /continuous_integration/guides/identify_highest_impact_jobs_with_critical_path/
4 changes: 4 additions & 0 deletions content/en/continuous_integration/pipelines/jenkins.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ Set up tracing in Jenkins to collect data across various stages of your pipeline
| [Parameters][24] | Parameters | Set custom parameters (such as the default branch name or Git information) when a pipeline is triggered. |
| [Pipeline failure reasons][25] | Pipeline failure reasons | Identify pipeline failure reasons from error messages. |
| [Running pipelines][32] | Running pipelines | View pipeline executions that are running. Requires Jenkins plugin version >= 8.0.0 |
| [Filter CI Jobs on the critical path][33] | Filter CI Jobs on the critical path | Filter by jobs on the critical path. |
| [Execution time][34] | Execution time | View the amount of time pipelines have been running jobs. |

The following Jenkins versions are supported:

Expand Down Expand Up @@ -1256,3 +1258,5 @@ try restarting the Jenkins instance.
[30]: /developers/dogstatsd/
[31]: /containers/docker/apm/#tracing-from-the-host
[32]: /glossary/#running-pipeline
[33]: /continuous_integration/guides/identify_highest_impact_jobs_with_critical_path/
[34]: /glossary/#pipeline-execution-time
Loading
Loading