Skip to content

CI improvements - #418

Merged
Jay Bosamiya (Microsoft) (jaybosamiya-ms) merged 5 commits into
mainfrom
jayb/ci-improvements
Oct 15, 2025
Merged

Jay Bosamiya (Microsoft) (jaybosamiya-ms) merged 5 commits into
mainfrom
jayb/ci-improvements

Conversation

@jaybosamiya-ms

@jaybosamiya-ms Jay Bosamiya (Microsoft) (jaybosamiya-ms) commented Oct 14, 2025 •

Copy link
Copy Markdown
Member

This PR makes a few small QoL improvements to the CI:

  1. If you push a new commit while there are ongoing checks, it will cancel the old ones. Saves some CI time. Does mean you can get a cancellation notification, but that is at least separate from a failure notification (which at least makes me take a look at why it failed, before I realize "oh, I had pushed a new commit that fixed this already").
  2. We already have caches for build dependencies but these caches explicitly don't store "intermediate" state. Our custom caching (Caching for pre-processing in runner tests #363) that works really well locally gets blasted away on the CI, which means we are spending a whole bunch of time on the CI simply re-doing the same things on every run, making results take longer. I've now updated the CI build to keep track of such artifacts, so that builds are faster. Basically, our builds should be ~6 minutes faster on the CI (the semver-checks and codeql is still the bottleneck, but those ones are less "problematic" in terms of getting quick feedback)
  3. Make the CI assert that Cargo.lock will remain unchanged upon building (i.e., confirm it is up to date). This also updates our Cargo.lock to include a couple of missing lines.
  4. Make the CI resistant to the mild flakiness that shows up sometimes on the LVBS runs by allowing up to two retries. This does not change anything locally (so flakiness is intentionally still apparent locally), but only on the CI where the flakiness is currently folks effectively are simply re-running the CI for that flakiness already.
  5. Set up a hard 10-minute timeout per test; this is to account for tests where something causes them to run too long. Ideally, we'd set this to a smaller number, but at least a 10-minute limit means that we won't have things running until someone manually notices that it has been running too long or some other limits kick in.

@github-actions

Copy link
Copy Markdown

🤖 SemverChecks 🤖 No breaking API changes detected

Note: this does not mean API is unchanged, or even that there are no breaking changes; simply, none of the detections triggered.

@wdcui Weidong Cui (wdcui) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you, Jay!

Merged via the queue into main with commit 1f67a23 Oct 15, 2025
@jaybosamiya-ms
Jay Bosamiya (Microsoft) (jaybosamiya-ms) deleted the jayb/ci-improvements branch October 15, 2025 03:14
Jay Bosamiya (Microsoft) (jaybosamiya-ms) added a commit that referenced this pull request Oct 7, 2026
This PR makes a few small QoL improvements to the CI:
1. If you push a new commit while there are ongoing checks, it will
cancel the old ones. Saves some CI time. Does mean you can get a
cancellation notification, but that is at least separate from a failure
notification (which at least makes me take a look at why it failed,
before I realize "oh, I had pushed a new commit that fixed this
already").
2. We already have caches for build dependencies but these caches
explicitly don't store "intermediate" state. Our custom caching
(#363) that works really well
locally gets blasted away on the CI, which means we are spending a whole
bunch of time on the CI simply re-doing the same things on every run,
making results take longer. I've now updated the CI build to keep track
of such artifacts, so that builds are faster. Basically, our builds
should be ~6 minutes faster on the CI (the semver-checks and codeql is
still the bottleneck, but those ones are less "problematic" in terms of
getting quick feedback)
3. Make the CI assert that `Cargo.lock` will remain unchanged upon
building (i.e., confirm it is up to date). This also updates our
`Cargo.lock` to include a couple of missing lines.
4. Make the CI resistant to the mild flakiness that shows up sometimes
on the LVBS runs by allowing up to two retries. This does not change
anything locally (so flakiness is _intentionally_ still apparent
locally), but only on the CI where the flakiness is currently folks
effectively are simply re-running the CI for that flakiness already.
5. Set up a hard 10-minute timeout per test; this is to account for
tests where something causes them to run too long. Ideally, we'd set
this to a smaller number, but at least a 10-minute limit means that we
won't have things running until someone manually notices that it has
been running too long or some other limits kick in.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants