Skip to content

Reduce integration test overhead #32475

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Nov 14, 2024

Conversation

bohde
Copy link
Contributor

@bohde bohde commented Nov 11, 2024

In profiling integration tests, I found a couple places where per-test overhead could be reduced:

  • Avoiding disk IO by synchronizing instead of deleting & copying test Git repository data. This saves ~100ms per test on my machine
  • When flushing queues in PrintCurrentTest, invoke FlushWithContext in a parallel.

Reduce overhead of preparing the test Git repos by avoiding
unnecessary copying of already existing files.
For non-immediate queues, FlushWithContext takes at least 40ms to
verify that all items are processed. Calling this in parallel allows
for just a single 40ms delay, instead of 40ms per queue.
@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Nov 11, 2024
@github-actions github-actions bot added modifies/go Pull requests that update Go code modifies/migrations labels Nov 11, 2024
@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Nov 11, 2024
@wxiaoguang
Copy link
Contributor

When flushing queues in PrintCurrentTest, invoke FlushWithContext in a parallel.

We could make it even faster (without any waiting time). Could I edit your PR to implement my idea?

@bohde
Copy link
Contributor Author

bohde commented Nov 13, 2024

We could make it even faster (without any waiting time). Could I edit your PR to implement my idea?

I'd be fine with this!

@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Nov 14, 2024
@wxiaoguang wxiaoguang force-pushed the rb/reduce-integration-test-overhead branch from 60c0327 to ace4671 Compare November 14, 2024 05:35
@wxiaoguang wxiaoguang force-pushed the rb/reduce-integration-test-overhead branch from ffb110b to 527c049 Compare November 14, 2024 05:57
@wxiaoguang
Copy link
Contributor

Done from my side.

@bohde
Copy link
Contributor Author

bohde commented Nov 14, 2024

I think these changes look good!

@lunny lunny added the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Nov 14, 2024
@lunny lunny enabled auto-merge (squash) November 14, 2024 18:49
@lunny lunny merged commit 68731c0 into go-gitea:main Nov 14, 2024
26 checks passed
@GiteaBot GiteaBot added this to the 1.24.0 milestone Nov 14, 2024
@GiteaBot GiteaBot removed the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Nov 14, 2024
zjjhot added a commit to zjjhot/gitea that referenced this pull request Nov 15, 2024
* giteaofficial/main:
  Fix oauth2 error handle not return immediately (go-gitea#32514)
  Fix incorrect project page CSS class (go-gitea#32510)
  Add avif image file support (go-gitea#32508)
  Reduce integration test overhead (go-gitea#32475)
  Remove jQuery import from some files (go-gitea#32512)
  Trim title before insert/update to database to match the size requirements of database (go-gitea#32498)
  Reimplement GetUserOrgsList to make it simple and clear (go-gitea#32486)
@lunny lunny modified the milestones: 1.24.0, 1.23.0 Nov 15, 2024
@wxiaoguang
Copy link
Contributor

Need a following up PR: Improve testing and try to fix MySQL hanging #32515

@go-gitea go-gitea locked as resolved and limited conversation to collaborators Feb 12, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. modifies/go Pull requests that update Go code modifies/migrations type/testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants