Skip to content

Race condition when running ES delete_by_query #32184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bsofiato opened this issue Oct 3, 2024 · 0 comments · Fixed by #32185
Closed

Race condition when running ES delete_by_query #32184

bsofiato opened this issue Oct 3, 2024 · 0 comments · Fixed by #32185
Labels
Milestone

Comments

@bsofiato
Copy link
Contributor

bsofiato commented Oct 3, 2024

Description

When running the unit test associated with code search (file modules/indexer/code/indexer_test.go), I ran into the issue in the screenshot below.

image

As explained here, this is due to a racing condition where ES uses an outdated index while looking for the documents to delete. The safe approach is to ask ES to refresh the index before.

I wrote a PR that changes the behavior when trying to delete the documents of a repo. If the deletion results in a conflict, Gitea will refresh the index and retry.

P.S. ES is running inside a docker container spun by the command below:

docker run --rm --memory="4g" -p 127.0.0.1:9200:9200 -p 127.0.0.1:9300:9300 -e "discovery.type=single-node" -v "$(pwd)/data/elasticsearch:/usr/share/elasticsearch/data" docker.elastic.co/elasticsearch/elasticsearch:7.16.3

Gitea Version

0b1b403

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

No response

Operating System

No response

How are you running Gitea?

I ran into this issue while running the tests inside VSCode

Database

None

@lunny lunny added this to the 1.22.3 milestone Oct 3, 2024
techknowlogick pushed a commit that referenced this issue Oct 3, 2024
GiteaBot pushed a commit to GiteaBot/gitea that referenced this issue Oct 3, 2024
lunny pushed a commit that referenced this issue Oct 3, 2024
…ch (#32185) (#32188)

Backport #32185 by @bsofiato

Resolves #32184

Signed-off-by: Bruno Sofiato <[email protected]>
Co-authored-by: Bruno Sofiato <[email protected]>
@go-gitea go-gitea locked as resolved and limited conversation to collaborators Jan 1, 2025
project-mirrors-bot-tu bot pushed a commit to project-mirrors/forgejo-as-gitea-fork that referenced this issue Jan 23, 2025
…ch (go-gitea#32185)

Resolves go-gitea#32184

---------

Signed-off-by: Bruno Sofiato <[email protected]>
(cherry picked from commit d266d19)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants