Skip to content

Commit 5c0fc90

Browse files
authored
fix: Elasticsearch: Request Entity Too Large #28117 (#29062)
Fix for gitea putting everything into one request without batching and sending it to Elasticsearch for indexing as issued in #28117 This issue occured in large repositories while Gitea tries to index the code using ElasticSearch. I've applied necessary changes that takes batch length from below config (app.ini) ``` [queue.code_indexer] BATCH_LENGTH=<length_int> ``` and batches all requests to Elasticsearch in chunks as configured in the above config
1 parent 2bac85d commit 5c0fc90

File tree

1 file changed

+11
-5
lines changed

1 file changed

+11
-5
lines changed

modules/indexer/code/elasticsearch/elasticsearch.go

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -180,11 +180,17 @@ func (b *Indexer) Index(ctx context.Context, repo *repo_model.Repository, sha st
180180
}
181181

182182
if len(reqs) > 0 {
183-
_, err := b.inner.Client.Bulk().
184-
Index(b.inner.VersionedIndexName()).
185-
Add(reqs...).
186-
Do(ctx)
187-
return err
183+
esBatchSize := 50
184+
185+
for i := 0; i < len(reqs); i += esBatchSize {
186+
_, err := b.inner.Client.Bulk().
187+
Index(b.inner.VersionedIndexName()).
188+
Add(reqs[i:min(i+esBatchSize, len(reqs))]...).
189+
Do(ctx)
190+
if err != nil {
191+
return err
192+
}
193+
}
188194
}
189195
return nil
190196
}

0 commit comments

Comments
 (0)