Skip to content

Commit b1ad8cc

Browse files
GiteaBotbohdekdumontnu
authored
Reduce memory usage for chunked artifact uploads to MinIO (#31325) (#31338)
Backport #31325 by @bohde When using the MinIO storage driver for Actions Artifacts, we found that the chunked artifact required significantly more memory usage to both upload and merge than the local storage driver. This seems to be related to hardcoding a value of `-1` for the size to the MinIO client [which has a warning about memory usage in the respective docs](https://pkg.go.dev/github.com/minio/minio-go/v7#Client.PutObject). Specifying the size in both the upload and merge case reduces memory usage of the MinIO client. Co-authored-by: Rowan Bohde <[email protected]> Co-authored-by: Kyle D <[email protected]>
1 parent 758f84f commit b1ad8cc

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

routers/api/actions/artifacts_chunks.go

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ func saveUploadChunkBase(st storage.ObjectStorage, ctx *ArtifactContext,
3939
r = io.TeeReader(r, hasher)
4040
}
4141
// save chunk to storage
42-
writtenSize, err := st.Save(storagePath, r, -1)
42+
writtenSize, err := st.Save(storagePath, r, contentSize)
4343
if err != nil {
4444
return -1, fmt.Errorf("save chunk to storage error: %v", err)
4545
}
@@ -208,7 +208,7 @@ func mergeChunksForArtifact(ctx *ArtifactContext, chunks []*chunkFileItem, st st
208208

209209
// save merged file
210210
storagePath := fmt.Sprintf("%d/%d/%d.%s", artifact.RunID%255, artifact.ID%255, time.Now().UnixNano(), extension)
211-
written, err := st.Save(storagePath, mergedReader, -1)
211+
written, err := st.Save(storagePath, mergedReader, artifact.FileCompressedSize)
212212
if err != nil {
213213
return fmt.Errorf("save merged file error: %v", err)
214214
}

0 commit comments

Comments
 (0)