fix: isolate REST timeout middleware handler context#49982
Conversation
Related to milvus-io#49981 The REST timeout middleware could return a timeout response while the handler goroutine continued using the original Gin context. Once Gin recycled that context, late handler writes could race with a later request and trigger concurrent map writes. Run timeout-wrapped handlers with a copied Gin context and a fully buffered response recorder instead. The original context and real response writer now remain owned by the middleware goroutine, while normal completion explicitly propagates selected metadata and commits the recorder. Timeout closes the recorder and writes the 408 response through the real writer, so late handler writes are discarded safely. Add tests for recorder isolation, normal buffered commits, metadata and abort propagation, late writes after timeout, and race validation of the late-write timeout path. Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: congqixia The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
[ci-v2-notice] To rerun ci-v2 checks, comment with:
If you have any questions or requests, please contact @zhikunyao. |
✅ CI Loop Results
|
| Stage | Result | Duration | Tests |
|---|---|---|---|
| ✅ Build | SUCCESS | 12.9min | - |
| ✅ Code-Check | SUCCESS | 4.8min | - |
| ✅ UT-GO | SUCCESS | 15.1min | 1012 passed |
| ✅ UT-Integration | SUCCESS | 23.7min | 46 passed |
| ✅ UT-CPP-Cov | SUCCESS | 52.4min | 7725 passed |
Total: 69min | Pipeline | Artifacts
Overall Coverage: 67.4%
Diff Coverage: Go 72.1% (101 hit, 39 miss, 140 measurable lines)
Total Patch Coverage: 72.1% (101/140 measurable lines)
|
/lgtm |
Codecov Report❌ Patch coverage is
❌ Your patch status has failed because the patch coverage (79.48%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## master #49982 +/- ##
==========================================
- Coverage 78.55% 78.51% -0.05%
==========================================
Files 2209 2209
Lines 383180 383231 +51
==========================================
- Hits 301012 300891 -121
- Misses 73039 73188 +149
- Partials 9129 9152 +23
🚀 New features to boost your workflow:
|
Cherry-pick from master pr: #49982 Related to #49981 The REST timeout middleware could return a timeout response while the handler goroutine continued using the original Gin context. Once Gin recycled that context, late handler writes could race with a later request and trigger concurrent map writes. Run timeout-wrapped handlers with a copied Gin context and a fully buffered response recorder instead. The original context and real response writer now remain owned by the middleware goroutine, while normal completion explicitly propagates selected metadata and commits the recorder. Timeout closes the recorder and writes the 408 response through the real writer, so late handler writes are discarded safely. Add tests for recorder isolation, normal buffered commits, metadata and abort propagation, late writes after timeout, and race validation of the late-write timeout path. Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
/ci-rerun-e2e-default |
1 similar comment
|
/ci-rerun-e2e-default |
Related to milvus-io#49981 The REST timeout middleware could return a timeout response while the handler goroutine continued using the original Gin context. Once Gin recycled that context, late handler writes could race with a later request and trigger concurrent map writes. Run timeout-wrapped handlers with a copied Gin context and a fully buffered response recorder instead. The original context and real response writer now remain owned by the middleware goroutine, while normal completion explicitly propagates selected metadata and commits the recorder. Timeout closes the recorder and writes the 408 response through the real writer, so late handler writes are discarded safely. Add tests for recorder isolation, normal buffered commits, metadata and abort propagation, late writes after timeout, and race validation of the late-write timeout path. Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
…50006) Cherry-pick from master pr: #49982 Related to #49981 The REST timeout middleware could return a timeout response while the handler goroutine continued using the original Gin context. Once Gin recycled that context, late handler writes could race with a later request and trigger concurrent map writes. Run timeout-wrapped handlers with a copied Gin context and a fully buffered response recorder instead. The original context and real response writer now remain owned by the middleware goroutine, while normal completion explicitly propagates selected metadata and commits the recorder. Timeout closes the recorder and writes the 408 response through the real writer, so late handler writes are discarded safely. Add tests for recorder isolation, normal buffered commits, metadata and abort propagation, late writes after timeout, and race validation of the late-write timeout path. Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Related to #49981
The REST timeout middleware could return a timeout response while the handler goroutine continued using the original Gin context. Once Gin recycled that context, late handler writes could race with a later request and trigger concurrent map writes.
Run timeout-wrapped handlers with a copied Gin context and a fully buffered response recorder instead. The original context and real response writer now remain owned by the middleware goroutine, while normal completion explicitly propagates selected metadata and commits the recorder. Timeout closes the recorder and writes the 408 response through the real writer, so late handler writes are discarded safely.
Add tests for recorder isolation, normal buffered commits, metadata and abort propagation, late writes after timeout, and race validation of the late-write timeout path.