[test] Fix flaky gateway e2e tests

[Jeffwan]

Pull Request Description

flaky test fix

• TestModelRPSLimit: The Redis limiter uses fixed one-second windows. The test sent requests sequentially, and the first mock response could take long enough for the second request to enter the next window and succeed. The fix sends two short requests concurrently and verifies that one succeeds while the other receives HTTP 429.
• TestPrefixCacheRouting: A novel prefix has no cache affinity and falls back to least-request routing, which randomly breaks ties between equally loaded pods. Therefore, expecting it to select a different pod was probabilistic and flaky. The fix keeps only the deterministic assertion that an identical prompt reuses the warmed pod.

Related Issues

Resolves: #2498

Important: Before submitting, please complete the description above and review the checklist below.

---

Contribution Guidelines (Expand for Details)

We appreciate your contribution to aibrix! To ensure a smooth review process and maintain high code quality, please adhere to the following guidelines:

Pull Request Title Format

Your PR title should start with one of these prefixes to indicate the nature of the change:

• [Bug]: Corrections to existing functionality
• [CI]: Changes to build process or CI pipeline
• [Docs]: Updates or additions to documentation
• [API]: Modifications to aibrix's API or interface
• [CLI]: Changes or additions to the Command Line Interface
• [Misc]: For changes not covered above (use sparingly)

Note: For changes spanning multiple categories, use multiple prefixes in order of importance.

Submission Checklist

• PR title includes appropriate prefix(es)
• Changes are clearly explained in the PR description
• New and existing tests pass successfully
• Code adheres to project style and best practices
• Documentation updated to reflect changes (if applicable)
• Thorough testing completed, no regressions introduced

By submitting this PR, you confirm that you've read these guidelines and your changes align with the project's contribution standards.

[gemini-code-assist]

Code Review

This pull request refactors end-to-end tests to improve reliability. In model_rps_limit_test.go, sequential requests are replaced with concurrent requests to reliably test the 1 RPS limit within the same window, and a helper function is introduced to assert that exactly one request is allowed and one is rejected. In routing_strategy_test.go, the flaky probabilistic test for novel prefix routing is removed. The reviewer suggests reusing a single OpenAI client instance across concurrent requests in sendConcurrentRequests to minimize TCP/TLS handshake overhead and timing jitter, ensuring requests are dispatched as simultaneously as possible.

[Copilot]

Pull request overview

This PR targets flakiness in the gateway E2E test suite by making routing and rate-limit assertions deterministic under timing variance and load-balancing tie-breaking.

Changes:

• Simplifies TestPrefixCacheRouting to only assert deterministic warm-pod reuse (removing probabilistic “novel prefix routes elsewhere” logic).
• Updates TestModelRPSLimit to issue two short requests concurrently to reliably land in the same fixed Redis 1s window and assert one 429.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
test/e2e/routing_strategy_test.go	Removes probabilistic assertions from prefix-cache routing E2E test to avoid tie-break flakiness.
test/e2e/model_rps_limit_test.go	Sends concurrent requests and checks exactly one allowed/one throttled to avoid timing-dependent RPS window rollover.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.