-
Notifications
You must be signed in to change notification settings - Fork 631
Add stateless/stateful mcp backend config #13201
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add stateless/stateful mcp backend config #13201
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds configuration for stateful/stateless session routing for MCP backends. The new sessionRouting field allows users to specify whether requests from a session should be routed to the same backend replica (Stateful) or can be distributed across replicas (Stateless), with Stateful being the default behavior.
- Added
SessionRoutingtype and field toMCPBackendstruct in API - Updated translation logic to convert
sessionRoutingconfiguration tostatefulModein backend output - Updated CRD template to include the new enum field with validation
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| api/v1alpha1/agentgateway/agentgateway_backend_types.go | Defines the new SessionRouting type with Stateful/Stateless constants and adds the sessionRouting field to MCPBackend |
| pkg/kgateway/agentgatewaysyncer/backend/translate.go | Implements translation logic to convert sessionRouting API field to statefulMode backend field with STATEFUL as the default |
| install/helm/agentgateway-crds/templates/agentgateway.dev_agentgatewaybackends.yaml | Adds CRD validation for the new sessionRouting enum field |
| pkg/kgateway/agentgatewaysyncer/backend/testdata/backend/mcp-static-no-protocol.yaml | Updates test to verify Stateless session routing configuration |
| pkg/kgateway/agentgatewaysyncer/backend/testdata/backend/mcp-service-selector-sse.yaml | Updates test to verify Stateless session routing configuration with service selector |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: npolshakova <[email protected]>
02617bc to
ece6094
Compare
howardjohn
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM just nitpick on the docs
| // +required | ||
| Targets []McpTargetSelector `json:"targets"` | ||
|
|
||
| // SessionRouting configures whether to ensure all requests from a session go to the same backend replica |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This description is slightly inaccurate IMO. The primary aspect of Stateful mode is that an mcp-session will be created (via mcp-session-id). In order to make that behave properly we also then pin requests to a specific backend replica, but that is more of the secondary goal of the toggle -- the mcp session is user-facing while the pinning is internal.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay reworded here: b5416ee Let me know what you think!
Signed-off-by: npolshakova <[email protected]>
Description
Adds configuration for stateful/stateless mcp backends
Change Type
Changelog