Skip to content

First part of refactoring the ExtProc used by both the EPP and the BBR#2428

Merged
k8s-ci-robot merged 12 commits intokubernetes-sigs:mainfrom
shmuelk:ext-proc-refactor-1
Mar 2, 2026
Merged

First part of refactoring the ExtProc used by both the EPP and the BBR#2428
k8s-ci-robot merged 12 commits intokubernetes-sigs:mainfrom
shmuelk:ext-proc-refactor-1

Conversation

@shmuelk
Copy link
Copy Markdown
Contributor

@shmuelk shmuelk commented Feb 26, 2026

What type of PR is this?
/kind cleanup

What this PR does / why we need it:
Both the Endpoint Picker (EPP) and the Body Based Router (BBR) are Envoy ExtProc filters. There is much code that has been copy and pasted from one to the other.

This PR is the first in a series of PRs that will refactor out the common Ext Proc processing code into a set of reusable packages. This is being done to try and keep the PRs somewhat smaller.

In particular this PR:
1. Moves the basic header handling code to a common package
2. Moves the request id and default flow id constants to common packages
3. Updates other parts of the code base due to the above changes

Which issue(s) this PR fixes:
Fixes #2427
Ref: #2411

Does this PR introduce a user-facing change?:

NONE

@k8s-ci-robot k8s-ci-robot added the kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. label Feb 26, 2026
@netlify
Copy link
Copy Markdown

netlify Bot commented Feb 26, 2026

Deploy Preview for gateway-api-inference-extension ready!

Name Link
🔨 Latest commit 21b697e
🔍 Latest deploy log https://app.netlify.com/projects/gateway-api-inference-extension/deploys/69a53a42bd517e0008cd926d
😎 Deploy Preview https://deploy-preview-2428--gateway-api-inference-extension.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 26, 2026
@shmuelk
Copy link
Copy Markdown
Contributor Author

shmuelk commented Feb 26, 2026

/test pull-gateway-api-inference-extension-verify-main

@shmuelk shmuelk force-pushed the ext-proc-refactor-1 branch from b490418 to 50215fb Compare March 1, 2026 10:17
limitations under the License.
*/

package request
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: what do you think about putting this under pkg/common/envoy/...?
the functions here are using envoy structs.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

I moved most of the code under pkg/common/requests to pkg/common/envoy/requests. I left behind the definition of the header key, which isn't Envoy related.

Comment thread pkg/common/request/headers.go Outdated
Copy link
Copy Markdown
Contributor

@nirrozenbaum nirrozenbaum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

Thanks @shmuelk, looks great.
I left one nit about the package name, holding in case you want to handle.
not a blocker (you can unblock is you don't agree).

/hold

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 1, 2026
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 1, 2026
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: nirrozenbaum, shmuelk

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Mar 1, 2026
@shmuelk
Copy link
Copy Markdown
Contributor Author

shmuelk commented Mar 2, 2026

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 2, 2026
shmuelk added 11 commits March 2, 2026 09:14
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
@shmuelk shmuelk force-pushed the ext-proc-refactor-1 branch from 69795ff to 4b25c46 Compare March 2, 2026 07:14
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
@nirrozenbaum
Copy link
Copy Markdown
Contributor

Thanks!

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 2, 2026
@k8s-ci-robot k8s-ci-robot merged commit 4c6c187 into kubernetes-sigs:main Mar 2, 2026
11 checks passed
RyanRosario pushed a commit to RyanRosario/gateway-api-inference-extension that referenced this pull request Mar 9, 2026
kubernetes-sigs#2428)

* Moved base header helpers to pkg/common

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* BBR to use requestIdKey from pkg/common

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Moved default flow ID to  pkg/metadata

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Updates due to move of requestId header key

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Fixed copyright text

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Increased timeout, times out if Git is slow

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Reverted change to ready timeout

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Increased metrics curl timeout for slower systems

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Updated new test with updated header key location

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Moved envoy related code under new common/envoy package

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Changes due to package refactoring

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Changes due to package refactoring

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

---------

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
elevran pushed a commit to llm-d/llm-d-inference-scheduler that referenced this pull request Apr 23, 2026
kubernetes-sigs/gateway-api-inference-extension#2428)

* Moved base header helpers to pkg/common

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* BBR to use requestIdKey from pkg/common

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Moved default flow ID to  pkg/metadata

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Updates due to move of requestId header key

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Fixed copyright text

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Increased timeout, times out if Git is slow

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Reverted change to ready timeout

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Increased metrics curl timeout for slower systems

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Updated new test with updated header key location

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Moved envoy related code under new common/envoy package

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Changes due to package refactoring

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Changes due to package refactoring

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

---------

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor the Ext Proc code used by both the EPP and the BBR

3 participants