fix: insert AI Gateway extproc after the buffer filter

[ChrisJBurns]

Description

Problem

If a buffer filter (envoy.filters.http.buffer) is added to the chain, for example via an EnvoyPatchPolicy to raise the request buffer limit, the AI Gateway extproc could be placed ahead of it. The AI Gateway extproc runs with RequestBodyMode: BUFFERED, so when it runs before the buffer filter it buffers the body against Envoy Gateway's default 32 KiB downstream per-connection limit and returns 413 Payload Too Large for larger bodies before the buffer filter's higher limit applies.

The buffer filter raises the stream's buffer limit to max_request_bytes, but only for filters that decode after it. This was hit in practice with large request bodies such as Gemini CLI tool definitions.

Solution

When a buffer filter sits at or after the computed ext_proc insertion point, insert the AI Gateway extproc immediately after the last buffer filter. Envoy Gateway already orders the buffer filter before ext_proc, so this honors that order. Behavior is unchanged when no buffer filter is present.

This works because EnvoyPatchPolicy patches are applied before the extension hook runs and Envoy Gateway does not re-sort filters afterward, so the buffer filter is visible to this code.

The extension server only moves its own filter, not the user's buffer filter, so this fixes the 413 for AI Gateway's extproc specifically and not for other co-resident BUFFERED ext_procs.

Testing

Added table tests for three cases: ext_proc before buffer (inserted after buffer), buffer before ext_proc (inserted after buffer), and no buffer filter (unchanged).

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.75%. Comparing base (3f817c0) to head (36dd3f6).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2227      +/-   ##
==========================================
+ Coverage   84.73%   84.75%   +0.01%     
==========================================
  Files         144      144              
  Lines       21189    21195       +6     
==========================================
+ Hits        17955    17963       +8     
+ Misses       2152     2151       -1     
+ Partials     1082     1081       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[johnugeorge]

@siddharth1036

[nacx]

/reetst

[siddharth1036]

Hi @ChrisJBurns , just wanted to check I am working on a PR: #2158 where you will be able to configure exact (or prefixed, under discussion) filter name which would be allowed to run before ai-gateway-extproc, could it be useful to solve your usecase as well?

[ChrisJBurns]

Thanks for flagging this @siddharth1036 - we're definitely in the same code path. I think they're complementary rather than overlapping though: your allowlist is opt-in, but the buffer filter ordering needs to apply by default since it's a correctness fix (large bodies get a 413 otherwise). Could we treat the buffer filter as an always-on default in #2158, or let the automatic buffer handling stack on top of your allowlist?