fix: split the ext proc config to avoid k8s size limits

[zhaohuabing]

Description

This PR splits the ext proc/mcpproxy config into multiple secrets, so it can bypass the k8s size limits.

Related Issues/PRs (if applicable)

Fixes #1613

cc @nacx

[codecov-commenter]

Codecov Report

❌ Patch coverage is 80.51948% with 60 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.82%. Comparing base (0ad64b8) to head (daef4f4).

Files with missing lines	Patch %	Lines
internal/filterapi/watcher.go	66.12%	13 Missing and 8 partials ⚠️
internal/controller/filter_config_bundle.go	72.72%	11 Missing and 7 partials ⚠️
internal/controller/gateway.go	47.05%	6 Missing and 3 partials ⚠️
internal/controller/gateway_mutator.go	95.45%	2 Missing and 2 partials ⚠️
internal/controller/secret_name.go	90.24%	2 Missing and 2 partials ⚠️
internal/filterapi/config_bundle.go	88.23%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1865      +/-   ##
==========================================
- Coverage   84.91%   84.82%   -0.10%     
==========================================
  Files         145      148       +3     
  Lines       21593    21838     +245     
==========================================
+ Hits        18336    18524     +188     
- Misses       2161     2194      +33     
- Partials     1096     1120      +24     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[nacx]

Overall, it looks good.

So the summary of the PR is that:

• It will split the secret if it exceeds the max object k8s size, and split to a max of 8 chunks (for now).
• If there is no size overflow, empty chunks up to the 8 max ones will be created to avoid having to remount and cycle the pod.
• The extproc receives a new parameter, -configBundlePath, that if present will start the new watcher to watch for the new chunked config.

If this understanding is OK :) I have a few questions:

• Can we avoid creating the empty chunks? We could still add the volumes and secret projections to the pod, but mark those optional? Would that work to avoid having to create the empty secrets?
• The controller will pass the new arg -configBundlePath to the extproc if it finds the "index" secret. That secret is always created now, correct?
  • Could we provide a way to opt-in to this new feature, so that the controller does not create this secret (and extproc does not instantiate the new watchers, etc, etc)? This is quite a sensitive feature, and it would be great to have an option to explicitly opt-in.

Thanks for this PR!

[zhaohuabing]

/retest

[zhaohuabing]

Can we avoid creating the empty chunks? We could still add the volumes and secret projections to the pod, but mark those optional? Would that work to avoid having to create the empty secrets?

Fixed.

The controller will pass the new arg -configBundlePath to the extproc if it finds the "index" secret. That secret is always created now, correct?
Could we provide a way to opt-in to this new feature, so that the controller does not create this secret (and extproc does not instantiate the new watchers, etc, etc)? This is quite a sensitive feature, and it would be great to have an option to explicitly opt-in.

Both the chunked secrets are always created, and the legacy secret is kept for upgrade where both the old and new version of the AIGW could coexist. We can remove the legacy one in the next release.

In theory, we could opt-in the chunked secrets, but I have a few concerns to this approach:

1. The chunked Secret layout is intended to be an internal controller/extproc implementation detail. Adding an opt-in would expose that implementation detail as user-facing configuration.

2. It would also introduce another supported mode that we need to keep testing across the controller, mutator, and extproc startup paths. It may not double the test effort, but it would add meaningful test coverage(translate output, unit tests, e2e tests to cover both modes) and ongoing maintenance cost.

[zhaohuabing]

/retest

[nacx]

/retest

[zhaohuabing]

/retest