[kfirtoledo]

Enable MultiConnector to use the new continuous cross-layer KV cache layout
described in RFC #27742 and implemented in #27743.
The MultiConnector aggregates cross-layer support across all underlying connectors
and returns true only if all connectors support it.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.

[gemini-code-assist]

Code Review

This pull request enables the MultiConnector to support the new continuous cross-layer KV cache layout. The changes correctly propagate the prefer_cross_layer_blocks property and the register_cross_layers_kv_cache method to the underlying connectors. My review identifies a potential edge case in the implementation of prefer_cross_layer_blocks where it could return an incorrect value if no connectors are configured, which could lead to unexpected behavior. I've provided a suggestion to make the implementation more robust.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

[mergify]

Hi @kfirtoledo, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[orozery]

@NickLucche per your suggestion here we defined
the prefer_cross_layer_blocks as a ClassVar.
This does not work well for the MultiConnector, as its prefer_cross_layer_blocks can only be determined at runtime.
I think we should change this ClassVar to a @property. WDYT?

[NickLucche]

Thanks for doing this @kfirtoledo!
And yes the property change is fine here @orozery , given this feature was just introduced.

Would you mind adding a small unit test to keep this feature in check? It looks trivial, but we had similar issues with MC before..
Other than that this is lgtm.

[kfirtoledo]

@NickLucche and @orozery, I added the unit tests, PTAL

[mergify]

Hi @kfirtoledo, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[orozery]

lgtm. Thanks!

[NickLucche]

Thanks @kfirtoledo !