[CORE-15271] kafka/server: expose user based client quotas by IoannisRP · Pull Request #29425 · redpanda-data/redpanda

IoannisRP · 2026-01-27T19:30:02Z

Implements : CORE-15271

Expose entity type user through the describe/alter client quota kafka api.

Backports Required

Release Notes

Features

Implement user-based client quotas.

Copilot

Pull request overview

This PR implements user-based client quotas for Redpanda, enabling the Kafka API to support quota management at the user (principal) level in addition to the existing client-id based quotas.

Changes:

Added support for user entity type in Kafka client quota APIs (describe/alter)
Wired up user principal tracking throughout the quota system
Implemented feature flag gating for user quotas to support graceful upgrades
Added comprehensive test coverage for user quota functionality

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/rptest/tests/quota_management_test.py	Added test cases for user quotas, compound user+client quotas, upgrade scenarios, and comparison operators for entity types
tests/rptest/clients/kcl.py	Added optional node parameter to raw_alter_quotas for routing requests during upgrade tests
src/v/kafka/server/server.cc	Wired user principal into partition mutation quota tracking for delete_topics
src/v/kafka/server/handlers/create_topics.cc	Wired user principal into partition mutation quota tracking for create_topics
src/v/kafka/server/handlers/create_partitions.cc	Wired user principal into partition mutation quota tracking for create_partitions
src/v/kafka/server/handlers/client_quotas.cc	Implemented user entity type support in describe/alter quota handlers with feature flag gating
src/v/kafka/server/connection_context.h	Made get_principal() public to enable quota tracking by user
src/v/kafka/server/connection_context.cc	Wired user principal into fetch/produce throughput quota tracking
src/v/features/feature_table.h	Added user_based_client_quota feature flag
src/v/features/feature_table.cc	Added string representation for user_based_client_quota feature

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Copilot · 2026-01-27T19:35:55Z

+    def _get_ordering(self):
+        return {name: idx for idx, name in enumerate(self.__class__)}
+
+    def __lt__(self, other):
+        ordering = self._get_ordering()


The method name _get_ordering uses a single leading underscore, which in Python typically indicates a 'private' method. However, this method is called by the public __lt__ method and constructs ordering for comparison operations. Consider renaming it to something more descriptive like _create_enum_ordering or _build_ordering_dict to better convey its purpose of creating an ordering dictionary from the enum class.

Suggested change

def _get_ordering(self):

return {name: idx for idx, name in enumerate(self.__class__)}

def __lt__(self, other):

ordering = self._get_ordering()

def _build_ordering_dict(self):

return {name: idx for idx, name in enumerate(self.__class__)}

def __lt__(self, other):

ordering = self._build_ordering_dict()

I like that you felt then need to point out that it has a leading underscore and it is bad, but also to suggest another name with leading underscore! 😅

Copilot · 2026-01-27T19:35:55Z

+bool has_feature_user_quota(const request_context& ctx) {
+    constexpr auto feature = features::feature::user_based_client_quota;
+    const auto& ft = ctx.feature_table().local();
+    return ft.is_active(feature);
+}
+


The function name has_feature_user_quota is inconsistent with the actual feature name user_based_client_quota. Consider renaming to has_user_based_client_quota or has_feature_user_based_client_quota to match the feature naming convention.

Suggested change

bool has_feature_user_quota(const request_context& ctx) {

constexpr auto feature = features::feature::user_based_client_quota;

const auto& ft = ctx.feature_table().local();

return ft.is_active(feature);

}

bool has_user_based_client_quota(const request_context& ctx) {

constexpr auto feature = features::feature::user_based_client_quota;

const auto& ft = ctx.feature_table().local();

return ft.is_active(feature);

}

bool has_feature_user_quota(const request_context& ctx) {

return has_user_based_client_quota(ctx);

}

Copilot · 2026-01-27T19:35:55Z

+    return ft.is_active(feature);
+}
+
+constexpr auto has_user = [](const auto& e) { return e.entity_type == "user"; };


The lambda has_user is defined in the anonymous namespace but its name is too generic for namespace-level scope. Consider making it a local variable in the functions where it's used (lines 371, 502), or rename it to be more specific like is_user_entity_type to better indicate it's checking the entity_type field.

Suggested change

constexpr auto has_user = [](const auto& e) { return e.entity_type == "user"; };

constexpr auto is_user_entity_type = [](const auto& e) {

return e.entity_type == "user";

};

vbotbuildovich · 2026-01-27T21:13:22Z

CI test results

test results on build#79721

test_class	test_method	test_arguments	test_kind	job_url	test_status	passed	reason	test_history
TestReadReplicaService	test_identical_lwms_after_delete_records	{"cloud_storage_type": 1, "partition_count": 5}	integration	https://buildkite.com/redpanda/redpanda/builds/79721#019c0112-88d5-42fb-82b2-46b605bda0e0	FLAKY	10/11	Test PASSES after retries.No significant increase in flaky rate(baseline=0.0197, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.1000, p1=0.3487, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=TestReadReplicaService&test_method=test_identical_lwms_after_delete_records

pgellert · 2026-01-28T16:00:13Z

-    // either there's only a single key part or the key is a user+client
-    // compound key
-    if (entity.size() != 1) {
+    if (entity.size() > 2) {


Do we need to check for entity.size() == 0 as well here?

pgellert · 2026-01-28T16:12:57Z

    schema_registry_authz = 1ULL << 3U,
    topic_ids_api = 1ULL << 4U,
    controller_forced_reconfiguration = 1ULL << 5U,
+    user_based_client_quota = 1ULL << 6U,


Should this be 13? What was 6 used for previously?

6 used to be license in v24.3.x. 13 was replication_factor_change in the same version. I see more values that have been reused. 5 used to be serde_raft_0 and now it is controller_forced_reconfiguration

So it should be fine to reuse any number, as the header says.

IoannisRP · 2026-01-28T17:16:36Z

changes in force-push:

rebase to dev

michael-redpanda

looks good to me

michael-redpanda · 2026-01-28T17:10:46Z

+                //} else if (component.entity_type == "ip") {
+                //    return ip_predicate;


leave in? Remove?

these are the "guides" on where to extend when/if we want to support ip-based quotas.

They were already there from previous work and they were quite useful while adding user. So i still kept the ip ones for future reference.

whoops

michael-redpanda

looks fine minus gellert's comments

For backwards compatibility, a feature flag is added to protect against using the user quota entity while the cluster during upgrade

IoannisRP · 2026-01-28T17:48:45Z

changes in force-push:

add missing "zero-size" check
remove feature flag check from describe

pgellert · 2026-01-29T11:12:26Z

+                {
+                    "Entity": [
+                        {
+                            "Type": "user",


Maybe for a separate PR, but as we discussed earlier, let's ensure that we also have an upgrade test that ensures that the serde changes work end-to-end (create some quotas on an old cluster, upgrade one node, create some old quotas mid-upgrade, upgrade fully, verify everything works).

Copilot AI review requested due to automatic review settings January 27, 2026 19:30

IoannisRP self-assigned this Jan 27, 2026

IoannisRP requested review from pgellert and removed request for Copilot January 27, 2026 19:30

github-actions Bot added the area/redpanda label Jan 27, 2026

IoannisRP requested review from a team and Copilot January 27, 2026 19:30

Copilot AI reviewed Jan 27, 2026

View reviewed changes

Comment thread src/v/kafka/server/handlers/client_quotas.cc Outdated

Comment thread src/v/kafka/server/handlers/client_quotas.cc Outdated

IoannisRP force-pushed the CORE-15271/user-based-quotas-kafka-api branch from 50f6274 to 3608650 Compare January 27, 2026 19:34

IoannisRP requested a review from Copilot January 27, 2026 19:34

Copilot AI reviewed Jan 27, 2026

View reviewed changes

IoannisRP force-pushed the CORE-15271/user-based-quotas-kafka-api branch from 3608650 to af744d8 Compare January 27, 2026 19:39

IoannisRP commented Jan 28, 2026

View reviewed changes

Comment thread src/v/kafka/server/handlers/client_quotas.cc Outdated

pgellert reviewed Jan 28, 2026

View reviewed changes

IoannisRP added 2 commits January 28, 2026 17:16

kafka/server: make connection_context get_principal public

d5dd1f9

kafka/server: wire in principal name in quota manager calls

264c842

IoannisRP force-pushed the CORE-15271/user-based-quotas-kafka-api branch from af744d8 to ebdb04b Compare January 28, 2026 17:16

michael-redpanda previously approved these changes Jan 28, 2026

View reviewed changes

michael-redpanda reviewed Jan 28, 2026

View reviewed changes

IoannisRP added 2 commits January 28, 2026 17:29

kafka/server: extend client quota kafka api to accept user as valid

4031196

kafka/server: add feature flag protecting against user quota

49ccca4

For backwards compatibility, a feature flag is added to protect against using the user quota entity while the cluster during upgrade

IoannisRP force-pushed the CORE-15271/user-based-quotas-kafka-api branch from ebdb04b to 49ccca4 Compare January 28, 2026 17:48

IoannisRP requested review from michael-redpanda and pgellert January 28, 2026 17:48

michael-redpanda approved these changes Jan 28, 2026

View reviewed changes

IoannisRP mentioned this pull request Jan 29, 2026

[CORE-15272] dt: add basic test for client quota metrics #29457

Merged

7 tasks

pgellert approved these changes Jan 29, 2026

View reviewed changes

IoannisRP merged commit aaec6a9 into redpanda-data:dev Jan 29, 2026
21 checks passed

		//} else if (component.entity_type == "ip") {
		// return ip_predicate;

Conversation

IoannisRP commented Jan 27, 2026 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backports Required

Release Notes

Features

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

IoannisRP Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

vbotbuildovich commented Jan 27, 2026

CI test results

Uh oh!

Uh oh!

pgellert Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

pgellert Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

IoannisRP Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

IoannisRP commented Jan 28, 2026

Uh oh!

michael-redpanda left a comment

Choose a reason for hiding this comment

Uh oh!

michael-redpanda Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

IoannisRP Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

michael-redpanda left a comment

Choose a reason for hiding this comment

Uh oh!

IoannisRP commented Jan 28, 2026

Uh oh!

pgellert Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

IoannisRP commented Jan 27, 2026 •

edited by atlassian Bot

Loading