affinity: add base models and manager skeleton #9997

lhy1024 · 2025-12-04T12:48:58Z

What problem does this PR solve?

Issue Number: Ref #9764

What is changed and how does it work?

Add base affinity models/state: group.go introduces Group/GroupState, availability/degraded/expired conditions, ID validation, and region affinity checks.
Manager skeleton: manager.go adds Manager with dual locking (metaMutex + RWMutex)

Check List

Tests

Unit test

Release note

None.

Signed-off-by: lhy1024 <admin@liudos.us>

codecov · 2025-12-04T13:10:10Z

Codecov Report

❌ Patch coverage is 14.89796% with 417 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.07%. Comparing base (dadce5d) to head (b191733).
⚠️ Report is 7 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #9997      +/-   ##
==========================================
- Coverage   78.53%   78.07%   -0.47%     
==========================================
  Files         503      510       +7     
  Lines       67322    67900     +578     
==========================================
+ Hits        52874    53013     +139     
- Misses      10615    11045     +430     
- Partials     3833     3842       +9

Flag	Coverage Δ
unittests	`78.07% <14.89%> (-0.47%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

lhy1024 · 2025-12-05T02:17:47Z

@bufferflies @okJiang PTAL

bufferflies · 2025-12-05T02:28:50Z

pkg/schedule/affinity/stub_affinity.go

+}
+
+//nolint:revive
+func (m *Manager) UpdateAffinityGroupPeers(string, uint64, []uint64) (*GroupState, error) {


Suggested change

func (m *Manager) UpdateAffinityGroupPeers(string, uint64, []uint64) (*GroupState, error) {

func (*Manager) UpdateAffinityGroupPeers(string, uint64, []uint64) (*GroupState, error) {

ok, but it will be used really in another pr later.

bufferflies · 2025-12-05T02:37:32Z

pkg/schedule/affinity/group.go

+	if slice.HasDupInSorted(voterStoreIDs) {
+		return errs.ErrAffinityGroupContent.FastGenByArgs("duplicate voter store ID")
+	}
+	if !slices.Contains(voterStoreIDs, g.LeaderStoreID) {


In generative speaking, the leader should belong to the voters. How about renaming voterStoreIDs as followerStoreIDs if you don't allow the leader to exist in this list.

Yes, we now only return error when LeaderStoreID is not in voterStoreIDs

bufferflies · 2025-12-05T02:39:35Z

pkg/schedule/affinity/group.go

+	}
+	for _, storeID := range voterStoreIDs {
+		if m.storeSetInformer.GetStore(storeID) == nil {
+			return errs.ErrAffinityGroupContent.FastGenByArgs("store does not exist")


How about informing the user that the store was not found?

bufferflies · 2025-12-05T02:40:15Z

pkg/schedule/affinity/group.go

+// GroupKeyRanges represents key ranges with group id.
+type GroupKeyRanges struct {
+	KeyRanges []keyutil.KeyRange
+	GroupID   string


How about adding more to the format of this groupID?

Or we add it in pd-ctl about group format, I have added it in pd-ctl. We don't limit the format in server, it should be a convention for pd-ctl and SQL.

The region label also does not conform to the server's specified format.

bufferflies · 2025-12-05T02:52:15Z

pkg/schedule/affinity/group_test.go

+	re.NoError(err)
+
+	group := &Group{
+		ID:            "TEST-group_1",


affinity_group_id = tidb_p_t

it only for test

Signed-off-by: lhy1024 <admin@liudos.us>

okJiang · 2025-12-05T07:34:52Z

/cc @okJiang

pkg/schedule/affinity/group.go

okJiang · 2025-12-05T09:13:02Z

pkg/schedule/affinity/group.go

+)
+
+// Phase is a status intended for API display
+type Phase string


Moving Phase and condition to a new file is better for me.

I think it is not necessary for now. It will cause problems for subsequent code merges. Once all code has been merged, we can consider making further adjustments if necessary.

pkg/schedule/affinity/group.go

okJiang · 2025-12-05T09:17:24Z

pkg/schedule/affinity/group.go

+	// AffinityVer initializes at 1 and increments by 1 each time the Group changes.
+	AffinityVer uint64
+	// AffinityRegionCount indicates how many Regions have all Voter and Leader peers in the correct stores. (AffinityVer equals).
+	AffinityRegionCount int


This is better.

Suggested change

AffinityRegionCount int

AffinitizedRegionCount int

affinity is also adj.

okJiang · 2025-12-05T09:21:37Z

pkg/schedule/affinity/group.go

+	// AffinityVer initializes at 1 and increments by 1 each time the Group changes.
+	AffinityVer uint64
+	// AffinityRegionCount indicates how many Regions have all Voter and Leader peers in the correct stores. (AffinityVer equals).
+	AffinityRegionCount int


Why exist two same fields? L171-L173

runtimeGroupInfo is struct for memory, GroupState is struct for api handler

Why was it designed this way?

pkg/schedule/affinity/affinity_test.go

okJiang · 2025-12-05T09:40:27Z

pkg/schedule/affinity/affinity_test.go

How about renaming to test_utils.go

Files ending with _test will only be compiled during test.

Signed-off-by: lhy1024 <admin@liudos.us>

ti-chi-bot · 2025-12-08T04:44:03Z

[LGTM Timeline notifier]

Timeline:

2025-12-08 02:09:40.88449881 +0000 UTC m=+834125.698276382: ☑️ agreed by okJiang.
2025-12-08 04:44:02.344331637 +0000 UTC m=+843387.158109208: ☑️ agreed by bufferflies.

lhy1024 · 2025-12-08T05:09:12Z

@niubell PTAL

niubell

The first PR to add the basic structures, will add the corresponding tests in next PRs.

lhy1024 · 2025-12-08T06:14:03Z

The first PR to add the basic structures, will add the corresponding tests in next PRs.

yes, they will be added in 9998 and 9999

ti-chi-bot · 2025-12-08T06:15:08Z

@niubell: adding LGTM is restricted to approvers and reviewers in OWNERS files.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

ti-chi-bot · 2025-12-08T06:15:10Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bufferflies, niubell, okJiang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [bufferflies]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

lhy1024 · 2025-12-08T06:16:38Z

/test pull-error-log-review

ti-chi-bot · 2025-12-08T06:20:48Z

In response to a cherrypick label: new pull request created to branch release-8.5: #10015.

rleungx · 2025-12-11T07:16:15Z

pkg/schedule/affinity/group.go

+	case storeDown:
+		return "down"
+	case storeRemovingOrRemoved:
+		return "removing-or-removed"


prefer to split them

rleungx · 2025-12-11T07:30:18Z

pkg/schedule/affinity/group.go

+}
+
+// IsAffinitySchedulingEnabled indicates whether affinity scheduling is allowed.
+func (g *runtimeGroupInfo) IsAffinitySchedulingEnabled() bool {


The name is the same as IsAffinitySchedulingEnabled in the config, but the meaning is different.

ref tikv#9764 Signed-off-by: lhy1024 <admin@liudos.us>

ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. dco-signoff: yes Indicates the PR's author has signed the dco. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Dec 4, 2025

affinity: add base models and manager skeleton

9ba9b3a

Signed-off-by: lhy1024 <admin@liudos.us>

lhy1024 force-pushed the affinity-pr2A branch from 4b84a5a to 9ba9b3a Compare December 4, 2025 12:50

This was referenced Dec 4, 2025

affinity: add txn and key range operations #9998

Merged

affinity: add policy, metrics collection and tests #9999

Merged

Support data affinity schedule for performance optimization #9764

Open

bufferflies reviewed Dec 5, 2025

View reviewed changes

add store ID

77edba8

Signed-off-by: lhy1024 <admin@liudos.us>

lhy1024 requested a review from bufferflies December 5, 2025 03:33

ti-chi-bot bot added the needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. label Dec 5, 2025

ti-chi-bot bot requested a review from okJiang December 5, 2025 07:34

okJiang reviewed Dec 5, 2025

View reviewed changes

address comments: rename and split condition

b191733

Signed-off-by: lhy1024 <admin@liudos.us>

okJiang approved these changes Dec 8, 2025

View reviewed changes

ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Dec 8, 2025

bufferflies approved these changes Dec 8, 2025

View reviewed changes

ti-chi-bot bot added lgtm approved and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Dec 8, 2025

niubell reviewed Dec 8, 2025

View reviewed changes

niubell approved these changes Dec 8, 2025

View reviewed changes

ti-chi-bot bot merged commit 23550eb into tikv:master Dec 8, 2025
38 of 42 checks passed

ti-chi-bot mentioned this pull request Dec 8, 2025

affinity: add base models and manager skeleton (#9997) #10015

Closed

rleungx reviewed Dec 11, 2025

View reviewed changes

HunDunDM pushed a commit to HunDunDM/pd that referenced this pull request Dec 11, 2025

affinity: add base models and manager skeleton (tikv#9997)

65701f0

ref tikv#9764 Signed-off-by: lhy1024 <admin@liudos.us>

lhy1024 added a commit to lhy1024/pd that referenced this pull request Dec 14, 2025

affinity: add base models and manager skeleton (tikv#9997)

95a6d78

ref tikv#9764 Signed-off-by: lhy1024 <admin@liudos.us>

lhy1024 added a commit to lhy1024/pd that referenced this pull request Dec 16, 2025

affinity: add base models and manager skeleton (tikv#9997)

fd8a2e0

ref tikv#9764 Signed-off-by: lhy1024 <admin@liudos.us>

lhy1024 added a commit to lhy1024/pd that referenced this pull request Dec 16, 2025

affinity: add base models and manager skeleton (tikv#9997)

111360b

ref tikv#9764 Signed-off-by: lhy1024 <admin@liudos.us>

lhy1024 mentioned this pull request Dec 17, 2025

cherrypick data affinity feature to release-8.5 #10061

Merged

	func (m Manager) UpdateAffinityGroupPeers(string, uint64, []uint64) (GroupState, error) {
	func (Manager) UpdateAffinityGroupPeers(string, uint64, []uint64) (GroupState, error) {

affinity: add base models and manager skeleton #9997

affinity: add base models and manager skeleton #9997

Uh oh!

Conversation

lhy1024 commented Dec 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

What is changed and how does it work?

Check List

Release note

Uh oh!

codecov bot commented Dec 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

lhy1024 commented Dec 5, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

okJiang commented Dec 5, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ti-chi-bot bot commented Dec 8, 2025

[LGTM Timeline notifier]

Uh oh!

lhy1024 commented Dec 8, 2025

Uh oh!

niubell left a comment

Choose a reason for hiding this comment

Uh oh!

lhy1024 commented Dec 8, 2025

Uh oh!

ti-chi-bot bot commented Dec 8, 2025

Uh oh!

ti-chi-bot bot commented Dec 8, 2025

Uh oh!

lhy1024 commented Dec 8, 2025

lhy1024 commented Dec 4, 2025 •

edited

Loading

codecov bot commented Dec 4, 2025 •

edited

Loading