Skip to content

koord-scheduler: support force sync data from informer#830

Merged
koordinator-bot[bot] merged 1 commit intokoordinator-sh:mainfrom
eahydra:support_force_sync_cache
Nov 24, 2022
Merged

koord-scheduler: support force sync data from informer#830
koordinator-bot[bot] merged 1 commit intokoordinator-sh:mainfrom
eahydra:support_force_sync_cache

Conversation

@eahydra
Copy link
Member

@eahydra eahydra commented Nov 22, 2022

Signed-off-by: Joseph joseph.t.lee@outlook.com

Ⅰ. Describe what this PR does

Currently, the existing scheduling plugins of koord-scheduler will construct the internal state data of the plugin based on the data in the Informer, such as which Pods are maintained in NodeNUMAResource and which CPUs are allocated, which Pods are maintained in DeviceShare and which GPU instances are allocated, and so on. These data must be correct, otherwise it may cause problems with the resources allocated during Pod scheduling, or the orchestration results may not meet expectations, which will seriously affect the runtime problems or availability of workloads.

In the current existing code, the EventHandler is registered with the Informer, and the internal state can be constructed after WaitForCacheSync returns, but WaitForCacheSync only means that the data queried from the APIServer is synchronized to the Store inside the Informer, and does not perceive the Event in the EventHandler whether processing is complete. This also means that when WaitForCacheSync returns, EventHandler may not have finished processing the existing Add Event, and this problem will be more obvious in a larger cluster.

Therefore, we need a mechanism that can help the plugin to correctly convert the data in the Informer to the internal state of the plugin, and ensure that subsequent Add/Update/Delete Events can be processed correctly. Hence the introduction of ForceSyncedEventHandler.

When the plugin is initialized, you can register the EventHandler through frameworkexthelp.ForceSyncFromInformer. This function will automatically start the Informer, and wait for the data to be synchronized successfully before pouring the current data into the EventHandler.

@codecov
Copy link

codecov bot commented Nov 22, 2022

Codecov Report

Base: 67.93% // Head: 67.93% // Increases project coverage by +0.00% 🎉

Coverage data is based on head (b7f2eba) compared to base (9ff814a).
Patch coverage: 85.71% of modified lines in pull request are covered.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #830   +/-   ##
=======================================
  Coverage   67.93%   67.93%           
=======================================
  Files         216      217    +1     
  Lines       24381    24432   +51     
=======================================
+ Hits        16564    16599   +35     
- Misses       6668     6685   +17     
+ Partials     1149     1148    -1     
Flag Coverage Δ
unittests 67.93% <85.71%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...r/plugins/elasticquota/core/group_quota_manager.go 72.67% <0.00%> (-0.60%) ⬇️
.../scheduler/plugins/elasticquota/core/quota_info.go 82.11% <16.66%> (-1.71%) ⬇️
pkg/scheduler/plugins/deviceshare/device_cache.go 86.21% <82.75%> (-0.36%) ⬇️
...heduler/frameworkext/helper/synced_eventhandler.go 83.63% <83.63%> (ø)
pkg/scheduler/plugins/coscheduling/core/core.go 40.13% <100.00%> (+0.82%) ⬆️
pkg/scheduler/plugins/coscheduling/coscheduling.go 58.97% <100.00%> (-3.24%) ⬇️
...kg/scheduler/plugins/deviceshare/device_handler.go 100.00% <100.00%> (+7.93%) ⬆️
pkg/scheduler/plugins/deviceshare/plugin.go 88.81% <100.00%> (-1.24%) ⬇️
pkg/scheduler/plugins/deviceshare/pod_handler.go 95.18% <100.00%> (+0.58%) ⬆️
pkg/scheduler/plugins/elasticquota/node_handler.go 67.18% <100.00%> (-1.00%) ⬇️
... and 10 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@eahydra eahydra force-pushed the support_force_sync_cache branch from 205cf28 to 197e512 Compare November 22, 2022 14:01
@eahydra eahydra requested a review from jasonliu747 November 22, 2022 14:02
@eahydra eahydra force-pushed the support_force_sync_cache branch from 197e512 to 08372e8 Compare November 22, 2022 14:20
@eahydra
Copy link
Member Author

eahydra commented Nov 22, 2022

Discussion from kubernetes kubernetes/kubernetes#113763

@eahydra eahydra force-pushed the support_force_sync_cache branch from 08372e8 to 79e1d0f Compare November 23, 2022 05:34
@eahydra eahydra force-pushed the support_force_sync_cache branch from 79e1d0f to 3a6927d Compare November 23, 2022 08:59
@koordinator-bot koordinator-bot bot added size/XL and removed size/L labels Nov 23, 2022
Signed-off-by: Joseph <joseph.t.lee@outlook.com>
@eahydra eahydra force-pushed the support_force_sync_cache branch from 3a6927d to b7f2eba Compare November 23, 2022 12:31
Copy link
Contributor

@buptcozy buptcozy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ltgm

Copy link
Contributor

@buptcozy buptcozy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@koordinator-bot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: buptcozy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ZiMengSheng
Copy link
Member

/lgtm

@koordinator-bot koordinator-bot bot added the lgtm label Nov 24, 2022
@koordinator-bot koordinator-bot bot merged commit 11fe621 into koordinator-sh:main Nov 24, 2022
@FillZpp
Copy link
Member

FillZpp commented Dec 13, 2022

/milestone v1.1

@koordinator-bot koordinator-bot bot added this to the v1.1 milestone Dec 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants