Commit 8d0c6cf
committed
WIP: Set empirically determined Adaptive values
Based on some experiments on the cluster, it appears that it takes
anywhere from 9s-15s to startup a single worker. So set the startup cost
to the average value of 12s. Given it takes a bit to startup workers,
bump the interval for rechecking whether to adjust workers to 6s
(leaving the number of checks the same as the default of 3). Thus the
time for a job to startup and get processing some data should occur a
little after if not at least by the 2nd check, which avoid rapid
adjustment of workers, while still allowing it to be frequent enough to
notice if things need a slight tweak. This should slow down rapid
downscaling and upscaling a bit hopefully avoiding fluctuations.1 parent 14ee9e0 commit 8d0c6cf
1 file changed
+2
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
91 | 91 | | |
92 | 92 | | |
93 | 93 | | |
| 94 | + | |
| 95 | + | |
94 | 96 | | |
95 | 97 | | |
96 | 98 | | |
| |||
0 commit comments