Skip to content

The job task resync logic is not right #384

@hzxuzhonghu

Description

@hzxuzhonghu

/kind bug

When i deleted several pods of volcano job, i see too many errors

E0724 01:27:37.980526       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:37.984007       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
I0724 01:27:37.985337       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-4> of Job <default/test-job3>
I0724 01:27:37.985356       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-5> of Job <default/test-job3>
E0724 01:27:37.986466       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
I0724 01:27:37.986642       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-0> of Job <default/test-job3>
I0724 01:27:37.989292       1 job_controller_actions.go:349] Finished Job <default/test-job3> sync up
E0724 01:27:37.989859       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:37.991320       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:37.992678       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:37.994137       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:37.995630       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:37.996875       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:37.998123       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:37.999642       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.001281       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.002655       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.005832       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.007604       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.009749       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.012752       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.014255       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.018370       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.021022       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.022929       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.024827       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.028570       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.030376       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.032411       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.054733       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.056256       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.059800       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.062486       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.064431       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.066314       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.070034       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.071687       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.073770       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.136685       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.138391       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.141394       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.143954       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.145752       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.147767       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.151643       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.152966       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.155177       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.298918       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.300686       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.302956       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.305450       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.307124       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.309255       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.313123       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.314624       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.316636       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
I0724 01:27:38.360519       1 job_controller.go:295] Try to handle request <Job: default/test-job3, Task:default-nginx, Event:PodEvicted, ExitCode:0, Action:, JobVersion: 0>
I0724 01:27:38.362306       1 job_controller_util.go:138] Request Job: default/test-job3, Task:default-nginx, Event:PodEvicted, ExitCode:0, Action:, JobVersion: 0 is outdated, will perform sync instead.
I0724 01:27:38.362367       1 job_controller.go:312] Execute <SyncJob> on Job <default/test-job3> in <Inqueue> by <*state.inqueueState>.
I0724 01:27:38.362388       1 job_controller_actions.go:200] Starting to sync up Job <default/test-job3>
I0724 01:27:38.362413       1 job_controller_actions.go:204] Current Version is: 2 of job: default/test-job3
I0724 01:27:38.362582       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:38.362651       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:38.362711       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:38.362759       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:38.362793       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:38.362812       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:38.362831       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:38.362847       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:38.362879       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:38.367274       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-5> of Job <default/test-job3>
I0724 01:27:38.369116       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-0> of Job <default/test-job3>
I0724 01:27:38.369309       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-4> of Job <default/test-job3>
E0724 01:27:38.375256       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
I0724 01:27:38.375573       1 job_controller_actions.go:349] Finished Job <default/test-job3> sync up
I0724 01:27:38.376551       1 job_controller.go:295] Try to handle request <Job: default/test-job3, Task:default-nginx, Event:PodEvicted, ExitCode:0, Action:, JobVersion: 0>
I0724 01:27:38.376599       1 job_controller_util.go:138] Request Job: default/test-job3, Task:default-nginx, Event:PodEvicted, ExitCode:0, Action:, JobVersion: 0 is outdated, will perform sync instead.
I0724 01:27:38.376617       1 job_controller.go:312] Execute <SyncJob> on Job <default/test-job3> in <Inqueue> by <*state.inqueueState>.
I0724 01:27:38.376634       1 job_controller_actions.go:200] Starting to sync up Job <default/test-job3>
I0724 01:27:38.376670       1 job_controller_actions.go:204] Current Version is: 2 of job: default/test-job3
I0724 01:27:38.376699       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:38.376714       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:38.376720       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:38.376731       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:38.376740       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:38.376745       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:38.376773       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:38.376782       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:38.376787       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
E0724 01:27:38.377810       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.380311       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
I0724 01:27:38.382917       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-4> of Job <default/test-job3>
I0724 01:27:38.382992       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-0> of Job <default/test-job3>
I0724 01:27:38.383030       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-5> of Job <default/test-job3>
I0724 01:27:38.385520       1 job_controller_actions.go:349] Finished Job <default/test-job3> sync up
E0724 01:27:38.386814       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.389489       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.390835       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.392569       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.394562       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.396442       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.402527       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.403972       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.406115       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.408370       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.410995       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.417899       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.423943       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.425382       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.427496       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.449767       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.452509       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.459443       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.465388       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.466756       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.468831       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.531792       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.534176       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.540979       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.547070       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.548394       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.552692       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.620892       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.622410       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.624744       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.626849       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.628675       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.630753       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.634712       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.636217       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.638155       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.693954       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:38.696049       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.702660       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.708647       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:38.710255       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:38.714836       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
I0724 01:27:38.959630       1 job_controller.go:295] Try to handle request <Job: default/test-job3, Task:default-nginx, Event:OutOfSync, ExitCode:0, Action:, JobVersion: 2>
I0724 01:27:38.959736       1 job_controller.go:312] Execute <SyncJob> on Job <default/test-job3> in <Inqueue> by <*state.inqueueState>.
I0724 01:27:38.959753       1 job_controller_actions.go:200] Starting to sync up Job <default/test-job3>
I0724 01:27:38.959849       1 job_controller_actions.go:204] Current Version is: 2 of job: default/test-job3
I0724 01:27:38.959883       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:38.959908       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:38.960327       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:38.960362       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:38.960403       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:38.960419       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:38.960437       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:38.960443       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:38.960446       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:38.974735       1 queue_controller.go:141] Begin sync queue default
I0724 01:27:38.974759       1 queue_controller.go:184] queue default jobs pending 0, running 1, unknown 0
I0724 01:27:38.981630       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-0> of Job <default/test-job3>
I0724 01:27:38.985997       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-4> of Job <default/test-job3>
I0724 01:27:38.986191       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-5> of Job <default/test-job3>
I0724 01:27:38.993089       1 job_controller_actions.go:349] Finished Job <default/test-job3> sync up
I0724 01:27:38.993118       1 job_controller.go:295] Try to handle request <Job: default/test-job3, Task:, Event:, ExitCode:0, Action:, JobVersion: 0>
I0724 01:27:38.993134       1 job_controller_util.go:138] Request Job: default/test-job3, Task:, Event:, ExitCode:0, Action:, JobVersion: 0 is outdated, will perform sync instead.
I0724 01:27:38.993221       1 job_controller.go:312] Execute <SyncJob> on Job <default/test-job3> in <Inqueue> by <*state.inqueueState>.
I0724 01:27:38.993328       1 job_controller_actions.go:200] Starting to sync up Job <default/test-job3>
I0724 01:27:38.993421       1 job_controller_actions.go:204] Current Version is: 2 of job: default/test-job3
I0724 01:27:38.993502       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:38.993547       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:38.993621       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:38.993970       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:38.993984       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:38.993991       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:38.993999       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:38.994056       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:38.994067       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:39.001406       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-4> of Job <default/test-job3>
I0724 01:27:39.002708       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-0> of Job <default/test-job3>
I0724 01:27:39.002924       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-5> of Job <default/test-job3>
I0724 01:27:39.005152       1 job_controller_actions.go:349] Finished Job <default/test-job3> sync up
I0724 01:27:39.005176       1 job_controller.go:295] Try to handle request <Job: default/test-job3, Task:default-nginx, Event:OutOfSync, ExitCode:0, Action:, JobVersion: 2>
I0724 01:27:39.005190       1 job_controller.go:312] Execute <SyncJob> on Job <default/test-job3> in <Inqueue> by <*state.inqueueState>.
I0724 01:27:39.005210       1 job_controller_actions.go:200] Starting to sync up Job <default/test-job3>
I0724 01:27:39.005251       1 job_controller_actions.go:204] Current Version is: 2 of job: default/test-job3
I0724 01:27:39.005287       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:39.005315       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:39.005340       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:39.005382       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:39.005392       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:39.005397       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:39.005405       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:39.005411       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:39.005439       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:39.008724       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-5> of Job <default/test-job3>
I0724 01:27:39.011724       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-0> of Job <default/test-job3>
I0724 01:27:39.011731       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-4> of Job <default/test-job3>
I0724 01:27:39.014872       1 job_controller_actions.go:349] Finished Job <default/test-job3> sync up
E0724 01:27:39.016559       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:39.019339       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:39.061866       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:39.162028       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:39.262304       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:39.264299       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:39.265805       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:39.267156       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:39.268923       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:39.270320       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:39.273563       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:39.284362       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:39.287309       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:39.288883       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:39.362586       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:39.462377       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:39.562188       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:39.664435       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:39.762053       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:39.862046       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:39.962108       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:40.061825       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:40.161979       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:40.261924       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:40.362164       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:40.461918       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:40.562077       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
I0724 01:27:40.658250       1 job_controller.go:295] Try to handle request <Job: default/test-job3, Task:default-nginx, Event:OutOfSync, ExitCode:0, Action:, JobVersion: 2>
I0724 01:27:40.658314       1 job_controller.go:312] Execute <SyncJob> on Job <default/test-job3> in <Inqueue> by <*state.inqueueState>.
I0724 01:27:40.658334       1 job_controller_actions.go:200] Starting to sync up Job <default/test-job3>
I0724 01:27:40.658359       1 job_controller_actions.go:204] Current Version is: 2 of job: default/test-job3
I0724 01:27:40.658409       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:40.658426       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:40.658441       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:40.658463       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:40.658476       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:40.658491       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:40.658509       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:40.658523       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:40.658536       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
E0724 01:27:40.665204       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
I0724 01:27:40.669913       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-0> of Job <default/test-job3>
I0724 01:27:40.671038       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-4> of Job <default/test-job3>
I0724 01:27:40.673972       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-5> of Job <default/test-job3>
I0724 01:27:40.683646       1 job_controller_actions.go:349] Finished Job <default/test-job3> sync up
I0724 01:27:40.683698       1 job_controller.go:295] Try to handle request <Job: default/test-job3, Task:default-nginx, Event:OutOfSync, ExitCode:0, Action:, JobVersion: 2>
I0724 01:27:40.683717       1 job_controller.go:312] Execute <SyncJob> on Job <default/test-job3> in <Inqueue> by <*state.inqueueState>.
I0724 01:27:40.683807       1 job_controller_actions.go:200] Starting to sync up Job <default/test-job3>
I0724 01:27:40.683887       1 job_controller_actions.go:204] Current Version is: 2 of job: default/test-job3
I0724 01:27:40.686066       1 job_controller_handler.go:91] Job update event is ignored since no update in 'Spec'.
I0724 01:27:40.687095       1 garbagecollector.go:107] Updating job default/test-job3
I0724 01:27:40.687537       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:40.687614       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:40.687665       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:40.687736       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:40.687777       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:40.687818       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:40.688292       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:40.688309       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:40.688314       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:40.692972       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-0> of Job <default/test-job3>
I0724 01:27:40.695007       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-5> of Job <default/test-job3>
I0724 01:27:40.695022       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-4> of Job <default/test-job3>
I0724 01:27:40.699559       1 job_controller_actions.go:349] Finished Job <default/test-job3> sync up
I0724 01:27:40.700264       1 garbagecollector.go:107] Updating job default/test-job3
I0724 01:27:40.700306       1 job_controller_handler.go:91] Job update event is ignored since no update in 'Spec'.
I0724 01:27:40.755421       1 job_controller.go:295] Try to handle request <Job: default/test-job3, Task:default-nginx, Event:OutOfSync, ExitCode:0, Action:, JobVersion: 2>
I0724 01:27:40.755470       1 job_controller.go:312] Execute <SyncJob> on Job <default/test-job3> in <Inqueue> by <*state.inqueueState>.
I0724 01:27:40.755501       1 job_controller_actions.go:200] Starting to sync up Job <default/test-job3>
I0724 01:27:40.755515       1 job_controller_actions.go:204] Current Version is: 2 of job: default/test-job3
I0724 01:27:40.755556       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:40.755672       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:40.755808       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:40.755842       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:40.755857       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:40.755892       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:40.755917       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:40.755930       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:40.755942       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:40.763335       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-0> of Job <default/test-job3>
I0724 01:27:40.763335       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-4> of Job <default/test-job3>
I0724 01:27:40.763427       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-5> of Job <default/test-job3>
E0724 01:27:40.763718       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
I0724 01:27:40.767425       1 garbagecollector.go:107] Updating job default/test-job3
I0724 01:27:40.768696       1 job_controller_actions.go:349] Finished Job <default/test-job3> sync up
I0724 01:27:40.768846       1 job_controller.go:295] Try to handle request <Job: default/test-job3, Task:, Event:OutOfSync, ExitCode:0, Action:, JobVersion: 0>
I0724 01:27:40.769003       1 job_controller.go:312] Execute <SyncJob> on Job <default/test-job3> in <Running> by <*state.runningState>.
I0724 01:27:40.769032       1 job_controller_actions.go:200] Starting to sync up Job <default/test-job3>
I0724 01:27:40.769053       1 job_controller_actions.go:204] Current Version is: 2 of job: default/test-job3
I0724 01:27:40.769074       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:40.769089       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:40.769104       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:40.769128       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:40.769149       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:40.769164       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:40.769182       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: env on job: <default/test-job3>
I0724 01:27:40.769194       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: ssh on job: <default/test-job3>
I0724 01:27:40.769216       1 job_controller_plugins.go:40] Starting to execute plugin at <pluginOnPodCreate>: svc on job: <default/test-job3>
I0724 01:27:40.775287       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-5> of Job <default/test-job3>
I0724 01:27:40.777404       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-0> of Job <default/test-job3>
I0724 01:27:40.777459       1 job_controller_actions.go:273] Created Task <test-job3-default-nginx-4> of Job <default/test-job3>
I0724 01:27:40.780248       1 job_controller_actions.go:349] Finished Job <default/test-job3> sync up
E0724 01:27:40.861923       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:40.961956       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:41.061717       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:41.161968       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:41.262458       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:41.362189       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:41.462186       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:41.562085       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:41.662156       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:41.762104       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:41.862073       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:41.962265       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:42.061912       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:42.162277       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:42.262132       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:42.362207       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:42.462064       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:42.562144       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:42.662230       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:42.762120       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:42.861985       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:42.962093       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:43.061985       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:43.162004       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:43.262080       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:43.362057       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:43.462070       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:43.562228       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:43.662153       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:43.762370       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:43.862050       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:43.962180       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:44.061882       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:44.162066       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:44.262120       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:44.362175       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:44.462173       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-4>, retry it, err can not find pod <default/test-job3-default-nginx-4> in cache
E0724 01:27:44.562121       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache
E0724 01:27:44.662120       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-0>, retry it, err can not find pod <default/test-job3-default-nginx-0> in cache
E0724 01:27:44.762155       1 job_controller_resync.go:62] Failed to sync pod <default/test-job3-default-nginx-5>, retry it, err can not find pod <default/test-job3-default-nginx-5> in cache

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions