-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Description
currently, the implementation of cyclic lr does not support decreasing policies which could be really useful.
I wanted to have one cycle per epoch and decrease the cycle amplitude gradually so here is what I did:
set cyclic_times equal to my number of epochs that is 12
define a gamma parameter in CyclicLrUpdaterHook and add these lines in the get_lr method:
if start_iter==0:
end_ratio=end_ratio * self.gamma**runner.epoch
else:
start_ratio=start_ratio * self.gamma**runner.epoch
the logic here is since there are two phases that one start from zero to max and the other start from the max to the min, I had to decrease the end_ratio in phase one and start_ratio in phase two.
this is working now and after each epoch, the cycle amplitude drops depends on the gamma.
I'm sure that this is not the best way to implement this so I hope this feature gets implemented in the right way.