Skip to content

decreasing the cycle amplitude after each period in cyclic lr #783

@minouei-kl

Description

@minouei-kl

currently, the implementation of cyclic lr does not support decreasing policies which could be really useful.
I wanted to have one cycle per epoch and decrease the cycle amplitude gradually so here is what I did:
set cyclic_times equal to my number of epochs that is 12
define a gamma parameter in CyclicLrUpdaterHook and add these lines in the get_lr method:

                if start_iter==0:
                    end_ratio=end_ratio * self.gamma**runner.epoch
                else:
                    start_ratio=start_ratio * self.gamma**runner.epoch

the logic here is since there are two phases that one start from zero to max and the other start from the max to the min, I had to decrease the end_ratio in phase one and start_ratio in phase two.
this is working now and after each epoch, the cycle amplitude drops depends on the gamma.
I'm sure that this is not the best way to implement this so I hope this feature gets implemented in the right way.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions