stopping gradient for encoder of Transformer

Hello,
I have a pretrained model. I want to load it to train for another task and don't want to train the encoder of transformer. Do you have any convenient way to do that?

Take a look at the FairseqEncoderDecoderModel base class. It should be straightforward to overload the forward method, load any kind of pretrained encoder, and detach() the encoder's output.