256x256 FFHQ StyleGAN-V2

[eladrich]

Hi,
I was wondering if you're aware of a pretrained 256x256 FFHQ StyleGANv2 model available? I think it would be really be helpful for tasks where the full model is too big to start with.

[justinpinkney]

A while back I tried to train one starting from an adapted version of the 1024x model, but the transferring process seemed to cripple it a bit, so gave up.

The only one I've seen is in the pytorch stylegan2 repo: https://github.com/rosinality/stylegan2-pytorch But this was trained without the learnable noise parameters, so is a little different to "official" StyleGAN2.

I might try training a 256x256 model on ffhq, but this might take me a little while as my computing resources are limited. If anyone has one already it would be great to know about it!

[eladrich]

Interesting, wonder what can make that happen. it seems like the last two scales, and their style vectors, make only minor changes in the output image so wouldn't expect them to affect the training process so much 🤔.

I've actually tried the one in the rosinality repo but it has a worse FID score, maybe the missing noise you mentioned, or some other small variant in the training process.

Hopefully someone will find the time and resources to release a tuned model :-)

[aydao]

I got you. Here's a 256px model (StyleGAN2 config-f) that I derived from the weights for FFHQ released by @NVlabs. All I did was cut off the top few layers of G and D, then finetune the 256x256 layers (especially the toRGB layer). Then I finetuned the whole network for just a few kimg, which helped get a tiny bit of improvement for the perceptual quality of samples.

Download link. Let me know if that works for you, or if you encounter any issues. I haven't tested it out thoroughly or computed FID, and it is possible there are problems with the performance of D.

[eps696]

@aydao could you please explain a bit more, how did you manage to finetune only select (toRGB) layers in F config?

[aydao]

I restricted the trainable variables by modifying the list of trainables for G and D, by filtering using this line here. Only the variables named in self.trainables for a dnnlib network will be trained (e.g., in the tensorflow computation graph, only those vars will be updated during backprop), all others will be "frozen" and remain the same.

For example, I think I used self.trainables = OrderedDict((name, var) for name, var in self.vars.items() if var.trainable and '256x256' in name) or something close to it.

[eps696]

thanks, will give it one more go. i tried modifying this line to freeze D at finetuning [for this] but didn't succeed (can't recall now what was wrong)

[justinpinkney]

Just checked the fid of the model trained above and it's 19.2520

[eps696]

@aydao tried freezing vars again, recalled the question - how did you replace G/D network, loaded from the source pkl, with the modified one (loaded from the code)? did you save the source weights as standard TF checkpoint, or..?

UPD: pardon for stupidity, have been missing resume_with_new_nets param all this time.. fixed now.

[justinpinkney]

OK so after training for a while the best I get for a 256x256 config-e trained from scratch is a FID of
11.2

I'll add it to the repo soon, but for now here is a Google Drive link: https://drive.google.com/open?id=1BUL-RIzXC7Bpnz2cn230CbA4eT7_Etp0

[justinpinkney]

If anyone ever trains a version to a better FID (or a config-f) or tries out the above model for transfer learning then I'd love to hear about it.

For now I'll close this issue.

[betterze]

@aydao Could you explain how you cut off the top few layers of G and D? You mean create a new model in 256x256 resolusion, then load the weights? I can not find a way to load the weights to a new model in lower resolusion in stylegan2. Could you explain how you did this? Thank you in advance.

[betterze]

Just find the solution. Use

Gs_target.copy_vars_from(Gs_source)

[betterze]

@aydao Would you mind elaborating about how you train the ffhq 256 or sharing the code? I can not reproduce the results. You mention that finetune the 256x256 layers (especially the toRGB layer). What do you mean by especially the toRGB layer, you mean first finetune the 256x256 toRGB layers of G and the 256x256 fromRGB layers of D, then finetune the whole layers of 256x256 of both G and D? Did you change the learning rate?

Since you highlight the toRGB layers, I believe you treat them differently compared to regular feature layers.

Thank you for your help.