Unnecessary computation in backward pass

[ppwwyyxx]

In the code of WassersteinGAN, they have this line:

        for p in netD.parameters():
            p.requires_grad = False # to avoid computation

I think it means that when you train G, by default you'll compute gradients for D as well (but not updating them), and vice versa. Setting the flag to False to avoid the computation should speed up the training a lot.
I found that my tensorflow implementation runs much faster than this code, and this is probably the reason.

[eriche2016]

it is still very slow when turnning this flag off, donot know the reason.

[eriche2016]

I find that training this net on GPU for 1 iterations ,with batch size 64 on celeba , costs me nearly 30 seconds.

[ppwwyyxx]

...That's probably your setup problem. It should be on the order of 0.4 seconds on a good GPU.

[eriche2016]

Hi, I use the default setup of the parameters. And my GPU has a memory of 12G. I donot know the reason why it is too slow..

[jazzsaxmafia]

Thank you for the tip! We were not aware of that.