It may make sense, but it also makes no sense. CNNs already have full view of the entire input image. That's how discriminators are able to discriminate in GANs.

We added attention and observed no benefits at all in our GAN experiments.