Evolution of StyleGAN

How StyleGAN is gradually evolving up to the current version.

Still Working on it

StyleGANs are a series of generative models that generate high-quality images from a latent space (e.g. random noise).

Evolution

ProgressiveGAN

.

Progressive Growing

Generate better low-resolution images, then growing to high-resolution.

G and D are only defined once. But in each G has several to_rgb layers, which are used to generate images at different resolutions. After training, only the last to_rgb layer is used to generate the image at high resolution. The layers related to smaller resolution are trained first, and then fine-tuned together with the layers related to larger resolution.

Minibatch Standard Deviation

Encouraging the minibatches of generated and training images to show similar statistics.

From official code:

class MinibatchStdLayer(torch.nn.Module):
    def __init__(self, group_size, num_channels=1):
        super().__init__()
        self.group_size = group_size
        self.num_channels = num_channels

    def forward(self, x):
        N, C, H, W = x.shape
        with misc.suppress_tracer_warnings(): # as_tensor results are registered as constants
            G = torch.min(torch.as_tensor(self.group_size), torch.as_tensor(N)) if self.group_size is not None else N
        F = self.num_channels
        c = C // F

        y = x.reshape(G, -1, F, c, H, W)    # [GnFcHW] Split minibatch N into n groups of size G, and channels C into F groups of size c.
        y = y - y.mean(dim=0)               # [GnFcHW] Subtract mean over group.
        y = y.square().mean(dim=0)          # [nFcHW]  Calc variance over group.
        y = (y + 1e-8).sqrt()               # [nFcHW]  Calc stddev over group.
        y = y.mean(dim=[2,3,4])             # [nF]     Take average over channels and pixels.
        y = y.reshape(-1, F, 1, 1)          # [nF11]   Add missing dimensions.
        y = y.repeat(G, 1, H, W)            # [NFHW]   Replicate over group and pixels.
        x = torch.cat([x, y], dim=1)        # [NCHW]   Append to input as new channels.
        return x

Basically, it generates an additional channel with the variance of all input images of size [CHW] and append it to the input.

StyleGAN

StyleGAN2

StyleGAN2-ada

StyleGAN3

Project to latent space

It involves project a real images to the latent space of StyleGAN, which is a preliminary to munipulate image.

StyleGAN2 projector

e2e projector

Control Image

It involves control certain attribute of an image by first map the image into the latent space of StyleGAN, then move the latent space towards the targeting direction.

GANSpace

GANControl

MPG2

SeFa

InterFaceGAN

StyleSpace

StyleClip

Latent Optimization

Latent Mapper

StyleGAN of All Trades

https://github.com/mchong6/SOAT

https://mp.weixin.qq.com/s/2G9JAWmXig0o8WR1J7dfkg