Neural Crossbreed: Neural Based Image Metamorphosis

Neural Crossbreed: Neural Based Image Metamorphosis

We propose Neural Crossbreed, a feed-forward neural network that can learn a semantic change of input images in a latent space to create the morphing effect. Because the network learns a semantic change, a sequence of meaningful intermediate images can be generated without requiring the user to specify explicit correspondences.In addition, the semantic change learning makes it possible to perform the morphing between the images that contain objects with significantly different poses or camera views. Furthermore, just as in conventional morphing techniques, our morphing network can handle shape and appearance transitions separately by disentangling the content and the style transfer for rich usability. We prepare a training dataset for morphing using a pre-trained BigGAN, which generates an intermediate image by interpolating two latent vectors at an intended morphing value. This is the first attempt to address image morphing using a pre-trained generative model in order to learn semantic transformation. The experiments show that Neural Crossbreed produces high quality morphed images, overcoming various limitations associated with conventional approaches. In addition, Neural Crossbreed can be further extended for diverse applications such as multi-image morphing, appearance transfer, and video frame interpolation.

神经杂交：基于神经的图像变形

我们提出了神经杂交技术，它是一种前馈神经网络，可以学习潜在空间中输入图像的语义变化以产生变形效果。由于网络学习了语义变化，因此可以生成一系列有意义的中间图像，而无需用户指定明确的对应关系。.. 此外，语义变化学习使在包含姿势或相机视图明显不同的对象的图像之间执行变形成为可能。此外，就像传统的变形技术一样，我们的变形网络可以通过解开内容和样式转移来分别处理形状和外观转换，以实现丰富的可用性。我们准备了一个训练数据集，用于使用预先训练的BigGAN进行变形，该BigGAN通过在预期的变形值处插入两个潜在矢量来生成中间图像。这是使用预训练的生成模型解决图像变形的首次尝试，以便学习语义转换。实验表明，神经杂交可以产生高质量的变形图像，克服了传统方法的各种局限性。此外， （阅读更多）