Recent Advances in Unsupervised Image-to-Image Translation

  • Xun Huang | Cornell University

Unsupervised image-to-image translation aims to map an image drawn from one distribution to an analogous image in a different distribution, without seeing any example pairs of analogous images. For example, given an image of a landscape taken in the summer, one may want to know what it would look like in the winter. There is not just a single answer. One could imagine many possibilities due to differences in weather, timing, lighting, etc. However, existing work can only deterministically produce a single output given the same input. To address this limitation, we propose a Multimodal Unsupervised Image-to-image Translation (MUNIT) framework that is able to produce diverse and realistic translation results. We further extend our model to the few-shot scenario, where only a few images in the target distribution are available and only at test time. This model, named FUNIT, is trained to translate images between many different pairs of distributions using a few examples so that it can be generalized to unseen target distributions. Extensive experimental comparisons demonstrate the effectiveness of the proposed frameworks.

[Slides]

Series: Microsoft Research Talks