A may-not-up-to-date Resume

selected publications

1. BMVC

   Multi-attribute Pizza Generator: Cross-domain Attribute Control with Conditional StyleGAN

   Han, Fangda, Hao, Guoyao, Guerrero, Ricardo, and Pavlovic, Vladimir

   2021

   Abs Bib HTML

   Multi-attribute conditional image generation is a challenging problem in computervision. We propose Multi-attribute Pizza Generator (MPG), a conditional Generative Neural Network (GAN) framework for synthesizing images from a trichotomy of attributes: content, view-geometry, and implicit visual style. We design MPG by extending the state-of-the-art StyleGAN2, using a new conditioning technique that guides the intermediate feature maps to learn multi-scale multi-attribute entangled representationsof controlling attributes. Because of the complex nature of the multi-attribute image generation problem, we regularize the image generation by predicting the explicit conditioning attributes (ingredients and view). To synthesize a pizza image with view attributesoutside the range of natural training images, we design a CGI pizza dataset PizzaView using 3D pizza models and employ it to train a view attribute regressor to regularize the generation process, bridging the real and CGI training datasets. To verify the efficacy of MPG, we test it on Pizza10, a carefully annotated multi-ingredient pizza image dataset. MPG can successfully generate photo-realistic pizza images with desired ingredients and view attributes, beyond the range of those observed in real-world training data.

   @misc{han2021multiattribute,
     abbr = {BMVC},
     bibtex_show = {true},
     selected = {true},
     html = {https://arxiv.org/abs/2110.11830},
     title = {Multi-attribute Pizza Generator: Cross-domain Attribute Control with Conditional StyleGAN},
     author = {Han, Fangda and Hao, Guoyao and Guerrero, Ricardo and Pavlovic, Vladimir},
     year = {2021},
     eprint = {2110.11830},
     archiveprefix = {arXiv},
     primaryclass = {cs.CV}
   }

2. AAAI

   Cross-Modal Coherence for Text-to-Image Retrieval

   Han, Fangda, Alikhani, Malihe, Ravi, Hareesh, Kapadia, Mubbasir, Pavlovic, Vladimir, and Stone, Matthew

   2021

   Abs Bib HTML

   Common image-text joint understanding techniques presume that images and the associated text can universally be characterized by a single implicit model. However, co-occurring images and text can be related in qualitatively different ways, and explicitly modeling it could improve the performance of current joint understanding models. In this paper, we train a Cross-Modal Coherence Modelfor text-to-image retrieval task. Our analysis shows that models trained with image–text coherence relations can retrieve images originally paired with target text more often than coherence-agnostic models. We also show via human evaluation that images retrieved by the proposed coherence-aware model are preferred over a coherence-agnostic baseline by a huge margin. Our findings provide insights into the ways that different modalities communicate and the role of coherence relations in capturing commonsense inferences in text and imagery.

   @misc{alikhani2021crossmodal,
     bibtex_show = {true},
     abbr = {AAAI},
     selected = {true},
     html = {https://arxiv.org/abs/2109.11047},
     title = {Cross-Modal Coherence for Text-to-Image Retrieval},
     author = {Han, Fangda and Alikhani, Malihe and Ravi, Hareesh and Kapadia, Mubbasir and Pavlovic, Vladimir and Stone, Matthew},
     year = {2021},
     eprint = {2109.11047},
     archiveprefix = {arXiv},
     primaryclass = {cs.CV},
     journal = {arXiv preprint arXiv:2109.11047}
   }

3. Arxiv

   MPG: A Multi-ingredient Pizza Image Generator with Conditional StyleGANs

   Han, Fangda, Hao, Guoyao, Guerrero, Ricardo, and Pavlovic, Vladimir

   arXiv preprint arXiv:2012.02821 2020

   Abs Bib HTML

   Multilabel conditional image generation is a challenging problem in computer vision. In this work we propose Multi-ingredient Pizza Generator (MPG), a conditional Generative Neural Network (GAN) framework for synthesizing multilabel images. We design MPG based on a state-of-the-art GAN structure called StyleGAN2, in which we develop a new conditioning technique by enforcing intermediate feature maps to learn scalewise label information. Because of the complex nature of the multilabel image generation problem, we also regularize synthetic image by predicting the corresponding ingredients as well as encourage the discriminator to distinguish between matched image and mismatched image. To verify the efficacy of MPG, we test it on Pizza10, which is a carefully annotated multi-ingredient pizza image dataset. MPG can successfully generate photo-realist pizza images with desired ingredients. The framework can be easily extend to other multilabel image generation scenarios.

   @article{han2020mpg,
     bibtex_show = {true},
     abbr = {Arxiv},
     selected = {true},
     html = {https://arxiv.org/abs/2012.02821},
     title = {MPG: A Multi-ingredient Pizza Image Generator with Conditional StyleGANs},
     author = {Han, Fangda and Hao, Guoyao and Guerrero, Ricardo and Pavlovic, Vladimir},
     journal = {arXiv preprint arXiv:2012.02821},
     year = {2020}
   }

4. WACV

   CookGAN: Meal Image Synthesis from Ingredients

   Han, Fangda, Guerrero, Ricardo, and Pavlovic, Vladimir

   In The IEEE Winter Conference on Applications of Computer Vision 2020

   Bib HTML

   @inproceedings{han2020cookgan,
     bibtex_show = {true},
     abbr = {WACV},
     selected = {true},
     html = {https://openaccess.thecvf.com/content_WACV_2020/papers/Han_CookGAN_Meal_Image_Synthesis_from_Ingredients_WACV_2020_paper.pdf},
     title = {CookGAN: Meal Image Synthesis from Ingredients},
     author = {Han, Fangda and Guerrero, Ricardo and Pavlovic, Vladimir},
     booktitle = {The IEEE Winter Conference on Applications of Computer Vision},
     pages = {1450--1458},
     year = {2020}
   }