Recent Results on Learning Filters and Style Transfer

  • Ming-Hsuan Yang | University of California, Merced

In the first part of this talk, I will present recent results on learning image filters for low-level vision. We formulate numerous low-level vision problems (e.g., edge-preserving filtering and denoising) as recursive image filtering via a hybrid neural network. The network contains several spatially variant recurrent neural networks (RNN) as equivalents of a group of distinct recursive filters for each pixel, and a deep convolutional neural network (CNN) that learns the weights of the RNNs. The deep CNN can learn regulations of recurrent propagation for various tasks and effectively guides recurrent propagation over an entire image. The proposed model does not need a large number of convolutional channels nor big kernels to learn features for low-level vision filters. It is much smaller and faster compared to a deep CNN based image filter. Experimental results show that many low-level vision tasks can be effectively learned and carried out in real-time by the proposed algorithm. In addition, we show that spatial propagation can be effectively carried out in a two-dimensional manner with better results.

In the second part, I will present recent results on style transfer. I will first present an algorithm to one style transfer network from training examples of 1,000 styles. Next, I will present recent results on universal style transfer without prior learning. As most style transfer methods significantly distort the content of the input image, I will also discuss the most recent results on photorealistic style transfer.

When time allows, I will also give previews of most recent results on portraiture rendering from a monocular camera, image/video segmentation, and semi-supervised optical flow.

Speaker Details

Ming-Hsuan Yang is a professor in Electrical Engineering and Computer Science at the University of California, Merced. He received the PhD degree in Computer Science from the University of Illinois at Urbana-Champaign in 2000. He serves as an area chair for several conferences including IEEE Conference on Computer Vision and Pattern Recognition, IEEE International Conference on Computer Vision, European Conference on Computer Vision, Asian Conference on Computer, AAAI National Conference on Artificial Intelligence, and IEEE International Conference on Automatic Face and Gesture Recognition. He serves as a program co-chair for IEEE International Conference on Computer Vision in 2019 as well as Asian Conference on Computer Vision in 2014, and general co-chair for Asian Conference on Computer Vision in 2016. He serves as an associate editor of the IEEE Transactions on Pattern Analysis and Machine Intelligence (2007 to 2011), International Journal of Computer Vision, Computer Visionand Image Understanding, Image and Vision Computing, and Journal of Artificial Intelligence Research. Yang received the Google faculty award in 2009, and the Distinguished Early Career Research award from the UC Merced senate in 2011, the Faculty Early Career Development (CAREER) award from the National Science Foundation in 2012, and the Distinguished Research Award from UC Merced Senate in 2015.

Watch Next