MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation
Ludan Ruan, Yiyang Ma, Huan Yang, Huiguo He, Bei Liu, Jianlong Fu, Nicholas Jing Yuan, Qin Jin, Baining Guo
arXiv | December 2022
Publications
Publications by Year
-
Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning
Yuchong Sun, Hongwei Xue, Ruihua Song, Bei Liu, Huan Yang, Jianlong Fu
NeurIPS 2022 | November 2022
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions
Hongwei Xue, Tiankai Hang, Yanhong Zeng, Yuchong Sun, Bei Liu, Huan Yang, Jianlong Fu, Baining Guo
CVPR 2022 | June 2022
Debiasing Event Understanding for Visual Commonsense Tasks
Minji Seo, YeonJoon Jung, Seungtaek Choi, seung-won hwang, Bei Liu
May 2022
-
Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training
Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo
NeurIPS 2021 | December 2021
Searching the Search Space of Vision Transformer
Minghao Chen, Kan Wu, Bolin Ni, Houwen Peng, Bei Liu, Jianlong Fu, Hongyang Chao, Haibin Ling
NeurIPS 2021 | December 2021
Learning Fine-Grained Motion Embedding for Landscape Animation
Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo
ACM Multimedia 2021 | October 2021
Oral
A Picture is Worth a Thousand Words: A Unified System for Diverse Captions and Rich Images Generation
Yupan Huang, Bei Liu, Jianlong Fu, Yutong Lu
ACM Multimedia 2021 | October 2021
Unifying multimodal transformer for bi-directional image and text generation
Yupan Huang, Hongwei Xue, Bei Liu, Yutong Lu
ACM Multimedia 2021 | October 2021
MMPT’21: International Joint Workshop on Multi-Modal Pre-Training for Multimedia Understanding
Bei Liu, Jianlong Fu, Shizhe Chen, Qin Jin, Alexander Hauptmann, Yong Rui
ICMR 2021 | August 2021
Reference-Based Defect Detection Network
Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao
IEEE Transactions on Image Processing | July 2021, Vol 30: pp. 6637-6647
Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning
Zhicheng Huang*, Zhaoyang Zeng*, Yupan Huang*, Bei Liu, Dongmei Fu, Jianlong Fu
CVPR 2021 | June 2021
Oral
-
Aesthetic-Aware Image Style Transfer
ACM Multimedia | October 2020
Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers
Zhicheng Huang*, Zhaoyang Zeng*, Bei Liu, Dongmei Fu, Jianlong Fu
MSR-TR-2020-51 | April 2020
Published by Microsoft
-
Neural Storyboard Artist: Visualizing Stories with Coherent Image Sequences
Shizhe Chen, Bei Liu, Jianlong Fu, Ruihua Song, Qin Jin, Pingping Lin, Xiaoyu Qi, Chunting Wang, Jin Zhou
ACM Multimedia | October 2019
SMP Challenge: An Overview of Social Media Prediction Challenge 2019
Bo Wu, Wen-Huang Cheng, Peiye Liu, Bei Liu, Zhaoyang Zeng, Jiebo Luo
ACM Multimedia | October 2019
WSOD^2: Learning Bottom-up and Top-down Objectness Distillation for Weakly-supervised Object Detection
Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao, Lei Zhang
ICCV | October 2019
Emotion Reinforced Visual Storytelling
Nanxing Li, Bei Liu, Zhizhong Han, Yu-Shen Liu, Jianlong Fu
ICMR | June 2019
Learning Rich Image Region Representation for Visual Question Answering
Bei Liu, Zhicheng Huang, Zhaoyang Zeng, Zheyu Chen, Jianlong Fu
MSR-TR-2020-1 | June 2019
Published by Microsoft
CVPR 2019, VQA Workshop
-
Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training
Bei Liu, Jianlong Fu, Makoto P. Kato, Masatoshi Yoshikawa
2018 ACM Multimedia | October 2018
Best Paper Award
Publications by Research Area
-
MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation
Ludan Ruan, Yiyang Ma, Huan Yang, Huiguo He, Bei Liu, Jianlong Fu, Nicholas Jing Yuan, Qin Jin, Baining Guo
arXiv | December 2022
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions
Hongwei Xue, Tiankai Hang, Yanhong Zeng, Yuchong Sun, Bei Liu, Huan Yang, Jianlong Fu, Baining Guo
CVPR 2022 | June 2022
Searching the Search Space of Vision Transformer
Minghao Chen, Kan Wu, Bolin Ni, Houwen Peng, Bei Liu, Jianlong Fu, Hongyang Chao, Haibin Ling
NeurIPS 2021 | December 2021
Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training
Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo
NeurIPS 2021 | December 2021
Learning Fine-Grained Motion Embedding for Landscape Animation
Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo
ACM Multimedia 2021 | October 2021
Oral
Unifying multimodal transformer for bi-directional image and text generation
Yupan Huang, Hongwei Xue, Bei Liu, Yutong Lu
ACM Multimedia 2021 | October 2021
A Picture is Worth a Thousand Words: A Unified System for Diverse Captions and Rich Images Generation
Yupan Huang, Bei Liu, Jianlong Fu, Yutong Lu
ACM Multimedia 2021 | October 2021
MMPT’21: International Joint Workshop on Multi-Modal Pre-Training for Multimedia Understanding
Bei Liu, Jianlong Fu, Shizhe Chen, Qin Jin, Alexander Hauptmann, Yong Rui
ICMR 2021 | August 2021
Reference-Based Defect Detection Network
Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao
IEEE Transactions on Image Processing | July 2021, Vol 30: pp. 6637-6647
Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning
Zhicheng Huang*, Zhaoyang Zeng*, Yupan Huang*, Bei Liu, Dongmei Fu, Jianlong Fu
CVPR 2021 | June 2021
Oral
Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers
Zhicheng Huang*, Zhaoyang Zeng*, Bei Liu, Dongmei Fu, Jianlong Fu
MSR-TR-2020-51 | April 2020
Published by Microsoft
WSOD^2: Learning Bottom-up and Top-down Objectness Distillation for Weakly-supervised Object Detection
Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao, Lei Zhang
ICCV | October 2019
Learning Rich Image Region Representation for Visual Question Answering
Bei Liu, Zhicheng Huang, Zhaoyang Zeng, Zheyu Chen, Jianlong Fu
MSR-TR-2020-1 | June 2019
Published by Microsoft
CVPR 2019, VQA Workshop
-
Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning
Yuchong Sun, Hongwei Xue, Ruihua Song, Bei Liu, Huan Yang, Jianlong Fu
NeurIPS 2022 | November 2022
Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning
Zhicheng Huang*, Zhaoyang Zeng*, Yupan Huang*, Bei Liu, Dongmei Fu, Jianlong Fu
CVPR 2021 | June 2021
Oral
Aesthetic-Aware Image Style Transfer
ACM Multimedia | October 2020
Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers
Zhicheng Huang*, Zhaoyang Zeng*, Bei Liu, Dongmei Fu, Jianlong Fu
MSR-TR-2020-51 | April 2020
Published by Microsoft
Neural Storyboard Artist: Visualizing Stories with Coherent Image Sequences
Shizhe Chen, Bei Liu, Jianlong Fu, Ruihua Song, Qin Jin, Pingping Lin, Xiaoyu Qi, Chunting Wang, Jin Zhou
ACM Multimedia | October 2019
Learning Rich Image Region Representation for Visual Question Answering
Bei Liu, Zhicheng Huang, Zhaoyang Zeng, Zheyu Chen, Jianlong Fu
MSR-TR-2020-1 | June 2019
Published by Microsoft
CVPR 2019, VQA Workshop
Emotion Reinforced Visual Storytelling
Nanxing Li, Bei Liu, Zhizhong Han, Yu-Shen Liu, Jianlong Fu
ICMR | June 2019
Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training
Bei Liu, Jianlong Fu, Makoto P. Kato, Masatoshi Yoshikawa
2018 ACM Multimedia | October 2018
Best Paper Award
-
Debiasing Event Understanding for Visual Commonsense Tasks
Minji Seo, YeonJoon Jung, Seungtaek Choi, seung-won hwang, Bei Liu
May 2022
Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training
Bei Liu, Jianlong Fu, Makoto P. Kato, Masatoshi Yoshikawa
2018 ACM Multimedia | October 2018
Best Paper Award
-
Learning Fine-Grained Motion Embedding for Landscape Animation
Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo
ACM Multimedia 2021 | October 2021
Oral
Unifying multimodal transformer for bi-directional image and text generation
Yupan Huang, Hongwei Xue, Bei Liu, Yutong Lu
ACM Multimedia 2021 | October 2021
A Picture is Worth a Thousand Words: A Unified System for Diverse Captions and Rich Images Generation
Yupan Huang, Bei Liu, Jianlong Fu, Yutong Lu
ACM Multimedia 2021 | October 2021
MMPT’21: International Joint Workshop on Multi-Modal Pre-Training for Multimedia Understanding
Bei Liu, Jianlong Fu, Shizhe Chen, Qin Jin, Alexander Hauptmann, Yong Rui
ICMR 2021 | August 2021
SMP Challenge: An Overview of Social Media Prediction Challenge 2019
Bo Wu, Wen-Huang Cheng, Peiye Liu, Bei Liu, Zhaoyang Zeng, Jiebo Luo
ACM Multimedia | October 2019
Neural Storyboard Artist: Visualizing Stories with Coherent Image Sequences
Shizhe Chen, Bei Liu, Jianlong Fu, Ruihua Song, Qin Jin, Pingping Lin, Xiaoyu Qi, Chunting Wang, Jin Zhou
ACM Multimedia | October 2019
Emotion Reinforced Visual Storytelling
Nanxing Li, Bei Liu, Zhizhong Han, Yu-Shen Liu, Jianlong Fu
ICMR | June 2019
Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training
Bei Liu, Jianlong Fu, Makoto P. Kato, Masatoshi Yoshikawa
2018 ACM Multimedia | October 2018
Best Paper Award
Publications by Type
-
MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation
Ludan Ruan, Yiyang Ma, Huan Yang, Huiguo He, Bei Liu, Jianlong Fu, Nicholas Jing Yuan, Qin Jin, Baining Guo
arXiv | December 2022
Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning
Yuchong Sun, Hongwei Xue, Ruihua Song, Bei Liu, Huan Yang, Jianlong Fu
NeurIPS 2022 | November 2022
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions
Hongwei Xue, Tiankai Hang, Yanhong Zeng, Yuchong Sun, Bei Liu, Huan Yang, Jianlong Fu, Baining Guo
CVPR 2022 | June 2022
Searching the Search Space of Vision Transformer
Minghao Chen, Kan Wu, Bolin Ni, Houwen Peng, Bei Liu, Jianlong Fu, Hongyang Chao, Haibin Ling
NeurIPS 2021 | December 2021
Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training
Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo
NeurIPS 2021 | December 2021
Learning Fine-Grained Motion Embedding for Landscape Animation
Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo
ACM Multimedia 2021 | October 2021
Oral
Unifying multimodal transformer for bi-directional image and text generation
Yupan Huang, Hongwei Xue, Bei Liu, Yutong Lu
ACM Multimedia 2021 | October 2021
A Picture is Worth a Thousand Words: A Unified System for Diverse Captions and Rich Images Generation
Yupan Huang, Bei Liu, Jianlong Fu, Yutong Lu
ACM Multimedia 2021 | October 2021
MMPT’21: International Joint Workshop on Multi-Modal Pre-Training for Multimedia Understanding
Bei Liu, Jianlong Fu, Shizhe Chen, Qin Jin, Alexander Hauptmann, Yong Rui
ICMR 2021 | August 2021
Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning
Zhicheng Huang*, Zhaoyang Zeng*, Yupan Huang*, Bei Liu, Dongmei Fu, Jianlong Fu
CVPR 2021 | June 2021
Oral
Aesthetic-Aware Image Style Transfer
ACM Multimedia | October 2020
SMP Challenge: An Overview of Social Media Prediction Challenge 2019
Bo Wu, Wen-Huang Cheng, Peiye Liu, Bei Liu, Zhaoyang Zeng, Jiebo Luo
ACM Multimedia | October 2019
Neural Storyboard Artist: Visualizing Stories with Coherent Image Sequences
Shizhe Chen, Bei Liu, Jianlong Fu, Ruihua Song, Qin Jin, Pingping Lin, Xiaoyu Qi, Chunting Wang, Jin Zhou
ACM Multimedia | October 2019
WSOD^2: Learning Bottom-up and Top-down Objectness Distillation for Weakly-supervised Object Detection
Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao, Lei Zhang
ICCV | October 2019
Emotion Reinforced Visual Storytelling
Nanxing Li, Bei Liu, Zhizhong Han, Yu-Shen Liu, Jianlong Fu
ICMR | June 2019
Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training
Bei Liu, Jianlong Fu, Makoto P. Kato, Masatoshi Yoshikawa
2018 ACM Multimedia | October 2018
Best Paper Award
-
Debiasing Event Understanding for Visual Commonsense Tasks
Minji Seo, YeonJoon Jung, Seungtaek Choi, seung-won hwang, Bei Liu
May 2022
-
Reference-Based Defect Detection Network
Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao
IEEE Transactions on Image Processing | July 2021, Vol 30: pp. 6637-6647
-
Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers
Zhicheng Huang*, Zhaoyang Zeng*, Bei Liu, Dongmei Fu, Jianlong Fu
MSR-TR-2020-51 | April 2020
Published by Microsoft
Learning Rich Image Region Representation for Visual Question Answering
Bei Liu, Zhicheng Huang, Zhaoyang Zeng, Zheyu Chen, Jianlong Fu
MSR-TR-2020-1 | June 2019
Published by Microsoft
CVPR 2019, VQA Workshop