Linjie Li

Researcher

论文与出版物

查看方式：

- VCode: a Multimodal Coding Benchmark with SVG as Symbolic Visual Representation
  
  Kevin Qinghong Lin, Yuhao Zheng, Hangyu Ran, Dantong Zhu, Dongxing Mao, Linjie Li, Philip H. S. Torr, Alex Jinpeng Wang
  
  November 2025
  
  Publication
- VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents
  
  Kangrui Wang, Pingyue Zhang, Zihan Wang, Yaning Gao, Linjie Li, Qineng Wang, Hanyang Chen, Chi Wan, Yiping Lu, Zhengyuan Yang, Lijuan Wang, Ranjay Krishna, Jiajun Wu, Fei-Fei Li, Yejin Choi, Manling Li
  
  NeurIPS 2025 | October 2025
  
  Publication
- EdiVal-Agent: An Object-Centric Framework for Automated, Scalable, Fine-Grained Evaluation of Multi-Turn Editing
  
  Tianyu Chen, Yasi Zhang, Zhi Zhang, Peiyu Yu, Shu Wang, Zhendong Wang, Kevin Lin, Xiaofei Wang, Zhengyuan Yang, Linjie Li, Chung-Ching Lin, Jianwen Xie, Oscar Leong, Lijuan Wang, Ying Nian Wu, Mingyuan Zhou
  
  September 2025
  
  Publication
- ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs
  
  Xiyao Wang, Zhengyuan Yang, Chao Feng, Yongyuan Liang, Yuhang Zhou, Xiaoyu Liu, Ziyi Zang, Ming Li, Chung-Ching Lin, Kevin Lin, Linjie Li, Furong Huang, Lijuan Wang
  
  NeurIPS 2025 | June 2025
  
  DOI Publication Publication
- Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning
  
  Minheng Ni, Zhengyuan Yang, Linjie Li, Chung-Ching Lin, Kevin Lin, Wangmeng Zuo, Lijuan Wang
  
  NeurIPS 2025 | May 2025
  
  DOI Publication Publication
- SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement
  
  Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Lin, Furong Huang, Lijuan Wang
  
  NeurIPS 2025 | April 2025
  
  DOI Publication Publication
- Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback
  
  Yafu Li, Xuyang Hu, Xiaoye Qu, Linjie Li, Yu Cheng
  
  ICML 2025 | January 2025
  
  DOI Publication Publication
- Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark
  
  Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, Yu Cheng
  
  ICML 2025 | January 2025
  
  DOI Publication Publication
- GenXD: Generating Any 3D and 4D Scenes
  
  Yuyang Zhao, Chung-Ching Lin, K. Lin, Zhiwen Yan, Linjie Li, Zhengyuan Yang, Jianfeng Wang, Gim Hee Lee, Lijuan Wang
  
  ICLR 2025 | November 2024
  
  DOI Publication Publication
- SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation
  
  Yining Hong, Beide Liu, Maxine Wu, Yuanhao Zhai, Kai-Wei Chang, Linjie Li, K. Lin, Chung-Ching Lin, Jianfeng Wang, Zhengyuan Yang, Yingnian Wu, Lijuan Wang
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models
  
  Peng Xia, Siwei Han, Shi Qiu, Yiyang Zhou, Zhaoyang Wang, Wenhao Zheng, Zhaorun Chen, Chenhang Cui, Mingyu Ding, Linjie Li, Lijuan Wang, Huaxiu Yao
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing
  
  Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, K. Lin, Jianfeng Wang, Lijuan Wang, Xin Eric Wang, Linjie Li
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- What can Foundation Models’ Embeddings do?
  
  Linjie Li, Jianwei Yang, Lijuan Wang
  
  October 2024
- Bring Metric Functions into Diffusion Models
  
  Jie An, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Zicheng Liu, Lijuan Wang, Jiebo Luo
  
  IJCAI 2024 | August 2024
  
  DOI Publication
- Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness
  
  Khyathi Raghavi Chandu, Linjie Li, Anas Awadalla, Ximing Lu, J. Park, Jack Hessel, Lijuan Wang, Yejin Choi
  
  ICLR 2025 | July 2024
  
  DOI Publication Publication
- VideoGUI: A Benchmark for GUI Automation from Instructional Videos
  
  Kevin Qinghong Lin, Linjie Li, Difei Gao, Qinchen Wu, Mingyi Yan, Zhengyuan Yang, Lijuan Wang, Mike Zheng Shou
  
  NeurIPS 2024 | June 2024
  
  DOI Publication Publication
- MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos
  
  Xuehai He, Weixi Feng, Kaizhi Zheng, Yujie Lu, Wanrong Zhu, Jiachen Li, Yue Fan, Jianfeng Wang, Linjie Li, Zhengyuan Yang, K. Lin, William Yang Wang, Lijuan Wang, Xin Eric Wang
  
  ICLR 2025 | June 2024
  
  DOI Publication Publication
- MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning
  
  Chaoyi Zhang, Kevin Lin, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Chung-Ching Lin, Zicheng Liu, Lijuan Wang
  
  CVPR 2024 | November 2023
  
  DOI Publication Publication
- The Generative AI Paradox: “What It Can Create, It May Not Understand”
  
  Peter West, Ximing Lu, Nouha Dziri, Faeze Brahman, Linjie Li, Jena D. Hwang, Liwei Jiang, Jillian R. Fisher, Abhilasha Ravichander, Khyathi Raghavi Chandu, Benjamin Newman, Pang Wei Koh, Allyson Ettinger, Yejin Choi
  
  ICLR 2024 | October 2023
  
  DOI Publication
- Multimodal Foundation Models: From Specialists to General-Purpose Assistants
  
  Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, Jianfeng Gao
  
  September 2023
  
  118-page book/survey on the literature review, evolution, trends and our position on multimodal foundation models. CVPR 2023 Tutorial: https://vlp-tutorial.github.io/2023/
  
  Publication 项目
- MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities
  
  Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, Lijuan Wang
  
  ICML 2024 | August 2023
  
  DOI Publication PDF Publication
- DisCo: Disentangled Control for Realistic Human Dance Generation
  
  Tan Wang, Linjie Li, Kevin Lin, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, Lijuan Wang
  
  CVPR 2024 | June 2023
  
  Publication
- Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
  
  Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Y. Yacoob, Lijuan Wang
  
  ICLR 2024 | June 2023
  
  Publication
- MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos
  
  Jielin Qiu, Jiacheng Zhu, William Jongwon Han, Aditesh Kumar, Karthik Mittal, Claire Jin, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Ding Zhao, Bo Li, Lijuan Wang
  
  CVPR 2024 | June 2023
  
  Publication
- Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone
  
  Zi-Yi Dou, Aishwarya Kamath, Zhe Gan, Pengchuan Zhang, Jianfeng Wang, Linjie Li, Zicheng Liu, Ce Liu, Yann LeCun, Nanyun Peng, Jianfeng Gao, Lijuan Wang
  
  NeurIPS 2022 | November 2022
  
  Publication 项目
- SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning
  
  Kevin Lin, Linjie Li, Chung-Ching Lin, Faisal Ahmed, Zhe Gan, Zicheng Liu, Yumao Lu, Lijuan Wang
  
  CVPR 2022 | June 2022
  
  Publication 项目
- Cross-modal Representation Learning for Zero-shot Action Recognition
  
  Chung-Ching Lin, Kevin Lin, Linjie Li, Lijuan Wang, Zicheng Liu
  
  2022 Computer Vision and Pattern Recognition | May 2022
  
  Publication Publication
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication 项目
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint 项目
- VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
  
  Linjie Li, Jie Lei, Zhe Gan, Licheng Yu, Yen-Chun Chen, Rohit Pillai, Yu Cheng, Luowei Zhou, Xin Eric Wang, William Yang Wang, Tamara Lee Berg, Mohit Bansal, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  NeurIPS 2021 | December 2021
  
  Publication 项目
- VIOLET: End-to-End Video-Language Transformers with Masked Visual-token Modeling
  
  Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, Zicheng Liu
  
  arXiv preprint arXiv:2111.12681 | November 2021
  
  Publication 项目
- Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models
  
  Linjie Li, Jie Lei, Zhe Gan, Jingjing Liu
  
  ICCV 2021 | October 2021
  
  Publication 项目
- Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling
  
  Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L. Berg, Mohit Bansal, Jingjing Liu
  
  CVPR 2021 | June 2021
  
  CVPR 2021 Best Paper
  
  Publication Github 项目
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
  
  Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, JJ (Jingjing) Liu
  
  March 2021
  
  Publication
- Meta Module Network for Compositional Visual Reasoning
  
  Wenhu Chen, Zhe Gan, Linjie Li, Yu Cheng, William Yang Wang, JJ (Jingjing) Liu
  
  Winter Conference on Applications of Computer Vision (WACV 2021) | March 2021
  
  Publication
- A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
  
  Linjie Li, Zhe Gan, Jingjing Liu
  
  December 2020
  
  Publication 项目
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, JJ (Jingjing) Liu
  
  NeurIPS 2020 | December 2020
  
  PDF 项目
- HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
  
  Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu, Jingjing Liu
  
  Conf. on Empirical Methods in Natural Language Processing (EMNLP 2020) | November 2020
  
  Publication 项目
- UNITER: UNiversal Image-TExt Representation Learning
  
  Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, JJ (Jingjing) Liu
  
  ECCV 2020 | August 2020
  
  PDF 项目
- Graph Optimal Transport for Cross-Domain Alignment.
  
  Liqun Chen, Zhe Gan, Yu Cheng, Linjie Li, Lawrence Carin, JJ (Jingjing) Liu
  
  International Conference on Machine Learning (ICML 2020) | July 2020
  
  PDF

- VCode: a Multimodal Coding Benchmark with SVG as Symbolic Visual Representation
  
  Kevin Qinghong Lin, Yuhao Zheng, Hangyu Ran, Dantong Zhu, Dongxing Mao, Linjie Li, Philip H. S. Torr, Alex Jinpeng Wang
  
  November 2025
  
  Publication
- EdiVal-Agent: An Object-Centric Framework for Automated, Scalable, Fine-Grained Evaluation of Multi-Turn Editing
  
  Tianyu Chen, Yasi Zhang, Zhi Zhang, Peiyu Yu, Shu Wang, Zhendong Wang, Kevin Lin, Xiaofei Wang, Zhengyuan Yang, Linjie Li, Chung-Ching Lin, Jianwen Xie, Oscar Leong, Lijuan Wang, Ying Nian Wu, Mingyuan Zhou
  
  September 2025
  
  Publication
- GenXD: Generating Any 3D and 4D Scenes
  
  Yuyang Zhao, Chung-Ching Lin, K. Lin, Zhiwen Yan, Linjie Li, Zhengyuan Yang, Jianfeng Wang, Gim Hee Lee, Lijuan Wang
  
  ICLR 2025 | November 2024
  
  DOI Publication Publication
- SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation
  
  Yining Hong, Beide Liu, Maxine Wu, Yuanhao Zhai, Kai-Wei Chang, Linjie Li, K. Lin, Chung-Ching Lin, Jianfeng Wang, Zhengyuan Yang, Yingnian Wu, Lijuan Wang
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models
  
  Peng Xia, Siwei Han, Shi Qiu, Yiyang Zhou, Zhaoyang Wang, Wenhao Zheng, Zhaorun Chen, Chenhang Cui, Mingyu Ding, Linjie Li, Lijuan Wang, Huaxiu Yao
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing
  
  Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, K. Lin, Jianfeng Wang, Lijuan Wang, Xin Eric Wang, Linjie Li
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness
  
  Khyathi Raghavi Chandu, Linjie Li, Anas Awadalla, Ximing Lu, J. Park, Jack Hessel, Lijuan Wang, Yejin Choi
  
  ICLR 2025 | July 2024
  
  DOI Publication Publication
- MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos
  
  Xuehai He, Weixi Feng, Kaizhi Zheng, Yujie Lu, Wanrong Zhu, Jiachen Li, Yue Fan, Jianfeng Wang, Linjie Li, Zhengyuan Yang, K. Lin, William Yang Wang, Lijuan Wang, Xin Eric Wang
  
  ICLR 2025 | June 2024
  
  DOI Publication Publication
- MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning
  
  Chaoyi Zhang, Kevin Lin, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Chung-Ching Lin, Zicheng Liu, Lijuan Wang
  
  CVPR 2024 | November 2023
  
  DOI Publication Publication
- DisCo: Disentangled Control for Realistic Human Dance Generation
  
  Tan Wang, Linjie Li, Kevin Lin, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, Lijuan Wang
  
  CVPR 2024 | June 2023
  
  Publication
- MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos
  
  Jielin Qiu, Jiacheng Zhu, William Jongwon Han, Aditesh Kumar, Karthik Mittal, Claire Jin, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Ding Zhao, Bo Li, Lijuan Wang
  
  CVPR 2024 | June 2023
  
  Publication
- Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone
  
  Zi-Yi Dou, Aishwarya Kamath, Zhe Gan, Pengchuan Zhang, Jianfeng Wang, Linjie Li, Zicheng Liu, Ce Liu, Yann LeCun, Nanyun Peng, Jianfeng Gao, Lijuan Wang
  
  NeurIPS 2022 | November 2022
  
  Publication 项目
- SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning
  
  Kevin Lin, Linjie Li, Chung-Ching Lin, Faisal Ahmed, Zhe Gan, Zicheng Liu, Yumao Lu, Lijuan Wang
  
  CVPR 2022 | June 2022
  
  Publication 项目
- Cross-modal Representation Learning for Zero-shot Action Recognition
  
  Chung-Ching Lin, Kevin Lin, Linjie Li, Lijuan Wang, Zicheng Liu
  
  2022 Computer Vision and Pattern Recognition | May 2022
  
  Publication Publication
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication 项目
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint 项目
- VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
  
  Linjie Li, Jie Lei, Zhe Gan, Licheng Yu, Yen-Chun Chen, Rohit Pillai, Yu Cheng, Luowei Zhou, Xin Eric Wang, William Yang Wang, Tamara Lee Berg, Mohit Bansal, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  NeurIPS 2021 | December 2021
  
  Publication 项目
- VIOLET: End-to-End Video-Language Transformers with Masked Visual-token Modeling
  
  Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, Zicheng Liu
  
  arXiv preprint arXiv:2111.12681 | November 2021
  
  Publication 项目
- Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models
  
  Linjie Li, Jie Lei, Zhe Gan, Jingjing Liu
  
  ICCV 2021 | October 2021
  
  Publication 项目
- Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling
  
  Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L. Berg, Mohit Bansal, Jingjing Liu
  
  CVPR 2021 | June 2021
  
  CVPR 2021 Best Paper
  
  Publication Github 项目
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- Meta Module Network for Compositional Visual Reasoning
  
  Wenhu Chen, Zhe Gan, Linjie Li, Yu Cheng, William Yang Wang, JJ (Jingjing) Liu
  
  Winter Conference on Applications of Computer Vision (WACV 2021) | March 2021
  
  Publication
- A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
  
  Linjie Li, Zhe Gan, Jingjing Liu
  
  December 2020
  
  Publication 项目
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, JJ (Jingjing) Liu
  
  NeurIPS 2020 | December 2020
  
  PDF 项目
- HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
  
  Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu, Jingjing Liu
  
  Conf. on Empirical Methods in Natural Language Processing (EMNLP 2020) | November 2020
  
  Publication 项目
- UNITER: UNiversal Image-TExt Representation Learning
  
  Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, JJ (Jingjing) Liu
  
  ECCV 2020 | August 2020
  
  PDF 项目
- VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents
  
  Kangrui Wang, Pingyue Zhang, Zihan Wang, Yaning Gao, Linjie Li, Qineng Wang, Hanyang Chen, Chi Wan, Yiping Lu, Zhengyuan Yang, Lijuan Wang, Ranjay Krishna, Jiajun Wu, Fei-Fei Li, Yejin Choi, Manling Li
  
  NeurIPS 2025 | October 2025
  
  Publication
- ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs
  
  Xiyao Wang, Zhengyuan Yang, Chao Feng, Yongyuan Liang, Yuhang Zhou, Xiaoyu Liu, Ziyi Zang, Ming Li, Chung-Ching Lin, Kevin Lin, Linjie Li, Furong Huang, Lijuan Wang
  
  NeurIPS 2025 | June 2025
  
  DOI Publication Publication
- Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning
  
  Minheng Ni, Zhengyuan Yang, Linjie Li, Chung-Ching Lin, Kevin Lin, Wangmeng Zuo, Lijuan Wang
  
  NeurIPS 2025 | May 2025
  
  DOI Publication Publication
- SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement
  
  Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Lin, Furong Huang, Lijuan Wang
  
  NeurIPS 2025 | April 2025
  
  DOI Publication Publication
- Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback
  
  Yafu Li, Xuyang Hu, Xiaoye Qu, Linjie Li, Yu Cheng
  
  ICML 2025 | January 2025
  
  DOI Publication Publication
- Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark
  
  Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, Yu Cheng
  
  ICML 2025 | January 2025
  
  DOI Publication Publication
- GenXD: Generating Any 3D and 4D Scenes
  
  Yuyang Zhao, Chung-Ching Lin, K. Lin, Zhiwen Yan, Linjie Li, Zhengyuan Yang, Jianfeng Wang, Gim Hee Lee, Lijuan Wang
  
  ICLR 2025 | November 2024
  
  DOI Publication Publication
- SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation
  
  Yining Hong, Beide Liu, Maxine Wu, Yuanhao Zhai, Kai-Wei Chang, Linjie Li, K. Lin, Chung-Ching Lin, Jianfeng Wang, Zhengyuan Yang, Yingnian Wu, Lijuan Wang
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models
  
  Peng Xia, Siwei Han, Shi Qiu, Yiyang Zhou, Zhaoyang Wang, Wenhao Zheng, Zhaorun Chen, Chenhang Cui, Mingyu Ding, Linjie Li, Lijuan Wang, Huaxiu Yao
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing
  
  Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, K. Lin, Jianfeng Wang, Lijuan Wang, Xin Eric Wang, Linjie Li
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- What can Foundation Models’ Embeddings do?
  
  Linjie Li, Jianwei Yang, Lijuan Wang
  
  October 2024
- Bring Metric Functions into Diffusion Models
  
  Jie An, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Zicheng Liu, Lijuan Wang, Jiebo Luo
  
  IJCAI 2024 | August 2024
  
  DOI Publication
- Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness
  
  Khyathi Raghavi Chandu, Linjie Li, Anas Awadalla, Ximing Lu, J. Park, Jack Hessel, Lijuan Wang, Yejin Choi
  
  ICLR 2025 | July 2024
  
  DOI Publication Publication
- VideoGUI: A Benchmark for GUI Automation from Instructional Videos
  
  Kevin Qinghong Lin, Linjie Li, Difei Gao, Qinchen Wu, Mingyi Yan, Zhengyuan Yang, Lijuan Wang, Mike Zheng Shou
  
  NeurIPS 2024 | June 2024
  
  DOI Publication Publication
- MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos
  
  Xuehai He, Weixi Feng, Kaizhi Zheng, Yujie Lu, Wanrong Zhu, Jiachen Li, Yue Fan, Jianfeng Wang, Linjie Li, Zhengyuan Yang, K. Lin, William Yang Wang, Lijuan Wang, Xin Eric Wang
  
  ICLR 2025 | June 2024
  
  DOI Publication Publication
- The Generative AI Paradox: “What It Can Create, It May Not Understand”
  
  Peter West, Ximing Lu, Nouha Dziri, Faeze Brahman, Linjie Li, Jena D. Hwang, Liwei Jiang, Jillian R. Fisher, Abhilasha Ravichander, Khyathi Raghavi Chandu, Benjamin Newman, Pang Wei Koh, Allyson Ettinger, Yejin Choi
  
  ICLR 2024 | October 2023
  
  DOI Publication
- Multimodal Foundation Models: From Specialists to General-Purpose Assistants
  
  Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, Jianfeng Gao
  
  September 2023
  
  118-page book/survey on the literature review, evolution, trends and our position on multimodal foundation models. CVPR 2023 Tutorial: https://vlp-tutorial.github.io/2023/
  
  Publication 项目
- MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities
  
  Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, Lijuan Wang
  
  ICML 2024 | August 2023
  
  DOI Publication PDF Publication
- Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
  
  Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Y. Yacoob, Lijuan Wang
  
  ICLR 2024 | June 2023
  
  Publication
- Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone
  
  Zi-Yi Dou, Aishwarya Kamath, Zhe Gan, Pengchuan Zhang, Jianfeng Wang, Linjie Li, Zicheng Liu, Ce Liu, Yann LeCun, Nanyun Peng, Jianfeng Gao, Lijuan Wang
  
  NeurIPS 2022 | November 2022
  
  Publication 项目
- SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning
  
  Kevin Lin, Linjie Li, Chung-Ching Lin, Faisal Ahmed, Zhe Gan, Zicheng Liu, Yumao Lu, Lijuan Wang
  
  CVPR 2022 | June 2022
  
  Publication 项目
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication 项目
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint 项目
- VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
  
  Linjie Li, Jie Lei, Zhe Gan, Licheng Yu, Yen-Chun Chen, Rohit Pillai, Yu Cheng, Luowei Zhou, Xin Eric Wang, William Yang Wang, Tamara Lee Berg, Mohit Bansal, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  NeurIPS 2021 | December 2021
  
  Publication 项目
- VIOLET: End-to-End Video-Language Transformers with Masked Visual-token Modeling
  
  Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, Zicheng Liu
  
  arXiv preprint arXiv:2111.12681 | November 2021
  
  Publication 项目
- Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling
  
  Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L. Berg, Mohit Bansal, Jingjing Liu
  
  CVPR 2021 | June 2021
  
  CVPR 2021 Best Paper
  
  Publication Github 项目
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
  
  Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, JJ (Jingjing) Liu
  
  March 2021
  
  Publication
- Meta Module Network for Compositional Visual Reasoning
  
  Wenhu Chen, Zhe Gan, Linjie Li, Yu Cheng, William Yang Wang, JJ (Jingjing) Liu
  
  Winter Conference on Applications of Computer Vision (WACV 2021) | March 2021
  
  Publication
- A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
  
  Linjie Li, Zhe Gan, Jingjing Liu
  
  December 2020
  
  Publication 项目
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, JJ (Jingjing) Liu
  
  NeurIPS 2020 | December 2020
  
  PDF 项目
- HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
  
  Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu, Jingjing Liu
  
  Conf. on Empirical Methods in Natural Language Processing (EMNLP 2020) | November 2020
  
  Publication 项目
- UNITER: UNiversal Image-TExt Representation Learning
  
  Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, JJ (Jingjing) Liu
  
  ECCV 2020 | August 2020
  
  PDF 项目
- Graph Optimal Transport for Cross-Domain Alignment.
  
  Liqun Chen, Zhe Gan, Yu Cheng, Linjie Li, Lawrence Carin, JJ (Jingjing) Liu
  
  International Conference on Machine Learning (ICML 2020) | July 2020
  
  PDF
- EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing
  
  Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, K. Lin, Jianfeng Wang, Lijuan Wang, Xin Eric Wang, Linjie Li
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing
  
  Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, K. Lin, Jianfeng Wang, Lijuan Wang, Xin Eric Wang, Linjie Li
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning
  
  Kevin Lin, Linjie Li, Chung-Ching Lin, Faisal Ahmed, Zhe Gan, Zicheng Liu, Yumao Lu, Lijuan Wang
  
  CVPR 2022 | June 2022
  
  Publication 项目
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication 项目
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint 项目
- VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
  
  Linjie Li, Jie Lei, Zhe Gan, Licheng Yu, Yen-Chun Chen, Rohit Pillai, Yu Cheng, Luowei Zhou, Xin Eric Wang, William Yang Wang, Tamara Lee Berg, Mohit Bansal, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  NeurIPS 2021 | December 2021
  
  Publication 项目
- VIOLET: End-to-End Video-Language Transformers with Masked Visual-token Modeling
  
  Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, Zicheng Liu
  
  arXiv preprint arXiv:2111.12681 | November 2021
  
  Publication 项目
- Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling
  
  Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L. Berg, Mohit Bansal, Jingjing Liu
  
  CVPR 2021 | June 2021
  
  CVPR 2021 Best Paper
  
  Publication Github 项目
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
  
  Linjie Li, Zhe Gan, Jingjing Liu
  
  December 2020
  
  Publication 项目

- VCode: a Multimodal Coding Benchmark with SVG as Symbolic Visual Representation
  
  Kevin Qinghong Lin, Yuhao Zheng, Hangyu Ran, Dantong Zhu, Dongxing Mao, Linjie Li, Philip H. S. Torr, Alex Jinpeng Wang
  
  November 2025
  
  Publication
- What can Foundation Models’ Embeddings do?
  
  Linjie Li, Jianwei Yang, Lijuan Wang
  
  October 2024
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint 项目
- VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents
  
  Kangrui Wang, Pingyue Zhang, Zihan Wang, Yaning Gao, Linjie Li, Qineng Wang, Hanyang Chen, Chi Wan, Yiping Lu, Zhengyuan Yang, Lijuan Wang, Ranjay Krishna, Jiajun Wu, Fei-Fei Li, Yejin Choi, Manling Li
  
  NeurIPS 2025 | October 2025
  
  Publication
- ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs
  
  Xiyao Wang, Zhengyuan Yang, Chao Feng, Yongyuan Liang, Yuhang Zhou, Xiaoyu Liu, Ziyi Zang, Ming Li, Chung-Ching Lin, Kevin Lin, Linjie Li, Furong Huang, Lijuan Wang
  
  NeurIPS 2025 | June 2025
  
  DOI Publication Publication
- Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning
  
  Minheng Ni, Zhengyuan Yang, Linjie Li, Chung-Ching Lin, Kevin Lin, Wangmeng Zuo, Lijuan Wang
  
  NeurIPS 2025 | May 2025
  
  DOI Publication Publication
- SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement
  
  Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Lin, Furong Huang, Lijuan Wang
  
  NeurIPS 2025 | April 2025
  
  DOI Publication Publication
- Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback
  
  Yafu Li, Xuyang Hu, Xiaoye Qu, Linjie Li, Yu Cheng
  
  ICML 2025 | January 2025
  
  DOI Publication Publication
- Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark
  
  Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, Yu Cheng
  
  ICML 2025 | January 2025
  
  DOI Publication Publication
- GenXD: Generating Any 3D and 4D Scenes
  
  Yuyang Zhao, Chung-Ching Lin, K. Lin, Zhiwen Yan, Linjie Li, Zhengyuan Yang, Jianfeng Wang, Gim Hee Lee, Lijuan Wang
  
  ICLR 2025 | November 2024
  
  DOI Publication Publication
- SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation
  
  Yining Hong, Beide Liu, Maxine Wu, Yuanhao Zhai, Kai-Wei Chang, Linjie Li, K. Lin, Chung-Ching Lin, Jianfeng Wang, Zhengyuan Yang, Yingnian Wu, Lijuan Wang
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models
  
  Peng Xia, Siwei Han, Shi Qiu, Yiyang Zhou, Zhaoyang Wang, Wenhao Zheng, Zhaorun Chen, Chenhang Cui, Mingyu Ding, Linjie Li, Lijuan Wang, Huaxiu Yao
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing
  
  Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, K. Lin, Jianfeng Wang, Lijuan Wang, Xin Eric Wang, Linjie Li
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- Bring Metric Functions into Diffusion Models
  
  Jie An, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Zicheng Liu, Lijuan Wang, Jiebo Luo
  
  IJCAI 2024 | August 2024
  
  DOI Publication
- Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness
  
  Khyathi Raghavi Chandu, Linjie Li, Anas Awadalla, Ximing Lu, J. Park, Jack Hessel, Lijuan Wang, Yejin Choi
  
  ICLR 2025 | July 2024
  
  DOI Publication Publication
- VideoGUI: A Benchmark for GUI Automation from Instructional Videos
  
  Kevin Qinghong Lin, Linjie Li, Difei Gao, Qinchen Wu, Mingyi Yan, Zhengyuan Yang, Lijuan Wang, Mike Zheng Shou
  
  NeurIPS 2024 | June 2024
  
  DOI Publication Publication
- MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos
  
  Xuehai He, Weixi Feng, Kaizhi Zheng, Yujie Lu, Wanrong Zhu, Jiachen Li, Yue Fan, Jianfeng Wang, Linjie Li, Zhengyuan Yang, K. Lin, William Yang Wang, Lijuan Wang, Xin Eric Wang
  
  ICLR 2025 | June 2024
  
  DOI Publication Publication
- MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning
  
  Chaoyi Zhang, Kevin Lin, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Chung-Ching Lin, Zicheng Liu, Lijuan Wang
  
  CVPR 2024 | November 2023
  
  DOI Publication Publication
- The Generative AI Paradox: “What It Can Create, It May Not Understand”
  
  Peter West, Ximing Lu, Nouha Dziri, Faeze Brahman, Linjie Li, Jena D. Hwang, Liwei Jiang, Jillian R. Fisher, Abhilasha Ravichander, Khyathi Raghavi Chandu, Benjamin Newman, Pang Wei Koh, Allyson Ettinger, Yejin Choi
  
  ICLR 2024 | October 2023
  
  DOI Publication
- MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities
  
  Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, Lijuan Wang
  
  ICML 2024 | August 2023
  
  DOI Publication PDF Publication
- DisCo: Disentangled Control for Realistic Human Dance Generation
  
  Tan Wang, Linjie Li, Kevin Lin, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, Lijuan Wang
  
  CVPR 2024 | June 2023
  
  Publication
- Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
  
  Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Y. Yacoob, Lijuan Wang
  
  ICLR 2024 | June 2023
  
  Publication
- MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos
  
  Jielin Qiu, Jiacheng Zhu, William Jongwon Han, Aditesh Kumar, Karthik Mittal, Claire Jin, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Ding Zhao, Bo Li, Lijuan Wang
  
  CVPR 2024 | June 2023
  
  Publication
- Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone
  
  Zi-Yi Dou, Aishwarya Kamath, Zhe Gan, Pengchuan Zhang, Jianfeng Wang, Linjie Li, Zicheng Liu, Ce Liu, Yann LeCun, Nanyun Peng, Jianfeng Gao, Lijuan Wang
  
  NeurIPS 2022 | November 2022
  
  Publication 项目
- SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning
  
  Kevin Lin, Linjie Li, Chung-Ching Lin, Faisal Ahmed, Zhe Gan, Zicheng Liu, Yumao Lu, Lijuan Wang
  
  CVPR 2022 | June 2022
  
  Publication 项目
- Cross-modal Representation Learning for Zero-shot Action Recognition
  
  Chung-Ching Lin, Kevin Lin, Linjie Li, Lijuan Wang, Zicheng Liu
  
  2022 Computer Vision and Pattern Recognition | May 2022
  
  Publication Publication
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication 项目
- VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
  
  Linjie Li, Jie Lei, Zhe Gan, Licheng Yu, Yen-Chun Chen, Rohit Pillai, Yu Cheng, Luowei Zhou, Xin Eric Wang, William Yang Wang, Tamara Lee Berg, Mohit Bansal, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  NeurIPS 2021 | December 2021
  
  Publication 项目
- Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models
  
  Linjie Li, Jie Lei, Zhe Gan, Jingjing Liu
  
  ICCV 2021 | October 2021
  
  Publication 项目
- Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling
  
  Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L. Berg, Mohit Bansal, Jingjing Liu
  
  CVPR 2021 | June 2021
  
  CVPR 2021 Best Paper
  
  Publication Github 项目
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- Meta Module Network for Compositional Visual Reasoning
  
  Wenhu Chen, Zhe Gan, Linjie Li, Yu Cheng, William Yang Wang, JJ (Jingjing) Liu
  
  Winter Conference on Applications of Computer Vision (WACV 2021) | March 2021
  
  Publication
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, JJ (Jingjing) Liu
  
  NeurIPS 2020 | December 2020
  
  PDF 项目
- HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
  
  Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu, Jingjing Liu
  
  Conf. on Empirical Methods in Natural Language Processing (EMNLP 2020) | November 2020
  
  Publication 项目
- UNITER: UNiversal Image-TExt Representation Learning
  
  Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, JJ (Jingjing) Liu
  
  ECCV 2020 | August 2020
  
  PDF 项目
- Graph Optimal Transport for Cross-Domain Alignment.
  
  Liqun Chen, Zhe Gan, Yu Cheng, Linjie Li, Lawrence Carin, JJ (Jingjing) Liu
  
  International Conference on Machine Learning (ICML 2020) | July 2020
  
  PDF
- EdiVal-Agent: An Object-Centric Framework for Automated, Scalable, Fine-Grained Evaluation of Multi-Turn Editing
  
  Tianyu Chen, Yasi Zhang, Zhi Zhang, Peiyu Yu, Shu Wang, Zhendong Wang, Kevin Lin, Xiaofei Wang, Zhengyuan Yang, Linjie Li, Chung-Ching Lin, Jianwen Xie, Oscar Leong, Lijuan Wang, Ying Nian Wu, Mingyuan Zhou
  
  September 2025
  
  Publication
- LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
  
  Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, JJ (Jingjing) Liu
  
  March 2021
  
  Publication
- Multimodal Foundation Models: From Specialists to General-Purpose Assistants
  
  Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, Jianfeng Gao
  
  September 2023
  
  118-page book/survey on the literature review, evolution, trends and our position on multimodal foundation models. CVPR 2023 Tutorial: https://vlp-tutorial.github.io/2023/
  
  Publication 项目
- VIOLET: End-to-End Video-Language Transformers with Masked Visual-token Modeling
  
  Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, Zicheng Liu
  
  arXiv preprint arXiv:2111.12681 | November 2021
  
  Publication 项目
- A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
  
  Linjie Li, Zhe Gan, Jingjing Liu
  
  December 2020
  
  Publication 项目

Linjie Li

论文与出版物

VCode: a Multimodal Coding Benchmark with SVG as Symbolic Visual Representation

VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents

EdiVal-Agent: An Object-Centric Framework for Automated, Scalable, Fine-Grained Evaluation of Multi-Turn Editing

ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs

Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning

SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement

Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback

Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark

GenXD: Generating Any 3D and 4D Scenes

SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation

MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models

EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing

What can Foundation Models’ Embeddings do?

Bring Metric Functions into Diffusion Models

Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness

VideoGUI: A Benchmark for GUI Automation from Instructional Videos

MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos

MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning

The Generative AI Paradox: “What It Can Create, It May Not Understand”

Multimodal Foundation Models: From Specialists to General-Purpose Assistants

MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities

DisCo: Disentangled Control for Realistic Human Dance Generation

Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning

MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos

Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone

SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning

Cross-modal Representation Learning for Zero-shot Action Recognition

Playing Lottery Tickets with Vision and Language

MLP Architectures for Vision-and-Language Modeling: An Empirical Study

VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation

VIOLET: End-to-End Video-Language Transformers with Masked Visual-token Modeling

Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models

Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling

UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training

LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval

Meta Module Network for Compositional Visual Reasoning

A Closer Look at the Robustness of Vision-and-Language Pre-trained Models

Large-Scale Adversarial Training for Vision-and-Language Representation Learning

HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training

UNITER: UNiversal Image-TExt Representation Learning

Graph Optimal Transport for Cross-Domain Alignment.

VCode: a Multimodal Coding Benchmark with SVG as Symbolic Visual Representation

EdiVal-Agent: An Object-Centric Framework for Automated, Scalable, Fine-Grained Evaluation of Multi-Turn Editing

GenXD: Generating Any 3D and 4D Scenes

SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation

MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models

EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing

Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness

MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos

MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning

DisCo: Disentangled Control for Realistic Human Dance Generation

MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos

Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone

SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning

Cross-modal Representation Learning for Zero-shot Action Recognition

Playing Lottery Tickets with Vision and Language

MLP Architectures for Vision-and-Language Modeling: An Empirical Study

VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation

VIOLET: End-to-End Video-Language Transformers with Masked Visual-token Modeling

Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models

Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling

UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training

Meta Module Network for Compositional Visual Reasoning

A Closer Look at the Robustness of Vision-and-Language Pre-trained Models

Large-Scale Adversarial Training for Vision-and-Language Representation Learning

HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training

UNITER: UNiversal Image-TExt Representation Learning

VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents

ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs

Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning

SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement

Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback

Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark

GenXD: Generating Any 3D and 4D Scenes

SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation

MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models

EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing

What can Foundation Models’ Embeddings do?