Xiaodong Liu

Researcher

Publications

View by:

- SAS: Simulated Attention Score
  
  Chuanyang Zheng, Jiankai Sun, Yihang Gao, Yuehao Wang, Peihao Wang, Jing Xiong, Liliang Ren, Hao Cheng, Janardhan (Jana) Kulkarni, Yelong Shen, Atlas Wang, Mac Schwager, Anderson Schneider, Xiaodong Liu, Jianfeng Gao
  
  NeurIPS 2025 | July 2025
  
  DOI Publication
- Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities
  
  Chung-En Sun, Xiaodong Liu, Weiwei Yang, Tsui-Wei Weng, Hao Cheng, Aidan San, Michel Galley, Jianfeng Gao
  
  NAACL 2025 | April 2025
  
  Selected for oral presentation
  
  Publication
- Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass
  
  Tong Chen, Hao Fang, Patrick Xia, Xiaodong Liu, Ben Van Durme, Luke Zettlemoyer, Jianfeng Gao, Hao Cheng
  
  ICLR 2025 | November 2024
  
  DOI Publication Publication
- Bridging Discrete and Backpropagation: Straight-Through and Beyond
  
  Liyuan Liu, Chengyu Dong, Xiaodong Liu, Bin Yu, Jianfeng Gao
  
  NeurIPS 2023 (Oral) | December 2023
  
  Publication Github
- Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs
  
  Qingru Zhang, Chandan Singh, Liyuan Liu, Xiaodong Liu, Bin Yu, Jianfeng Gao, Tuo Zhao
  
  ICLR 2024 | November 2023
  
  DOI Publication
- Fast-ELECTRA for Efficient Pre-training
  
  Chengyu Dong, Liyuan Liu, Hao Cheng, Jingbo Shang, Jianfeng Gao, Xiaodong Liu
  
  ICLR 2024 | October 2023
  
  DOI Publication
- Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers
  
  Linyuan Gong, Chenyan Xiong, Xiaodong Liu, Payal Bajaj, Yiqing Xie, Alvin Cheung, Jianfeng Gao, Xia Song
  
  ACL 2023 | July 2023
  
  Publication
- Task-Aware Specialization for Efficient and Robust Dense Retrieval for Open-Domain Question Answering
  
  Hao Cheng, Hao Fang, Xiaodong Liu, Jianfeng Gao
  
  ACL 2023 | July 2023
  
  Publication
- AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation
  
  Ganesh Jawahar, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Young Jin Kim, Muhammad Abdul-Mageed, Laks Lakshmanan, V.S., Ahmed Awadallah, Sébastien Bubeck, Jianfeng Gao
  
  ACL 2023 | June 2023
  
  Publication
- Fine-tuning large neural language models for biomedical natural language processing
  
  Rob Tinn, Hao Cheng, Yu Gu, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon
  
  Patterns | June 2023, Vol 4(4)
  
  Publication Publication
- Chain-of-Skills: A Configurable Model for Open-domain Question Answering
  
  Kaixin Ma, Hao Cheng, Yu Zhang, Xiaodong Liu, Eric Nyberg, Jianfeng Gao
  
  ACL 2023 | May 2023
  
  Publication
- AutoMoE: Neural Architecture Search for Efficient Sparsely Activated Transformers
  
  Ganesh Jawahar, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Young Jin Kim, Muhammad Abdul-Mageed, Laks V. S. Lakshmanan, Ahmed Awadallah, Sébastien Bubeck, Jianfeng Gao
  
  ACL | May 2023
  
  Publication Project
- Efficient Long Sequence Modeling via State Space Augmented Transformer
  
  Simiao Zuo, Xiaodong Liu, Jian Jiao, Denis Charles, Eren Manavoglu, Tuo Zhao, Jianfeng Gao
  
  December 2022
  
  Preprint
  
  Publication Github
- AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning
  
  Yaqing Wang, Sahaj Agarwal, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Awadallah, Jianfeng Gao
  
  EMNLP | December 2022
  
  Publication Project
- AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models
  
  Dongkuan Xu, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Awadallah, Jianfeng Gao
  
  NeurIPS 2022 | December 2022
  
  Publication Publication Project Project
- Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge
  
  Kaixin Ma, Hao Cheng, Xiaodong Liu, Eric Nyberg, Jianfeng Gao
  
  EMNLP 2022 | October 2022
  
  Publication
- Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
  
  Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang
  
  IJCAI 2022 | July 2022
  
  human parity result on CommonsenseQA
  
  Publication
- Knowledge-Rich Self-Supervision for Biomedical Entity Linking
  
  Sheng Zhang, Hao Cheng, Shikhar Vashishth, Cliff Wong, Jinfeng Xiao, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon
  
  EMNLP 2022 | May 2022
  
  Publication
- Open Domain Question Answering with A Unified Knowledge Interface
  
  Kaixin Ma, Hao Cheng, Xiaodong Liu, Eric Nyberg, Jianfeng Gao
  
  ACL 2022 | May 2022
  
  Publication
- LiST: Lite Prompted Self-training Makes Parameter-efficient Few-shot Learners
  
  Yaqing Wang, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Awadallah, Jianfeng Gao
  
  NAACL 2022 | April 2022
  
  Publication Github Project Project
- METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals
  
  Payal Bajaj, Chenyan Xiong, Guolin Ke, Xiaodong Liu, Saurabh Tiwary, Tie-Yan Liu, Paul Bennett, Xia Song, Jianfeng Gao
  
  April 2022
  
  arXiv:2204.06644
  
  Publication
- No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models
  
  Chen Liang, Haoming Jiang, Simiao Zuo, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao
  
  ICLR 2022 | April 2022
  
  Publication Publication
- Taming Sparsely Activated Transformer with Stochastic Experts
  
  Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan Awadalla, Ruofei Zhang, Tuo Zhao, Jianfeng Gao
  
  ICLR 2022 | April 2022
  
  Publication Project
- Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer
  
  Greg Yang, Edward J. Hu, Igor Babuschkin, Szymon Sidor, David Farhi, Jakub Pachocki, Xiaodong Liu, Weizhu Chen, Jianfeng Gao
  
  NeurIPS 2021 | March 2022
  
  PDF Publication Github
- CLUES: Few-Shot Learning Evaluation in Natural Language Understanding
  
  Subhabrata (Subho) Mukherjee, Xiaodong Liu, Guoqing Zheng, Saghar Hosseini, Hao Cheng, Greg Yang, Chris Meek, Ahmed Awadallah, Jianfeng Gao
  
  NeurIPS 2021 | December 2021
  
  Publication Github Project
- Taming Sparsely Activated Transformer with Stochastic Experts.
  
  Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan, Ruofei Zhang, Tuo Zhao, Jianfeng Gao, Hany Hassan Awadalla
  
  October 2021
  
  Publication Publication
- Very Deep Transformers for Neural Machine Translation
  
  Xiaodong Liu, Kevin Duh, Liyuan Liu, Jianfeng Gao
  
  October 2021
  
  Publication Github
- ARCH: Efficient Adversarial Regularized Training with Caching
  
  Simiao Zuo, Chen Liang, Haoming Jiang, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao
  
  2021 Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach
  
  Simiao Zuo, Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Jianfeng Gao, Weizhu Chen, Tuo Zhao
  
  2021 Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- Targeted Adversarial Training for Natural Language Understanding
  
  Lis Pereira, Xiaodong Liu, Hao Cheng, Hoifung Poon, Jianfeng Gao, Ichiro Kobayashi
  
  NAACL | June 2021
  
  Publication Github
- Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization
  
  Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, Wei Chen
  
  ACL-IJCNLP 2021 | May 2021
  
  Publication
- DeBERTa: Decoding-Enhanced BERT with Disentangled Attention
  
  Pengcheng He, Xiaodong Liu, Jianfeng Gao, Wei Chen
  
  2021 International Conference on Learning Representations | May 2021
  
  Under review
  
  PDF Publication Github Project Project
- Token-wise Curriculum Learning for Neural Machine Translation
  
  Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Wei Chen, Jianfeng Gao, Tuo Zhao
  
  2021 Empirical Methods in Natural Language Processing | March 2021
  
  Publication
- Reader-Guided Passage Reranking for Open-Domain Question Answering
  
  Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, Wei Chen
  
  ACL-IJCNLP 2021 | December 2020
  
  Publication
- UnitedQA: A Hybrid Approach for Open Domain Question Answering
  
  Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Wei Chen, Jianfeng Gao
  
  ACL-IJCNLP 2021 | December 2020
  
  Publication
- Posterior Differential Regularization with f-divergence for Improving Model Robustness
  
  Hao Cheng, Xiaodong Liu, Lis Pereira, Yaoliang Yu, Jianfeng Gao
  
  October 2020
  
  Publication
- Generation-Augmented Retrieval for Open-domain Question Answering
  
  Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, Wei Chen
  
  ACL-IJCNLP 2021 | September 2020
  
  Publication
- HittER: Hierarchical Transformers for Knowledge Graph Embeddings
  
  Sanxing Chen, Xiaodong Liu, Jianfeng Gao, Jian Jiao, Ruofei Zhang, Yangfeng Ji
  
  EMNLP 2021 | August 2020
  
  Publication Github
- Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
  
  Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon
  
  August 2020
  
  PDF Video Access
- UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
  
  Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon
  
  37th International Conference on Machine Learning (ICML 2020) | July 2020
  
  Publication Github Project
- RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers
  
  Bailin Wang, Richard Shin, Xiaodong Liu, Alex Polozov, Matthew Richardson
  
  ACL 2020 | June 2020
  
  Publication Project
- Adversarial Training for Large Neural Language Models
  
  Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, Jianfeng Gao
  
  April 2020
  
  Publication Github Project
- On the Variance of the Adaptive Learning Rate and Beyond
  
  Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han
  
  Eighth International Conference on Learning Representations (ICLR) | April 2020
  
  Publication PDF Project
- SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization
  
  Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Tuo Zhao
  
  April 2020
  
  Publication Project
- Unified Language Model Pre-training for Natural Language Understanding and Generation
  
  Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon
  
  33rd Conference on Neural Information Processing Systems (NeurIPS 2019) | December 2019
  
  Publication Github Project
- DoubleTransfer at MEDIQA 2019: Multi-Source Transfer Learning for Natural Language Understanding in the Medical Domain.
  
  Yichong Xu, Xiaodong Liu, Chunyuan Li, Hoifung Poon, Jianfeng Gao
  
  BioNLP 2019 | August 2019
  
  PDF
- Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading
  
  Lianhui Qin, Michel Galley, Chris Brockett, Xiaodong Liu, Xiang Gao, Bill Dolan, Yejin Choi, Jianfeng Gao
  
  Proc. of ACL | June 2019
  
  PDF Project Project
- Multi-Task Deep Neural Networks for Natural Language Understanding
  
  Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao
  
  ACL 2019 | June 2019
  
  Publication Project
- Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding
  
  Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao
  
  April 2019
  
  PDF Publication Github Project
- Cyclical Annealing Schedule: A Simple Approach to Mitigate KL Vanishing
  
  Hao Fu, Chunyuan Li, Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, Lawrence Carin
  
  NAACL 2019 | February 2019
  
  PDF
- Dynamic Fusion Networks for Machine Reading Comprehension
  
  Yichong Xu, JJ (Jingjing) Liu, Jianfeng Gao, Yelong Shen, Xiaodong Liu
  
  February 2018
  
  PDF Github
- Stochastic Answer Networks for Machine Reading Comprehension
  
  Xiaodong Liu, Yelong Shen, Jianfeng Gao, Kevin Duh
  
  MSR-TR-2017-49 | December 2017
  
  Publication Project
- Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval
  
  Xiaodong Liu, Jianfeng Gao, Xiaodong He, Li Deng, Kevin Duh, Ye-Yi Wang
  
  May 2015
  
  Publication

- SAS: Simulated Attention Score
  
  Chuanyang Zheng, Jiankai Sun, Yihang Gao, Yuehao Wang, Peihao Wang, Jing Xiong, Liliang Ren, Hao Cheng, Janardhan (Jana) Kulkarni, Yelong Shen, Atlas Wang, Mac Schwager, Anderson Schneider, Xiaodong Liu, Jianfeng Gao
  
  NeurIPS 2025 | July 2025
  
  DOI Publication
- Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities
  
  Chung-En Sun, Xiaodong Liu, Weiwei Yang, Tsui-Wei Weng, Hao Cheng, Aidan San, Michel Galley, Jianfeng Gao
  
  NAACL 2025 | April 2025
  
  Selected for oral presentation
  
  Publication
- Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass
  
  Tong Chen, Hao Fang, Patrick Xia, Xiaodong Liu, Ben Van Durme, Luke Zettlemoyer, Jianfeng Gao, Hao Cheng
  
  ICLR 2025 | November 2024
  
  DOI Publication Publication
- Bridging Discrete and Backpropagation: Straight-Through and Beyond
  
  Liyuan Liu, Chengyu Dong, Xiaodong Liu, Bin Yu, Jianfeng Gao
  
  NeurIPS 2023 (Oral) | December 2023
  
  Publication Github
- Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs
  
  Qingru Zhang, Chandan Singh, Liyuan Liu, Xiaodong Liu, Bin Yu, Jianfeng Gao, Tuo Zhao
  
  ICLR 2024 | November 2023
  
  DOI Publication
- Fast-ELECTRA for Efficient Pre-training
  
  Chengyu Dong, Liyuan Liu, Hao Cheng, Jingbo Shang, Jianfeng Gao, Xiaodong Liu
  
  ICLR 2024 | October 2023
  
  DOI Publication
- Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers
  
  Linyuan Gong, Chenyan Xiong, Xiaodong Liu, Payal Bajaj, Yiqing Xie, Alvin Cheung, Jianfeng Gao, Xia Song
  
  ACL 2023 | July 2023
  
  Publication
- Task-Aware Specialization for Efficient and Robust Dense Retrieval for Open-Domain Question Answering
  
  Hao Cheng, Hao Fang, Xiaodong Liu, Jianfeng Gao
  
  ACL 2023 | July 2023
  
  Publication
- AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation
  
  Ganesh Jawahar, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Young Jin Kim, Muhammad Abdul-Mageed, Laks Lakshmanan, V.S., Ahmed Awadallah, Sébastien Bubeck, Jianfeng Gao
  
  ACL 2023 | June 2023
  
  Publication
- Fine-tuning large neural language models for biomedical natural language processing
  
  Rob Tinn, Hao Cheng, Yu Gu, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon
  
  Patterns | June 2023, Vol 4(4)
  
  Publication Publication
- Chain-of-Skills: A Configurable Model for Open-domain Question Answering
  
  Kaixin Ma, Hao Cheng, Yu Zhang, Xiaodong Liu, Eric Nyberg, Jianfeng Gao
  
  ACL 2023 | May 2023
  
  Publication
- AutoMoE: Neural Architecture Search for Efficient Sparsely Activated Transformers
  
  Ganesh Jawahar, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Young Jin Kim, Muhammad Abdul-Mageed, Laks V. S. Lakshmanan, Ahmed Awadallah, Sébastien Bubeck, Jianfeng Gao
  
  ACL | May 2023
  
  Publication Project
- Efficient Long Sequence Modeling via State Space Augmented Transformer
  
  Simiao Zuo, Xiaodong Liu, Jian Jiao, Denis Charles, Eren Manavoglu, Tuo Zhao, Jianfeng Gao
  
  December 2022
  
  Preprint
  
  Publication Github
- AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning
  
  Yaqing Wang, Sahaj Agarwal, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Awadallah, Jianfeng Gao
  
  EMNLP | December 2022
  
  Publication Project
- AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models
  
  Dongkuan Xu, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Awadallah, Jianfeng Gao
  
  NeurIPS 2022 | December 2022
  
  Publication Publication Project Project
- Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge
  
  Kaixin Ma, Hao Cheng, Xiaodong Liu, Eric Nyberg, Jianfeng Gao
  
  EMNLP 2022 | October 2022
  
  Publication
- Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
  
  Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang
  
  IJCAI 2022 | July 2022
  
  human parity result on CommonsenseQA
  
  Publication
- Knowledge-Rich Self-Supervision for Biomedical Entity Linking
  
  Sheng Zhang, Hao Cheng, Shikhar Vashishth, Cliff Wong, Jinfeng Xiao, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon
  
  EMNLP 2022 | May 2022
  
  Publication
- METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals
  
  Payal Bajaj, Chenyan Xiong, Guolin Ke, Xiaodong Liu, Saurabh Tiwary, Tie-Yan Liu, Paul Bennett, Xia Song, Jianfeng Gao
  
  April 2022
  
  arXiv:2204.06644
  
  Publication
- No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models
  
  Chen Liang, Haoming Jiang, Simiao Zuo, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao
  
  ICLR 2022 | April 2022
  
  Publication Publication
- Taming Sparsely Activated Transformer with Stochastic Experts
  
  Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan Awadalla, Ruofei Zhang, Tuo Zhao, Jianfeng Gao
  
  ICLR 2022 | April 2022
  
  Publication Project
- LiST: Lite Prompted Self-training Makes Parameter-efficient Few-shot Learners
  
  Yaqing Wang, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Awadallah, Jianfeng Gao
  
  NAACL 2022 | April 2022
  
  Publication Github Project Project
- Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer
  
  Greg Yang, Edward J. Hu, Igor Babuschkin, Szymon Sidor, David Farhi, Jakub Pachocki, Xiaodong Liu, Weizhu Chen, Jianfeng Gao
  
  NeurIPS 2021 | March 2022
  
  PDF Publication Github
- CLUES: Few-Shot Learning Evaluation in Natural Language Understanding
  
  Subhabrata (Subho) Mukherjee, Xiaodong Liu, Guoqing Zheng, Saghar Hosseini, Hao Cheng, Greg Yang, Chris Meek, Ahmed Awadallah, Jianfeng Gao
  
  NeurIPS 2021 | December 2021
  
  Publication Github Project
- Taming Sparsely Activated Transformer with Stochastic Experts.
  
  Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan, Ruofei Zhang, Tuo Zhao, Jianfeng Gao, Hany Hassan Awadalla
  
  October 2021
  
  Publication Publication
- Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach
  
  Simiao Zuo, Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Jianfeng Gao, Weizhu Chen, Tuo Zhao
  
  2021 Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- Targeted Adversarial Training for Natural Language Understanding
  
  Lis Pereira, Xiaodong Liu, Hao Cheng, Hoifung Poon, Jianfeng Gao, Ichiro Kobayashi
  
  NAACL | June 2021
  
  Publication Github
- DeBERTa: Decoding-Enhanced BERT with Disentangled Attention
  
  Pengcheng He, Xiaodong Liu, Jianfeng Gao, Wei Chen
  
  2021 International Conference on Learning Representations | May 2021
  
  Under review
  
  PDF Publication Github Project Project
- Reader-Guided Passage Reranking for Open-Domain Question Answering
  
  Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, Wei Chen
  
  ACL-IJCNLP 2021 | December 2020
  
  Publication
- UnitedQA: A Hybrid Approach for Open Domain Question Answering
  
  Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Wei Chen, Jianfeng Gao
  
  ACL-IJCNLP 2021 | December 2020
  
  Publication
- Posterior Differential Regularization with f-divergence for Improving Model Robustness
  
  Hao Cheng, Xiaodong Liu, Lis Pereira, Yaoliang Yu, Jianfeng Gao
  
  October 2020
  
  Publication
- HittER: Hierarchical Transformers for Knowledge Graph Embeddings
  
  Sanxing Chen, Xiaodong Liu, Jianfeng Gao, Jian Jiao, Ruofei Zhang, Yangfeng Ji
  
  EMNLP 2021 | August 2020
  
  Publication Github
- UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
  
  Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon
  
  37th International Conference on Machine Learning (ICML 2020) | July 2020
  
  Publication Github Project
- RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers
  
  Bailin Wang, Richard Shin, Xiaodong Liu, Alex Polozov, Matthew Richardson
  
  ACL 2020 | June 2020
  
  Publication Project
- SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization
  
  Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Tuo Zhao
  
  April 2020
  
  Publication Project
- Adversarial Training for Large Neural Language Models
  
  Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, Jianfeng Gao
  
  April 2020
  
  Publication Github Project
- On the Variance of the Adaptive Learning Rate and Beyond
  
  Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han
  
  Eighth International Conference on Learning Representations (ICLR) | April 2020
  
  Publication PDF Project
- Unified Language Model Pre-training for Natural Language Understanding and Generation
  
  Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon
  
  33rd Conference on Neural Information Processing Systems (NeurIPS 2019) | December 2019
  
  Publication Github Project
- DoubleTransfer at MEDIQA 2019: Multi-Source Transfer Learning for Natural Language Understanding in the Medical Domain.
  
  Yichong Xu, Xiaodong Liu, Chunyuan Li, Hoifung Poon, Jianfeng Gao
  
  BioNLP 2019 | August 2019
  
  PDF
- Multi-Task Deep Neural Networks for Natural Language Understanding
  
  Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao
  
  ACL 2019 | June 2019
  
  Publication Project
- Cyclical Annealing Schedule: A Simple Approach to Mitigate KL Vanishing
  
  Hao Fu, Chunyuan Li, Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, Lawrence Carin
  
  NAACL 2019 | February 2019
  
  PDF
- Dynamic Fusion Networks for Machine Reading Comprehension
  
  Yichong Xu, JJ (Jingjing) Liu, Jianfeng Gao, Yelong Shen, Xiaodong Liu
  
  February 2018
  
  PDF Github
- Stochastic Answer Networks for Machine Reading Comprehension
  
  Xiaodong Liu, Yelong Shen, Jianfeng Gao, Kevin Duh
  
  MSR-TR-2017-49 | December 2017
  
  Publication Project
- Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval
  
  Xiaodong Liu, Jianfeng Gao, Xiaodong He, Li Deng, Kevin Duh, Ye-Yi Wang
  
  May 2015
  
  Publication
- Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities
  
  Chung-En Sun, Xiaodong Liu, Weiwei Yang, Tsui-Wei Weng, Hao Cheng, Aidan San, Michel Galley, Jianfeng Gao
  
  NAACL 2025 | April 2025
  
  Selected for oral presentation
  
  Publication
- Targeted Adversarial Training for Natural Language Understanding
  
  Lis Pereira, Xiaodong Liu, Hao Cheng, Hoifung Poon, Jianfeng Gao, Ichiro Kobayashi
  
  NAACL | June 2021
  
  Publication Github
- Bridging Discrete and Backpropagation: Straight-Through and Beyond
  
  Liyuan Liu, Chengyu Dong, Xiaodong Liu, Bin Yu, Jianfeng Gao
  
  NeurIPS 2023 (Oral) | December 2023
  
  Publication Github
- On the Variance of the Adaptive Learning Rate and Beyond
  
  Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han
  
  Eighth International Conference on Learning Representations (ICLR) | April 2020
  
  Publication PDF Project
- Fine-tuning large neural language models for biomedical natural language processing
  
  Rob Tinn, Hao Cheng, Yu Gu, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon
  
  Patterns | June 2023, Vol 4(4)
  
  Publication Publication
- Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
  
  Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon
  
  August 2020
  
  PDF Video Access
- DoubleTransfer at MEDIQA 2019: Multi-Source Transfer Learning for Natural Language Understanding in the Medical Domain.
  
  Yichong Xu, Xiaodong Liu, Chunyuan Li, Hoifung Poon, Jianfeng Gao
  
  BioNLP 2019 | August 2019
  
  PDF
- Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers
  
  Linyuan Gong, Chenyan Xiong, Xiaodong Liu, Payal Bajaj, Yiqing Xie, Alvin Cheung, Jianfeng Gao, Xia Song
  
  ACL 2023 | July 2023
  
  Publication
- Task-Aware Specialization for Efficient and Robust Dense Retrieval for Open-Domain Question Answering
  
  Hao Cheng, Hao Fang, Xiaodong Liu, Jianfeng Gao
  
  ACL 2023 | July 2023
  
  Publication
- AutoMoE: Neural Architecture Search for Efficient Sparsely Activated Transformers
  
  Ganesh Jawahar, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Young Jin Kim, Muhammad Abdul-Mageed, Laks V. S. Lakshmanan, Ahmed Awadallah, Sébastien Bubeck, Jianfeng Gao
  
  ACL | May 2023
  
  Publication Project
- Efficient Long Sequence Modeling via State Space Augmented Transformer
  
  Simiao Zuo, Xiaodong Liu, Jian Jiao, Denis Charles, Eren Manavoglu, Tuo Zhao, Jianfeng Gao
  
  December 2022
  
  Preprint
  
  Publication Github
- AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning
  
  Yaqing Wang, Sahaj Agarwal, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Awadallah, Jianfeng Gao
  
  EMNLP | December 2022
  
  Publication Project
- AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models
  
  Dongkuan Xu, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Awadallah, Jianfeng Gao
  
  NeurIPS 2022 | December 2022
  
  Publication Publication Project Project
- Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge
  
  Kaixin Ma, Hao Cheng, Xiaodong Liu, Eric Nyberg, Jianfeng Gao
  
  EMNLP 2022 | October 2022
  
  Publication
- Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
  
  Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang
  
  IJCAI 2022 | July 2022
  
  human parity result on CommonsenseQA
  
  Publication
- Knowledge-Rich Self-Supervision for Biomedical Entity Linking
  
  Sheng Zhang, Hao Cheng, Shikhar Vashishth, Cliff Wong, Jinfeng Xiao, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon
  
  EMNLP 2022 | May 2022
  
  Publication
- Open Domain Question Answering with A Unified Knowledge Interface
  
  Kaixin Ma, Hao Cheng, Xiaodong Liu, Eric Nyberg, Jianfeng Gao
  
  ACL 2022 | May 2022
  
  Publication
- METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals
  
  Payal Bajaj, Chenyan Xiong, Guolin Ke, Xiaodong Liu, Saurabh Tiwary, Tie-Yan Liu, Paul Bennett, Xia Song, Jianfeng Gao
  
  April 2022
  
  arXiv:2204.06644
  
  Publication
- Taming Sparsely Activated Transformer with Stochastic Experts
  
  Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan Awadalla, Ruofei Zhang, Tuo Zhao, Jianfeng Gao
  
  ICLR 2022 | April 2022
  
  Publication Project
- LiST: Lite Prompted Self-training Makes Parameter-efficient Few-shot Learners
  
  Yaqing Wang, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Awadallah, Jianfeng Gao
  
  NAACL 2022 | April 2022
  
  Publication Github Project Project
- CLUES: Few-Shot Learning Evaluation in Natural Language Understanding
  
  Subhabrata (Subho) Mukherjee, Xiaodong Liu, Guoqing Zheng, Saghar Hosseini, Hao Cheng, Greg Yang, Chris Meek, Ahmed Awadallah, Jianfeng Gao
  
  NeurIPS 2021 | December 2021
  
  Publication Github Project
- Taming Sparsely Activated Transformer with Stochastic Experts.
  
  Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan, Ruofei Zhang, Tuo Zhao, Jianfeng Gao, Hany Hassan Awadalla
  
  October 2021
  
  Publication Publication
- Very Deep Transformers for Neural Machine Translation
  
  Xiaodong Liu, Kevin Duh, Liyuan Liu, Jianfeng Gao
  
  October 2021
  
  Publication Github
- ARCH: Efficient Adversarial Regularized Training with Caching
  
  Simiao Zuo, Chen Liang, Haoming Jiang, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao
  
  2021 Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach
  
  Simiao Zuo, Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Jianfeng Gao, Weizhu Chen, Tuo Zhao
  
  2021 Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization
  
  Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, Wei Chen
  
  ACL-IJCNLP 2021 | May 2021
  
  Publication
- Token-wise Curriculum Learning for Neural Machine Translation
  
  Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Wei Chen, Jianfeng Gao, Tuo Zhao
  
  2021 Empirical Methods in Natural Language Processing | March 2021
  
  Publication
- Reader-Guided Passage Reranking for Open-Domain Question Answering
  
  Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, Wei Chen
  
  ACL-IJCNLP 2021 | December 2020
  
  Publication
- Generation-Augmented Retrieval for Open-domain Question Answering
  
  Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, Wei Chen
  
  ACL-IJCNLP 2021 | September 2020
  
  Publication
- HittER: Hierarchical Transformers for Knowledge Graph Embeddings
  
  Sanxing Chen, Xiaodong Liu, Jianfeng Gao, Jian Jiao, Ruofei Zhang, Yangfeng Ji
  
  EMNLP 2021 | August 2020
  
  Publication Github
- RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers
  
  Bailin Wang, Richard Shin, Xiaodong Liu, Alex Polozov, Matthew Richardson
  
  ACL 2020 | June 2020
  
  Publication Project
- SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization
  
  Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Tuo Zhao
  
  April 2020
  
  Publication Project
- Adversarial Training for Large Neural Language Models
  
  Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, Jianfeng Gao
  
  April 2020
  
  Publication Github Project
- On the Variance of the Adaptive Learning Rate and Beyond
  
  Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han
  
  Eighth International Conference on Learning Representations (ICLR) | April 2020
  
  Publication PDF Project
- Unified Language Model Pre-training for Natural Language Understanding and Generation
  
  Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon
  
  33rd Conference on Neural Information Processing Systems (NeurIPS 2019) | December 2019
  
  Publication Github Project
- Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading
  
  Lianhui Qin, Michel Galley, Chris Brockett, Xiaodong Liu, Xiang Gao, Bill Dolan, Yejin Choi, Jianfeng Gao
  
  Proc. of ACL | June 2019
  
  PDF Project Project
- Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding
  
  Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao
  
  April 2019
  
  PDF Publication Github Project
- Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval
  
  Xiaodong Liu, Jianfeng Gao, Xiaodong He, Li Deng, Kevin Duh, Ye-Yi Wang
  
  May 2015
  
  Publication
- Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization
  
  Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, Wei Chen
  
  ACL-IJCNLP 2021 | May 2021
  
  Publication
- Posterior Differential Regularization with f-divergence for Improving Model Robustness
  
  Hao Cheng, Xiaodong Liu, Lis Pereira, Yaoliang Yu, Jianfeng Gao
  
  October 2020
  
  Publication
- On the Variance of the Adaptive Learning Rate and Beyond
  
  Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han
  
  Eighth International Conference on Learning Representations (ICLR) | April 2020
  
  Publication PDF Project

- SAS: Simulated Attention Score
  
  Chuanyang Zheng, Jiankai Sun, Yihang Gao, Yuehao Wang, Peihao Wang, Jing Xiong, Liliang Ren, Hao Cheng, Janardhan (Jana) Kulkarni, Yelong Shen, Atlas Wang, Mac Schwager, Anderson Schneider, Xiaodong Liu, Jianfeng Gao
  
  NeurIPS 2025 | July 2025
  
  DOI Publication
- Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities
  
  Chung-En Sun, Xiaodong Liu, Weiwei Yang, Tsui-Wei Weng, Hao Cheng, Aidan San, Michel Galley, Jianfeng Gao
  
  NAACL 2025 | April 2025
  
  Selected for oral presentation
  
  Publication
- Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass
  
  Tong Chen, Hao Fang, Patrick Xia, Xiaodong Liu, Ben Van Durme, Luke Zettlemoyer, Jianfeng Gao, Hao Cheng
  
  ICLR 2025 | November 2024
  
  DOI Publication Publication
- Bridging Discrete and Backpropagation: Straight-Through and Beyond
  
  Liyuan Liu, Chengyu Dong, Xiaodong Liu, Bin Yu, Jianfeng Gao
  
  NeurIPS 2023 (Oral) | December 2023
  
  Publication Github
- Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs
  
  Qingru Zhang, Chandan Singh, Liyuan Liu, Xiaodong Liu, Bin Yu, Jianfeng Gao, Tuo Zhao
  
  ICLR 2024 | November 2023
  
  DOI Publication
- Fast-ELECTRA for Efficient Pre-training
  
  Chengyu Dong, Liyuan Liu, Hao Cheng, Jingbo Shang, Jianfeng Gao, Xiaodong Liu
  
  ICLR 2024 | October 2023
  
  DOI Publication
- Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers
  
  Linyuan Gong, Chenyan Xiong, Xiaodong Liu, Payal Bajaj, Yiqing Xie, Alvin Cheung, Jianfeng Gao, Xia Song
  
  ACL 2023 | July 2023
  
  Publication
- Task-Aware Specialization for Efficient and Robust Dense Retrieval for Open-Domain Question Answering
  
  Hao Cheng, Hao Fang, Xiaodong Liu, Jianfeng Gao
  
  ACL 2023 | July 2023
  
  Publication
- AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation
  
  Ganesh Jawahar, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Young Jin Kim, Muhammad Abdul-Mageed, Laks Lakshmanan, V.S., Ahmed Awadallah, Sébastien Bubeck, Jianfeng Gao
  
  ACL 2023 | June 2023
  
  Publication
- Chain-of-Skills: A Configurable Model for Open-domain Question Answering
  
  Kaixin Ma, Hao Cheng, Yu Zhang, Xiaodong Liu, Eric Nyberg, Jianfeng Gao
  
  ACL 2023 | May 2023
  
  Publication
- AutoMoE: Neural Architecture Search for Efficient Sparsely Activated Transformers
  
  Ganesh Jawahar, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Young Jin Kim, Muhammad Abdul-Mageed, Laks V. S. Lakshmanan, Ahmed Awadallah, Sébastien Bubeck, Jianfeng Gao
  
  ACL | May 2023
  
  Publication Project
- AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models
  
  Dongkuan Xu, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Awadallah, Jianfeng Gao
  
  NeurIPS 2022 | December 2022
  
  Publication Publication Project Project
- Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge
  
  Kaixin Ma, Hao Cheng, Xiaodong Liu, Eric Nyberg, Jianfeng Gao
  
  EMNLP 2022 | October 2022
  
  Publication
- Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
  
  Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang
  
  IJCAI 2022 | July 2022
  
  human parity result on CommonsenseQA
  
  Publication
- Knowledge-Rich Self-Supervision for Biomedical Entity Linking
  
  Sheng Zhang, Hao Cheng, Shikhar Vashishth, Cliff Wong, Jinfeng Xiao, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon
  
  EMNLP 2022 | May 2022
  
  Publication
- Open Domain Question Answering with A Unified Knowledge Interface
  
  Kaixin Ma, Hao Cheng, Xiaodong Liu, Eric Nyberg, Jianfeng Gao
  
  ACL 2022 | May 2022
  
  Publication
- No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models
  
  Chen Liang, Haoming Jiang, Simiao Zuo, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao
  
  ICLR 2022 | April 2022
  
  Publication Publication
- Taming Sparsely Activated Transformer with Stochastic Experts
  
  Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan Awadalla, Ruofei Zhang, Tuo Zhao, Jianfeng Gao
  
  ICLR 2022 | April 2022
  
  Publication Project
- Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer
  
  Greg Yang, Edward J. Hu, Igor Babuschkin, Szymon Sidor, David Farhi, Jakub Pachocki, Xiaodong Liu, Weizhu Chen, Jianfeng Gao
  
  NeurIPS 2021 | March 2022
  
  PDF Publication Github
- CLUES: Few-Shot Learning Evaluation in Natural Language Understanding
  
  Subhabrata (Subho) Mukherjee, Xiaodong Liu, Guoqing Zheng, Saghar Hosseini, Hao Cheng, Greg Yang, Chris Meek, Ahmed Awadallah, Jianfeng Gao
  
  NeurIPS 2021 | December 2021
  
  Publication Github Project
- Taming Sparsely Activated Transformer with Stochastic Experts.
  
  Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan, Ruofei Zhang, Tuo Zhao, Jianfeng Gao, Hany Hassan Awadalla
  
  October 2021
  
  Publication Publication
- ARCH: Efficient Adversarial Regularized Training with Caching
  
  Simiao Zuo, Chen Liang, Haoming Jiang, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao
  
  2021 Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach
  
  Simiao Zuo, Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Jianfeng Gao, Weizhu Chen, Tuo Zhao
  
  2021 Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- Targeted Adversarial Training for Natural Language Understanding
  
  Lis Pereira, Xiaodong Liu, Hao Cheng, Hoifung Poon, Jianfeng Gao, Ichiro Kobayashi
  
  NAACL | June 2021
  
  Publication Github
- Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization
  
  Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, Wei Chen
  
  ACL-IJCNLP 2021 | May 2021
  
  Publication
- DeBERTa: Decoding-Enhanced BERT with Disentangled Attention
  
  Pengcheng He, Xiaodong Liu, Jianfeng Gao, Wei Chen
  
  2021 International Conference on Learning Representations | May 2021
  
  Under review
  
  PDF Publication Github Project Project
- Token-wise Curriculum Learning for Neural Machine Translation
  
  Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Wei Chen, Jianfeng Gao, Tuo Zhao
  
  2021 Empirical Methods in Natural Language Processing | March 2021
  
  Publication
- Reader-Guided Passage Reranking for Open-Domain Question Answering
  
  Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, Wei Chen
  
  ACL-IJCNLP 2021 | December 2020
  
  Publication
- UnitedQA: A Hybrid Approach for Open Domain Question Answering
  
  Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Wei Chen, Jianfeng Gao
  
  ACL-IJCNLP 2021 | December 2020
  
  Publication
- Generation-Augmented Retrieval for Open-domain Question Answering
  
  Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, Wei Chen
  
  ACL-IJCNLP 2021 | September 2020
  
  Publication
- HittER: Hierarchical Transformers for Knowledge Graph Embeddings
  
  Sanxing Chen, Xiaodong Liu, Jianfeng Gao, Jian Jiao, Ruofei Zhang, Yangfeng Ji
  
  EMNLP 2021 | August 2020
  
  Publication Github
- UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
  
  Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon
  
  37th International Conference on Machine Learning (ICML 2020) | July 2020
  
  Publication Github Project
- RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers
  
  Bailin Wang, Richard Shin, Xiaodong Liu, Alex Polozov, Matthew Richardson
  
  ACL 2020 | June 2020
  
  Publication Project
- On the Variance of the Adaptive Learning Rate and Beyond
  
  Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han
  
  Eighth International Conference on Learning Representations (ICLR) | April 2020
  
  Publication PDF Project
- Unified Language Model Pre-training for Natural Language Understanding and Generation
  
  Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon
  
  33rd Conference on Neural Information Processing Systems (NeurIPS 2019) | December 2019
  
  Publication Github Project
- DoubleTransfer at MEDIQA 2019: Multi-Source Transfer Learning for Natural Language Understanding in the Medical Domain.
  
  Yichong Xu, Xiaodong Liu, Chunyuan Li, Hoifung Poon, Jianfeng Gao
  
  BioNLP 2019 | August 2019
  
  PDF
- Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading
  
  Lianhui Qin, Michel Galley, Chris Brockett, Xiaodong Liu, Xiang Gao, Bill Dolan, Yejin Choi, Jianfeng Gao
  
  Proc. of ACL | June 2019
  
  PDF Project Project
- Multi-Task Deep Neural Networks for Natural Language Understanding
  
  Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao
  
  ACL 2019 | June 2019
  
  Publication Project
- Cyclical Annealing Schedule: A Simple Approach to Mitigate KL Vanishing
  
  Hao Fu, Chunyuan Li, Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, Lawrence Carin
  
  NAACL 2019 | February 2019
  
  PDF
- Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval
  
  Xiaodong Liu, Jianfeng Gao, Xiaodong He, Li Deng, Kevin Duh, Ye-Yi Wang
  
  May 2015
  
  Publication
- Fine-tuning large neural language models for biomedical natural language processing
  
  Rob Tinn, Hao Cheng, Yu Gu, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon
  
  Patterns | June 2023, Vol 4(4)
  
  Publication Publication
- AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning
  
  Yaqing Wang, Sahaj Agarwal, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Awadallah, Jianfeng Gao
  
  EMNLP | December 2022
  
  Publication Project
- LiST: Lite Prompted Self-training Makes Parameter-efficient Few-shot Learners
  
  Yaqing Wang, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Awadallah, Jianfeng Gao
  
  NAACL 2022 | April 2022
  
  Publication Github Project Project
- Efficient Long Sequence Modeling via State Space Augmented Transformer
  
  Simiao Zuo, Xiaodong Liu, Jian Jiao, Denis Charles, Eren Manavoglu, Tuo Zhao, Jianfeng Gao
  
  December 2022
  
  Preprint
  
  Publication Github
- METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals
  
  Payal Bajaj, Chenyan Xiong, Guolin Ke, Xiaodong Liu, Saurabh Tiwary, Tie-Yan Liu, Paul Bennett, Xia Song, Jianfeng Gao
  
  April 2022
  
  arXiv:2204.06644
  
  Publication
- Very Deep Transformers for Neural Machine Translation
  
  Xiaodong Liu, Kevin Duh, Liyuan Liu, Jianfeng Gao
  
  October 2021
  
  Publication Github
- Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding
  
  Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao
  
  April 2019
  
  PDF Publication Github Project
- Posterior Differential Regularization with f-divergence for Improving Model Robustness
  
  Hao Cheng, Xiaodong Liu, Lis Pereira, Yaoliang Yu, Jianfeng Gao
  
  October 2020
  
  Publication
- Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
  
  Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon
  
  August 2020
  
  PDF Video Access
- SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization
  
  Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Tuo Zhao
  
  April 2020
  
  Publication Project
- Adversarial Training for Large Neural Language Models
  
  Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, Jianfeng Gao
  
  April 2020
  
  Publication Github Project
- Dynamic Fusion Networks for Machine Reading Comprehension
  
  Yichong Xu, JJ (Jingjing) Liu, Jianfeng Gao, Yelong Shen, Xiaodong Liu
  
  February 2018
  
  PDF Github
- Stochastic Answer Networks for Machine Reading Comprehension
  
  Xiaodong Liu, Yelong Shen, Jianfeng Gao, Kevin Duh
  
  MSR-TR-2017-49 | December 2017
  
  Publication Project

Xiaodong Liu

Publications

SAS: Simulated Attention Score

Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities

Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass

Bridging Discrete and Backpropagation: Straight-Through and Beyond

Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs

Fast-ELECTRA for Efficient Pre-training

Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers

Task-Aware Specialization for Efficient and Robust Dense Retrieval for Open-Domain Question Answering

AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation

Fine-tuning large neural language models for biomedical natural language processing

Chain-of-Skills: A Configurable Model for Open-domain Question Answering

AutoMoE: Neural Architecture Search for Efficient Sparsely Activated Transformers

Efficient Long Sequence Modeling via State Space Augmented Transformer

AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning

AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models

Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge

Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention

Knowledge-Rich Self-Supervision for Biomedical Entity Linking

Open Domain Question Answering with A Unified Knowledge Interface

LiST: Lite Prompted Self-training Makes Parameter-efficient Few-shot Learners

METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals

No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models

Taming Sparsely Activated Transformer with Stochastic Experts

Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer

CLUES: Few-Shot Learning Evaluation in Natural Language Understanding

Taming Sparsely Activated Transformer with Stochastic Experts.

Very Deep Transformers for Neural Machine Translation

ARCH: Efficient Adversarial Regularized Training with Caching

Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach

Targeted Adversarial Training for Natural Language Understanding

Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization

DeBERTa: Decoding-Enhanced BERT with Disentangled Attention

Token-wise Curriculum Learning for Neural Machine Translation

Reader-Guided Passage Reranking for Open-Domain Question Answering

UnitedQA: A Hybrid Approach for Open Domain Question Answering

Posterior Differential Regularization with f-divergence for Improving Model Robustness

Generation-Augmented Retrieval for Open-domain Question Answering

HittER: Hierarchical Transformers for Knowledge Graph Embeddings

Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training

RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers

Adversarial Training for Large Neural Language Models

On the Variance of the Adaptive Learning Rate and Beyond

SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization

Unified Language Model Pre-training for Natural Language Understanding and Generation

DoubleTransfer at MEDIQA 2019: Multi-Source Transfer Learning for Natural Language Understanding in the Medical Domain.

Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading

Multi-Task Deep Neural Networks for Natural Language Understanding

Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding

Cyclical Annealing Schedule: A Simple Approach to Mitigate KL Vanishing

Dynamic Fusion Networks for Machine Reading Comprehension

Stochastic Answer Networks for Machine Reading Comprehension

Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval

SAS: Simulated Attention Score

Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities

Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass

Bridging Discrete and Backpropagation: Straight-Through and Beyond

Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs

Fast-ELECTRA for Efficient Pre-training

Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers

Task-Aware Specialization for Efficient and Robust Dense Retrieval for Open-Domain Question Answering

AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation

Fine-tuning large neural language models for biomedical natural language processing

Chain-of-Skills: A Configurable Model for Open-domain Question Answering

AutoMoE: Neural Architecture Search for Efficient Sparsely Activated Transformers

Efficient Long Sequence Modeling via State Space Augmented Transformer

AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning

AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models

Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge

Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention

Knowledge-Rich Self-Supervision for Biomedical Entity Linking

METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals

No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models

Taming Sparsely Activated Transformer with Stochastic Experts

LiST: Lite Prompted Self-training Makes Parameter-efficient Few-shot Learners

Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer

CLUES: Few-Shot Learning Evaluation in Natural Language Understanding

Taming Sparsely Activated Transformer with Stochastic Experts.