Effect of Document Packing on the Latent Multi-Hop Reasoning Capabilities of Large Language Models
Gabriele Prato, Shagun Sodhani, Alessandro Sordoni, Sarath Chandar, Alessandro Sordoni
December 2025
Alessandro Sordoni
Principal Researcher
出版物
-
-
-
Learning to Solve Complex Problems via Dataset Decomposition
Wanru Zhao, Lucas Caccia, Zhengyan Shi, Minseon Kim, Xingdi Yuan, Weijia Xu, Marc-Alexandre Côté, Alessandro Sordoni
NeurIPS 2025 | July 2025
-
Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts
Samin Yeasar Arnob, Zhan Su, Minseon Kim, Oleksiy Ostapenko, Riyasat Ohib, Esra'a Saleh, Doina Precup, Lucas Caccia, Alessandro Sordoni
COLM 2025 | July 2025
-
Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers
Kusha Sareen, Morgane M Moss, Alessandro Sordoni, Rishabh Agarwal, Arian Hosseini
COLM 2025 | July 2025
-
Training Plug-and-Play Knowledge Modules with Deep Context Distillation
Lucas Caccia, Alan Ansell, E. Ponti, Ivan Vuli'c, Alessandro Sordoni
COLM 2025 | July 2025
-
debug-gym: A Text-Based Environment for Interactive Debugging
Xingdi Yuan, Morgane M Moss, Charbel Feghali, Chinmay Singh, Darya Moldavskaya, Drew MacPhee, Lucas Caccia, Matheus Pereira, Minseon Kim, Alessandro Sordoni, Marc-Alexandre Côté
March 2025
-
-
-
Energy-efficient Reinforcement Learning by Discovering Neural Pathways
October 2024
-
A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning
Prateek Yadav, Colin Raffel, Mohammed Muqeeth, Lucas Caccia, Haokun Liu, Tian-Xiang Chen, Mohit Bansal, Leshem Choshen, Alessandro Sordoni
ArXiv | August 2024, 巻abs/2408.07057
-
Improving Context-Aware Preference Modeling for Language Models
Silviu Pitis, Ziang Xiao, Nicolas Le Roux, Alessandro Sordoni
NeurIPS 2024 | July 2024
-
Efficient Adversarial Training in LLMs with Continuous Attacks
Sophie Xhonneux, Alessandro Sordoni, Stephan Günnemann, Gauthier Gidel, Leo Schwinn
NeurIPS 2024 | May 2024
-
Towards Modular LLMs by Building and Reusing a Library of LoRAs
O. Ostapenko, Zhan Su, E. Ponti, Laurent Charlin, Nicolas Le Roux, Matheus Pereira, Lucas Caccia, Alessandro Sordoni
ICML 2024 | May 2024
-
-
-
Multi-Head Adapter Routing for Data-Efficient Fine-Tuning
Lucas Caccia, E. Ponti, Lu Liu, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni
ArXiv | November 2023, 巻abs/2211.03831
-
Guiding Language Model Reasoning with Planning Tokens
Xinyi Wang, Lucas Caccia, O. Ostapenko, Xingdi Yuan, Alessandro Sordoni
October 2023
-
Joint Prompt Optimization of Stacked LLMs using Variational Inference
Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner, Nicolas Le Roux
NeurIPS 2023 | October 2023
-
Multi-Head Adapter Routing for Cross-Task Generalization
Lucas Caccia, Edoardo Ponti, Zhan Su, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni
NeurIPS 2023 | October 2023
-
-
-
Better Language Model with Hypernym Class Prediction
He Bai, Tong Wang, Alessandro Sordoni, Peng Shi
ACL 2022 | May 2022
-
Unsupervised Dependency Graph Network
Yikang Shen, Shawn Tan, Alessandro Sordoni, Peng Li, Jie Zhou, Aaron Courville
ACL 2022 | May 2022
-
Evaluating Distributional Distortion in Neural Language Modeling
Benjamin LeBrun, Alessandro Sordoni, Timothy J. O'Donnell
ICLR 2022 | April 2022
-
Learning to Dequantise with Truncated Flows
Shawn Tan, Chin-Wei Huang, Alessandro Sordoni, Aaron Courville
ICLR 2022 | April 2022
-
-
-
Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU
Meghana Moorthy Bhat, Alessandro Sordoni, Subhabrata (Subho) Mukherjee
EMNLP 2021 | November 2021
-
The emergence of the shape bias results from communicative efficiency
Eva Portelance, Michael C. Frank, Dan Jurafsky, Alessandro Sordoni, Romain Laroche
Conference on Computational Natural Language Learning (CoNLL) | November 2021
-
Decomposed Mutual Information Estimation for Contrastive Representation Learning
Alessandro Sordoni, Nouha Dziri, Hannes Schulz, Geoff Gordon, Philip Bachman, Remi Tachet des Combes
2021 International Conference on Machine Learning | July 2021
-
Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle
Yikang Shen, Shawn Tan, Alessandro Sordoni, Siva Reddy, Aaron Courville
May 2021
-
Understanding by Understanding Not: Modeling Negation in Language Models
Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, Devon Hjelm, Alessandro Sordoni, Aaron Courville
May 2021
-
Linguistic dependencies and statistical dependence
Jacob Louis Hoover, Alessandro Sordoni, Wenyu Du, Timothy J. O'Donnell
2021 Empirical Methods in Natural Language Processing | April 2021
-
-
-
Exploring and Predicting Transferability Across NLP Tasks
Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
May 2020
-
Fast Domain Adaptation For Goal-Oriented Dialogue Using A Hybrid Generative-Retrieval Transformer
Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | April 2020
-
Hybrid Generative-Retrieval Transformers For Dialogue Domain Adaptation
Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
arXiv preprint arXiv:2003.01680 | March 2020
-
-
-
Report on the First HIPstIR Workshop on the Future of Information Retrieval
Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
SIGIR Forum | December 2019, 巻53(2): pp. 62-75
-
Ordered Memory
Yikang Shen, Shawn Tan, Arian Hosseini, Zhouhan Lin, Alessandro Sordoni, Aaron Courville
NeurIPS 2019 | November 2019
ArXiv
-
Robust Natural Language Inference Models with Example Forgetting
Yadollah Yaghoobzadeh, Remi Tachet des Combes, T. J. Hazen, Alessandro Sordoni
November 2019
-
Metalearned Neural Memory
Tsendsuren Munkhdalai, Alessandro Sordoni, Tong Wang, Adam Trischler
NeurIPS 2019 | July 2019
Preprint under review
-
Brief Report: Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
Yikeng Shen, Shawn Tan, Alessandro Sordoni, Aaron Courville
ICLR 2019 | May 2019
ICLR 2019 Best Paper Award
-
An Empirical Study of Example Forgetting during Deep Neural Network Learning
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, Geoff Gordon
ICLR 2019 | April 2019
-
-
-
Towards Text Generation with Adversarially Learned Neural Outlines
Sandeep Subramanian, Sai Rajeswar, Alessandro Sordoni, Adam Trischler, Aaron Courville, Christopher Pal
NeurIPS 2018 | December 2018
-
Counting to Explore and Generalize in Text-based Games
Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
European Workshop on Reinforcement Learning (EWRL) | October 2018
-
Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
Yikang Shen, Zhouhan Lin, Athul Paul Jacob, Alessandro Sordoni, Aaron Courville, Yoshua Bengio
Proceedings of ACL | July 2018
-
Counting to Explore and Generalize in Text-based Games
Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
ICML 2018 | June 2018
-
Augmented cyclegan: Learning many-to-many mappings from unpaired data
Amjad Almahairi, Sai Rajeswar, Alessandro Sordoni, Philip Bachman, Aaron Courville
ICML 2018 | June 2018
ArXiv
-
Focused Hierarchical RNNs for Conditional Sequence Processing
Nan Rosemary Ke, Konrad Zolna, Alessandro Sordoni, Zhouhan Lin, Adam Trischler, Yoshua Bengio, Joelle Pineau, Laurent Charlin, Chris Pal
ICML 2018 | June 2018
-
Twin Networks: Matching the Future for Sequence Generation
Dmitriy Serdyuk, Nan Rosemary Ke, Alessandro Sordoni, Chris Pal, Adam Trischler, Yoshua Bengio
ICLR 2018 | January 2018
-
-
-
Z-Forcing: Training Stochastic Recurrent Networks
Anirudh Goyal, Alessandro Sordoni, Marc-Alexandre Côté, Nan Rosemary Ke, Yoshua Bengio
31st Conference on Neural Information Processing Systems (NIPS 2017 - Advances in Neural Information Processing Systems) | November 2017
-
Learning Algorithms for Active Learning
Philip Bachman, Alessandro Sordoni, Adam Trischler
34th International Conference on Machine Learning (ICML 2017) | August 2017
-
-
-
Effect of Document Packing on the Latent Multi-Hop Reasoning Capabilities of Large Language Models
Gabriele Prato, Shagun Sodhani, Alessandro Sordoni, Sarath Chandar, Alessandro Sordoni
December 2025
-
Learning to Solve Complex Problems via Dataset Decomposition
Wanru Zhao, Lucas Caccia, Zhengyan Shi, Minseon Kim, Xingdi Yuan, Weijia Xu, Marc-Alexandre Côté, Alessandro Sordoni
NeurIPS 2025 | July 2025
-
Training Plug-and-Play Knowledge Modules with Deep Context Distillation
Lucas Caccia, Alan Ansell, E. Ponti, Ivan Vuli'c, Alessandro Sordoni
COLM 2025 | July 2025
-
Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers
Kusha Sareen, Morgane M Moss, Alessandro Sordoni, Rishabh Agarwal, Arian Hosseini
COLM 2025 | July 2025
-
Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts
Samin Yeasar Arnob, Zhan Su, Minseon Kim, Oleksiy Ostapenko, Riyasat Ohib, Esra'a Saleh, Doina Precup, Lucas Caccia, Alessandro Sordoni
COLM 2025 | July 2025
-
debug-gym: A Text-Based Environment for Interactive Debugging
Xingdi Yuan, Morgane M Moss, Charbel Feghali, Chinmay Singh, Darya Moldavskaya, Drew MacPhee, Lucas Caccia, Matheus Pereira, Minseon Kim, Alessandro Sordoni, Marc-Alexandre Côté
March 2025
-
Energy-efficient Reinforcement Learning by Discovering Neural Pathways
October 2024
-
A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning
Prateek Yadav, Colin Raffel, Mohammed Muqeeth, Lucas Caccia, Haokun Liu, Tian-Xiang Chen, Mohit Bansal, Leshem Choshen, Alessandro Sordoni
ArXiv | August 2024, 巻abs/2408.07057
-
Improving Context-Aware Preference Modeling for Language Models
Silviu Pitis, Ziang Xiao, Nicolas Le Roux, Alessandro Sordoni
NeurIPS 2024 | July 2024
-
Efficient Adversarial Training in LLMs with Continuous Attacks
Sophie Xhonneux, Alessandro Sordoni, Stephan Günnemann, Gauthier Gidel, Leo Schwinn
NeurIPS 2024 | May 2024
-
Towards Modular LLMs by Building and Reusing a Library of LoRAs
O. Ostapenko, Zhan Su, E. Ponti, Laurent Charlin, Nicolas Le Roux, Matheus Pereira, Lucas Caccia, Alessandro Sordoni
ICML 2024 | May 2024
-
Multi-Head Adapter Routing for Data-Efficient Fine-Tuning
Lucas Caccia, E. Ponti, Lu Liu, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni
ArXiv | November 2023, 巻abs/2211.03831
-
Guiding Language Model Reasoning with Planning Tokens
Xinyi Wang, Lucas Caccia, O. Ostapenko, Xingdi Yuan, Alessandro Sordoni
October 2023
-
Multi-Head Adapter Routing for Cross-Task Generalization
Lucas Caccia, Edoardo Ponti, Zhan Su, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni
NeurIPS 2023 | October 2023
-
Joint Prompt Optimization of Stacked LLMs using Variational Inference
Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner, Nicolas Le Roux
NeurIPS 2023 | October 2023
-
Learning to Dequantise with Truncated Flows
Shawn Tan, Chin-Wei Huang, Alessandro Sordoni, Aaron Courville
ICLR 2022 | April 2022
-
Evaluating Distributional Distortion in Neural Language Modeling
Benjamin LeBrun, Alessandro Sordoni, Timothy J. O'Donnell
ICLR 2022 | April 2022
-
The emergence of the shape bias results from communicative efficiency
Eva Portelance, Michael C. Frank, Dan Jurafsky, Alessandro Sordoni, Romain Laroche
Conference on Computational Natural Language Learning (CoNLL) | November 2021
-
Decomposed Mutual Information Estimation for Contrastive Representation Learning
Alessandro Sordoni, Nouha Dziri, Hannes Schulz, Geoff Gordon, Philip Bachman, Remi Tachet des Combes
2021 International Conference on Machine Learning | July 2021
-
Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle
Yikang Shen, Shawn Tan, Alessandro Sordoni, Siva Reddy, Aaron Courville
May 2021
-
Understanding by Understanding Not: Modeling Negation in Language Models
Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, Devon Hjelm, Alessandro Sordoni, Aaron Courville
May 2021
-
Hybrid Generative-Retrieval Transformers For Dialogue Domain Adaptation
Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
arXiv preprint arXiv:2003.01680 | March 2020
-
Report on the First HIPstIR Workshop on the Future of Information Retrieval
Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
SIGIR Forum | December 2019, 巻53(2): pp. 62-75
-
Robust Natural Language Inference Models with Example Forgetting
Yadollah Yaghoobzadeh, Remi Tachet des Combes, T. J. Hazen, Alessandro Sordoni
November 2019
-
Ordered Memory
Yikang Shen, Shawn Tan, Arian Hosseini, Zhouhan Lin, Alessandro Sordoni, Aaron Courville
NeurIPS 2019 | November 2019
ArXiv
-
Metalearned Neural Memory
Tsendsuren Munkhdalai, Alessandro Sordoni, Tong Wang, Adam Trischler
NeurIPS 2019 | July 2019
Preprint under review
-
Brief Report: Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
Yikeng Shen, Shawn Tan, Alessandro Sordoni, Aaron Courville
ICLR 2019 | May 2019
ICLR 2019 Best Paper Award
-
An Empirical Study of Example Forgetting during Deep Neural Network Learning
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, Geoff Gordon
ICLR 2019 | April 2019
-
Towards Text Generation with Adversarially Learned Neural Outlines
Sandeep Subramanian, Sai Rajeswar, Alessandro Sordoni, Adam Trischler, Aaron Courville, Christopher Pal
NeurIPS 2018 | December 2018
-
Counting to Explore and Generalize in Text-based Games
Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
European Workshop on Reinforcement Learning (EWRL) | October 2018
-
Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
Yikang Shen, Zhouhan Lin, Athul Paul Jacob, Alessandro Sordoni, Aaron Courville, Yoshua Bengio
Proceedings of ACL | July 2018
-
Counting to Explore and Generalize in Text-based Games
Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
ICML 2018 | June 2018
-
Focused Hierarchical RNNs for Conditional Sequence Processing
Nan Rosemary Ke, Konrad Zolna, Alessandro Sordoni, Zhouhan Lin, Adam Trischler, Yoshua Bengio, Joelle Pineau, Laurent Charlin, Chris Pal
ICML 2018 | June 2018
-
Augmented cyclegan: Learning many-to-many mappings from unpaired data
Amjad Almahairi, Sai Rajeswar, Alessandro Sordoni, Philip Bachman, Aaron Courville
ICML 2018 | June 2018
ArXiv
-
Twin Networks: Matching the Future for Sequence Generation
Dmitriy Serdyuk, Nan Rosemary Ke, Alessandro Sordoni, Chris Pal, Adam Trischler, Yoshua Bengio
ICLR 2018 | January 2018
-
Z-Forcing: Training Stochastic Recurrent Networks
Anirudh Goyal, Alessandro Sordoni, Marc-Alexandre Côté, Nan Rosemary Ke, Yoshua Bengio
31st Conference on Neural Information Processing Systems (NIPS 2017 - Advances in Neural Information Processing Systems) | November 2017
-
Learning Algorithms for Active Learning
Philip Bachman, Alessandro Sordoni, Adam Trischler
34th International Conference on Machine Learning (ICML 2017) | August 2017
-
-
-
debug-gym: A Text-Based Environment for Interactive Debugging
Xingdi Yuan, Morgane M Moss, Charbel Feghali, Chinmay Singh, Darya Moldavskaya, Drew MacPhee, Lucas Caccia, Matheus Pereira, Minseon Kim, Alessandro Sordoni, Marc-Alexandre Côté
March 2025
-
-
-
Unsupervised Dependency Graph Network
Yikang Shen, Shawn Tan, Alessandro Sordoni, Peng Li, Jie Zhou, Aaron Courville
ACL 2022 | May 2022
-
Better Language Model with Hypernym Class Prediction
He Bai, Tong Wang, Alessandro Sordoni, Peng Shi
ACL 2022 | May 2022
-
Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU
Meghana Moorthy Bhat, Alessandro Sordoni, Subhabrata (Subho) Mukherjee
EMNLP 2021 | November 2021
-
Understanding by Understanding Not: Modeling Negation in Language Models
Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, Devon Hjelm, Alessandro Sordoni, Aaron Courville
May 2021
-
Linguistic dependencies and statistical dependence
Jacob Louis Hoover, Alessandro Sordoni, Wenyu Du, Timothy J. O'Donnell
2021 Empirical Methods in Natural Language Processing | April 2021
-
Exploring and Predicting Transferability Across NLP Tasks
Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
May 2020
-
-
-
Fast Domain Adaptation For Goal-Oriented Dialogue Using A Hybrid Generative-Retrieval Transformer
Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | April 2020
-
-
-
Report on the First HIPstIR Workshop on the Future of Information Retrieval
Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
SIGIR Forum | December 2019, 巻53(2): pp. 62-75
-
-
-
Effect of Document Packing on the Latent Multi-Hop Reasoning Capabilities of Large Language Models
Gabriele Prato, Shagun Sodhani, Alessandro Sordoni, Sarath Chandar, Alessandro Sordoni
December 2025
-
debug-gym: A Text-Based Environment for Interactive Debugging
Xingdi Yuan, Morgane M Moss, Charbel Feghali, Chinmay Singh, Darya Moldavskaya, Drew MacPhee, Lucas Caccia, Matheus Pereira, Minseon Kim, Alessandro Sordoni, Marc-Alexandre Côté
March 2025
-
Guiding Language Model Reasoning with Planning Tokens
Xinyi Wang, Lucas Caccia, O. Ostapenko, Xingdi Yuan, Alessandro Sordoni
October 2023
-
Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle
Yikang Shen, Shawn Tan, Alessandro Sordoni, Siva Reddy, Aaron Courville
May 2021
-
Understanding by Understanding Not: Modeling Negation in Language Models
Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, Devon Hjelm, Alessandro Sordoni, Aaron Courville
May 2021
-
Exploring and Predicting Transferability Across NLP Tasks
Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
May 2020
-
Robust Natural Language Inference Models with Example Forgetting
Yadollah Yaghoobzadeh, Remi Tachet des Combes, T. J. Hazen, Alessandro Sordoni
November 2019
-
-
-
Learning to Solve Complex Problems via Dataset Decomposition
Wanru Zhao, Lucas Caccia, Zhengyan Shi, Minseon Kim, Xingdi Yuan, Weijia Xu, Marc-Alexandre Côté, Alessandro Sordoni
NeurIPS 2025 | July 2025
-
Training Plug-and-Play Knowledge Modules with Deep Context Distillation
Lucas Caccia, Alan Ansell, E. Ponti, Ivan Vuli'c, Alessandro Sordoni
COLM 2025 | July 2025
-
Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers
Kusha Sareen, Morgane M Moss, Alessandro Sordoni, Rishabh Agarwal, Arian Hosseini
COLM 2025 | July 2025
-
Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts
Samin Yeasar Arnob, Zhan Su, Minseon Kim, Oleksiy Ostapenko, Riyasat Ohib, Esra'a Saleh, Doina Precup, Lucas Caccia, Alessandro Sordoni
COLM 2025 | July 2025
-
Improving Context-Aware Preference Modeling for Language Models
Silviu Pitis, Ziang Xiao, Nicolas Le Roux, Alessandro Sordoni
NeurIPS 2024 | July 2024
-
Efficient Adversarial Training in LLMs with Continuous Attacks
Sophie Xhonneux, Alessandro Sordoni, Stephan Günnemann, Gauthier Gidel, Leo Schwinn
NeurIPS 2024 | May 2024
-
Towards Modular LLMs by Building and Reusing a Library of LoRAs
O. Ostapenko, Zhan Su, E. Ponti, Laurent Charlin, Nicolas Le Roux, Matheus Pereira, Lucas Caccia, Alessandro Sordoni
ICML 2024 | May 2024
-
Multi-Head Adapter Routing for Cross-Task Generalization
Lucas Caccia, Edoardo Ponti, Zhan Su, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni
NeurIPS 2023 | October 2023
-
Joint Prompt Optimization of Stacked LLMs using Variational Inference
Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner, Nicolas Le Roux
NeurIPS 2023 | October 2023
-
Unsupervised Dependency Graph Network
Yikang Shen, Shawn Tan, Alessandro Sordoni, Peng Li, Jie Zhou, Aaron Courville
ACL 2022 | May 2022
-
Better Language Model with Hypernym Class Prediction
He Bai, Tong Wang, Alessandro Sordoni, Peng Shi
ACL 2022 | May 2022
-
Learning to Dequantise with Truncated Flows
Shawn Tan, Chin-Wei Huang, Alessandro Sordoni, Aaron Courville
ICLR 2022 | April 2022
-
Evaluating Distributional Distortion in Neural Language Modeling
Benjamin LeBrun, Alessandro Sordoni, Timothy J. O'Donnell
ICLR 2022 | April 2022
-
The emergence of the shape bias results from communicative efficiency
Eva Portelance, Michael C. Frank, Dan Jurafsky, Alessandro Sordoni, Romain Laroche
Conference on Computational Natural Language Learning (CoNLL) | November 2021
-
Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU
Meghana Moorthy Bhat, Alessandro Sordoni, Subhabrata (Subho) Mukherjee
EMNLP 2021 | November 2021
-
Decomposed Mutual Information Estimation for Contrastive Representation Learning
Alessandro Sordoni, Nouha Dziri, Hannes Schulz, Geoff Gordon, Philip Bachman, Remi Tachet des Combes
2021 International Conference on Machine Learning | July 2021
-
Linguistic dependencies and statistical dependence
Jacob Louis Hoover, Alessandro Sordoni, Wenyu Du, Timothy J. O'Donnell
2021 Empirical Methods in Natural Language Processing | April 2021
-
Fast Domain Adaptation For Goal-Oriented Dialogue Using A Hybrid Generative-Retrieval Transformer
Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | April 2020
-
Ordered Memory
Yikang Shen, Shawn Tan, Arian Hosseini, Zhouhan Lin, Alessandro Sordoni, Aaron Courville
NeurIPS 2019 | November 2019
ArXiv
-
Metalearned Neural Memory
Tsendsuren Munkhdalai, Alessandro Sordoni, Tong Wang, Adam Trischler
NeurIPS 2019 | July 2019
Preprint under review
-
Brief Report: Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
Yikeng Shen, Shawn Tan, Alessandro Sordoni, Aaron Courville
ICLR 2019 | May 2019
ICLR 2019 Best Paper Award
-
An Empirical Study of Example Forgetting during Deep Neural Network Learning
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, Geoff Gordon
ICLR 2019 | April 2019
-
Towards Text Generation with Adversarially Learned Neural Outlines
Sandeep Subramanian, Sai Rajeswar, Alessandro Sordoni, Adam Trischler, Aaron Courville, Christopher Pal
NeurIPS 2018 | December 2018
-
Counting to Explore and Generalize in Text-based Games
Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
European Workshop on Reinforcement Learning (EWRL) | October 2018
-
Counting to Explore and Generalize in Text-based Games
Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
ICML 2018 | June 2018
-
Focused Hierarchical RNNs for Conditional Sequence Processing
Nan Rosemary Ke, Konrad Zolna, Alessandro Sordoni, Zhouhan Lin, Adam Trischler, Yoshua Bengio, Joelle Pineau, Laurent Charlin, Chris Pal
ICML 2018 | June 2018
-
Augmented cyclegan: Learning many-to-many mappings from unpaired data
Amjad Almahairi, Sai Rajeswar, Alessandro Sordoni, Philip Bachman, Aaron Courville
ICML 2018 | June 2018
ArXiv
-
Twin Networks: Matching the Future for Sequence Generation
Dmitriy Serdyuk, Nan Rosemary Ke, Alessandro Sordoni, Chris Pal, Adam Trischler, Yoshua Bengio
ICLR 2018 | January 2018
-
Z-Forcing: Training Stochastic Recurrent Networks
Anirudh Goyal, Alessandro Sordoni, Marc-Alexandre Côté, Nan Rosemary Ke, Yoshua Bengio
31st Conference on Neural Information Processing Systems (NIPS 2017 - Advances in Neural Information Processing Systems) | November 2017
-
Learning Algorithms for Active Learning
Philip Bachman, Alessandro Sordoni, Adam Trischler
34th International Conference on Machine Learning (ICML 2017) | August 2017
-
-
-
A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning
Prateek Yadav, Colin Raffel, Mohammed Muqeeth, Lucas Caccia, Haokun Liu, Tian-Xiang Chen, Mohit Bansal, Leshem Choshen, Alessandro Sordoni
ArXiv | August 2024, 巻abs/2408.07057
-
Multi-Head Adapter Routing for Data-Efficient Fine-Tuning
Lucas Caccia, E. Ponti, Lu Liu, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni
ArXiv | November 2023, 巻abs/2211.03831
-
Hybrid Generative-Retrieval Transformers For Dialogue Domain Adaptation
Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
arXiv preprint arXiv:2003.01680 | March 2020
-
Report on the First HIPstIR Workshop on the Future of Information Retrieval
Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
SIGIR Forum | December 2019, 巻53(2): pp. 62-75
-
Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
Yikang Shen, Zhouhan Lin, Athul Paul Jacob, Alessandro Sordoni, Aaron Courville, Yoshua Bengio
Proceedings of ACL | July 2018
-
-