Web and AI Sciences

Publications

View by:

- All claims are equal, but some claims are more equal than others: Importance-sensitive factuality evaluation of LLM generations
  
  Miriam Wanner, Leif Azzopardi, Paul Thomas, Soham Dan, Ben Van Durme, Nick Craswell
  
  October 2025
  
  preprint arXiv:2510.07083
  
  Publication
- Demographically-inspired query variants using an LLM
  
  Marwah Alaofi, Nicola Ferro, Paul Thomas, Falk Scholer, Mark Sanderson
  
  International Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR) | July 2025
  
  DOI Publication
- Preaching to the ChoIR: Lessons IR should share with AI
  
  Gianluca Demartini, Claudia Hauff, Matthew Lease, Stefano Mizzaro, Kevin Roitero, Mark Sanderson, Falk Scholer, Chirag Shah, Damiano Spina, Paul Thomas, Arjen P de Vries, Guido Zuccon
  
  International Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR) | July 2025
  
  DOI
- System comparison using automated generation of relevance judgements in multiple languages
  
  Paul Thomas, Douglas W Oard, Eugene Yang, Dawn Lawrie, James Mayfield
  
  2025 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2025
  
  Publication
- Report from the 4th Strategic Workshop on Information Retrieval in Lorne (SWIRL 2025)
  
  Johanne R Trippas, J. Shane Culpepper, Mohammad Aliannejadi, James Allan, Enrique Amigó, Jaime Arguello, Leif Azzopardi, Peter Bailey, Jamie Callan, Rob Capra, Nick Craswell, Bruce Croft, Jeff Dalton, Gianluca Demartini, Laura Dietz, Zhicheng Dou, Carsten Eickhoff, Michael Ekstrand, Nicola Ferro, Norbert Fuhr, Dorota Glowacka, Faegheh Hasibi, Danula Hettiachchi, Rosie Jones, Jaap Kamps, Noriko Kando, Sarvnaz Karimi, Makoto P. Kato, Bevan Koopman, Yiqun Liu, Chenglong Ma, Joel Mackenzie, Maria Maistro, Jiaxin Mao, Dana McKay, Bhaskar Mitra, Stefano Mizzaro, Alistair Moffat, Josiane Mothe, Iadh Ounis, Lida Rashidi, Yongli Ren, Mark Sanderson, Rodrygo Santos, Falk Scholer, Chirag A Shah, Laurianne Sitbon, Ian Soboroff, Damiano Spina, Paul Thomas, Julián Urbano, Arjen P. de Vries, Ryen W. White, Abby Yuan, Hamed Zamani, Oleg Zendel, Min Zhang, Shengyao Zhuang, Justin Zobel, Guido Zuccon
  
  ACM SIGIR Forum | June 2025, Vol 59(1): pp. 1-68
  
  DOI PDF
- LLM4Eval@WSDM 2025: Large Language Model for Evaluation in Information Retrieval
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L A Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  2025 Web Search and Data Mining | March 2025
  
  Publication
- Judging the Judges: A Collection of LLM-Generated Relevance Judgements
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  February 2025
  
  Preprint
- JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment
  
  Hossein A. Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra
  
  2025 The Web Conference | January 2025
  
  Publication Preprint PDF
- SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval
  
  Hossein A. Rahmani, Xi Wang, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas
  
  2025 The Web Conference | January 2025
  
  Publication PDF Preprint
- LLMs can be Fooled into Labelling a Document as Relevant (Best café near me; this paper is perfectly relevant)
  
  Marwah Alaofi, Paul Thomas, Falk Scholer, Mark Sanderson
  
  International ACM SIGIR Conference on Information Retrieval in the Asia Pacific | December 2024
  
  Best paper nomination
  
  PDF DOI
- Proceedings of The First Workshop on Large Language Models for Evaluation in Information Retrieval (LLM4Eval 2024)
  
  Clemencia Siro, Mohammad Aliannejadi, Hossein A. Rahmani, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  September 2024
  
  Publication
- LLMJudge: LLMs for Relevance Judgments
  
  Hossein A Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas, Charles L A Clarke, Mohammad Aliannejadi, Clemencia Siro, Guglielmo Faggioli
  
  August 2024
  
  Publication Preprint
- Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  August 2024
  
  Publication Preprint PDF
- Large Language Models Can Accurately Predict Searcher Preferences
  
  Paul Thomas, Seth Spielman, Nick Craswell, Bhaskar Mitra
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  An earlier version of this paper appeared as arXiv preprint arXiv:2309.10621v1 [cs.IR].
  
  Publication PDF PDF
- LLM4Eval: Large Language Model for Evaluation in IR
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  Workshop description
  
  Publication PDF
- What Matters in a Measure? A Perspective from Large-Scale Search Evaluation
  
  Paul Thomas, Gabriella Kazai, Nick Craswell, Seth Spielman
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  Best paper nomination
  
  DOI PDF
- Enhancing human annotation: Leveraging large language models and efficient batch processing
  
  Oleg Zendel, J Shane Culpepper, Falk Scholer, Paul Thomas
  
  Conference on Human Information Interaction and Retrieval | March 2024
  
  DOI PDF
- Can generative LLMs create query variants for test collections? An exploratory study
  
  Marwah Alaofi, Luke Gallagher, Mark Sanderson, Falk Scholer, Paul Thomas
  
  2023 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2023
  
  PDF DOI
- Taking Search to Task
  
  Chirag Shah, Ryen W. White, Paul Thomas, Bhaskar Mitra, Shawon Sarkar, Nicholas Belkin
  
  Proceedings of the 8th ACM SIGIR Conference On Human Information Interaction And Retrieval | January 2023
  
  Publication Preprint PDF
- A Multi-objective Optimization Framework for Multi-stakeholder Fairness-aware Recommendation
  
  Haolun Wu, Chen Ma, Bhaskar Mitra, Fernando Diaz, Xue Liu
  
  ACM Transactions on Information Systems (TOIS) | August 2022
  
  Publication Preprint PDF
- A flexible framework for offline effectiveness metrics
  
  Alistair Moffat, Joel Mackenzie, Paul Thomas, Leif Azzopardi
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2022
  
  Publication DOI
- Are We There Yet? A Decision Framework for Replacing Term-Based Retrieval with Dense Retrieval Systems
  
  Sebastian Hofstätter, Nick Craswell, Bhaskar Mitra, Hamed Zamani, Allan Hanbury
  
  June 2022
  
  Preprint
- Overview of the TREC 2021 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Text REtrieval Conference (TREC) | May 2022
  
  Publication PDF PDF Access
- Fostering Coopetition While Plugging Leaks: The Design and Implementation of the MS MARCO Leaderboards
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF Access
- Inconsistent Ranking Assumptions in Medical Search and Their Downstream Consequences
  
  Daniel Cohen, Kevin Du, Bhaskar Mitra, Laura Mercurio, Navid Rekabsaz, Carsten Eickhoff
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF
- Joint Multisided Exposure Fairness for Recommendation
  
  Bhaskar Mitra, Haolun Wu, Chen Ma, Fernando Diaz, Xue Liu
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF
- Search interfaces for biomedical searching: How do gaze, user perception, search behaviour and search performance relate?
  
  Ying-Hsang Liu, Paul Thomas, Tom Gedeon, Nicolay Rusnachenko
  
  ACM SIGIR Conference on Human Information Interaction and Retrieval | March 2022
  
  DOI
- The crowd is made of people: Observations from large-scale crowd labelling
  
  Paul Thomas, Gabriella Kazai, Ryen W. White, Nick Craswell
  
  ACM SIGIR Conference on Human Information Interaction and Retrieval | March 2022
  
  Publication DOI
- Report on the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021)
  
  Chirag Shah, Torsten Suel, Fernando Diaz, Bhaskar Mitra, Bárbara Poblete, Hussein Suleman, Suzan Verberne
  
  SIGIR Forum | February 2022, Vol 55(2)
  
  Publication PDF
- Bias-aware Fair Neural Ranking for Addressing Stereotypical Gender Biases
  
  Shirin SeyedSalehi, Amin Bigdeli, Negar Arabzadeh, Bhaskar Mitra, Morteza Zihayat, Ebrahim Bagheri
  
  Proceedings of the 25th International Conference on Extending Database Technology (EDBT) | February 2022
  
  Publication PDF PDF
- Exposing Query Identification for Search Transparency
  
  Ruohan Li, Jianxiang Li, Bhaskar Mitra, Fernando Diaz, Asia Biega
  
  2022 The Web Conference | January 2022
  
  Publication Preprint PDF PDF
- Less is Less: When Are Snippets Insufficient for Human vs Machine Relevance Estimation?
  
  Gabriella Kazai, Bhaskar Mitra, Anlei Dong, Nick Craswell, Linjun Yang
  
  European Conference on Information Retrieval | November 2021
  
  Publication Preprint PDF PDF
- Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness
  
  Nicola Neophytou, Bhaskar Mitra, Catherine Stinson
  
  European Conference on Information Retrieval | November 2021
  
  Publication Preprint PDF PDF
- Analysing mixed initiatives and search strategies during conversational search
  
  Mohammad Aliannejadi, Leif Azzopardi, Hamed Zamani, Evangelos Kanoulas, Paul Thomas, Nick Craswell
  
  ACM International Conference on Information and Knowledge Management | November 2021
  
  DOI
- MS MARCO Chameleons: Challenging the MS MARCO Leaderboard with Extremely Obstinate Queries
  
  Negar Arabzadeh, Bhaskar Mitra, Ebrahim Bagheri
  
  Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM) | August 2021
  
  Publication PDF PDF Access
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  ACM Transactions on Information Systems | August 2021, Vol 39(4)
  
  Article 39
  
  Publication
- Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval (Dissertation Abstract)
  
  Bhaskar Mitra
  
  SIGIR Forum | July 2021, Vol 55(1)
  
  Publication PDF PDF
- Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
  
  Sebastian Hofstatter, Bhaskar Mitra, Hamed Zamani, Nick Craswell, Allan Hanbury
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
  
  Publication Preprint PDF PDF
- Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Models
  
  Daniel Cohen, Bhaskar Mitra, Oleg Lesota, Navid Rekabsaz, Carsten Eickhoff
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
  
  Publication Preprint PDF
- Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF
- MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees, Ian Soboroff
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- Proceedings of the 2021 Conference on Human Information Interaction and Retrieval
  
  Falk Scholer, Paul Thomas, David Elsweiler, Hideo Joho, Noriko Kando, Catherine Smith
  
  March 2021
  
  Publication
- Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | March 2021
  
  Publication Preprint PDF Access
- Social competence in conversational retrieval
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  Future Conversations | March 2021
  
  Publication
- Overview of the TREC 2020 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos
  
  Text REtrieval Conference (TREC) | February 2021
  
  Publication Preprint PDF PDF Access
- Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification
  
  Jaime Arguello, Adam Ferguson, Emery Fine, Bhaskar Mitra, Hamed Zamani, Fernando Diaz
  
  Proceedings of the 6th international ACM SIGIR Conference on Human Information Interaction and Retrieval | January 2021
  
  Publication Preprint PDF PDF Github Access
- Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval
  
  Bhaskar Mitra
  
  December 2020
  
  PhD Thesis: University College London | December 2020
  
  Publication Preprint PDF PDF
- Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Text REtrieval Conference (TREC) | November 2020
  
  Publication Preprint PDF PDF Access
- Simulating information retrieval test collections
  
  David Hawking, Bodo von Billerbeck, Paul Thomas, Nick Craswell
  
  Published by Morgan & Claypool | September 2020
  
  Editor(s): Gary Marchionini
  
  ISBN: 9781681739571
  
  Synthesis Lectures on Information Concepts, Retrieval, and Services
  
  Publication DOI
- Semantic Product Search for Matching Structured Product Catalogs in E-Commerce
  
  Jason Ingyu Choi, Surya Kallumadi, Bhaskar Mitra, Eugene Agichtein, Faizan Javed
  
  August 2020
  
  Preprint
- Conformer-Kernel with Query Term Independence for Document Retrieval
  
  Bhaskar Mitra, Sebastian Hofstätter, Hamed Zamani, Nick Craswell
  
  July 2020
  
  Preprint Github
- Evaluating Stochastic Rankings with Expected Exposure
  
  Fernando Diaz, Bhaskar Mitra, Michael D. Ekstrand, Asia J. Biega, Ben Carterette
  
  Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | July 2020
  
  Best Long Research Paper Nominee
  
  Publication Preprint PDF
- Expressions of style in information-seeking conversation with an embodied agent
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2020
  
  PDF DOI
- MIMICS: A Large-Scale Data Collection for Search Clarification
  
  Hamed Zamani, Gord Lueck, Everest Chen, Rodolfo Quispe, Flint Luu, Nick Craswell
  
  June 2020
- ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search
  
  Nick Craswell, Daniel Campos, Bhaskar Mitra, Emine Yilmaz, Bodo von Billerbeck
  
  Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | June 2020
  
  Publication Preprint PDF Access
- Analyzing and Learning from User Interactions for Search Clarification
  
  Hamed Zamani, Bhaskar Mitra, Everest Chen, Gord Lueck, Fernando Diaz, Paul Bennett, Nick Craswell, Susan Dumais
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF PDF
- Local Self-Attention over Long Text for Efficient Document Retrieval
  
  Sebastian Hofstätter, Hamed Zamani, Bhaskar Mitra, Nick Craswell, Allan Hanbury
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF
- On the Reliability of Test Collections for Evaluating Systems of Different Types
  
  Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Daniel Campos
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | April 2020
  
  Publication Preprint PDF PDF Access
- Data-driven evaluation metrics for heterogeneous search engine result pages
  
  Leif Azzopardi, Ryen W. White, Paul Thomas, Nick Craswell
  
  Proceedings of the Conference on Human Information Interaction and Retrieval | March 2020
  
  DOI Publication PDF
- Overview of the TREC 2019 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees
  
  Text REtrieval Conference (TREC) | March 2020
  
  Publication PDF PDF Access
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  International Workshop on Conversational Approaches to Information Retrieval | March 2020
  
  An extended version of this paper also appears as Thomas et al., ACM Trans. Info. Sys. 39(2), 2021: https://dl.acm.org/doi/pdf/10.1145/3439869.
  
  Publication
- Towards a model for spoken conversational search
  
  Johanne R Trippas, Damiano Spina, Paul Thomas, Mark Sanderson, Hideo Joho, Lawrence Cavedon
  
  Information Processing and Management | March 2020, Vol 57(2)
- Duet at TREC 2019 Deep Learning Track
  
  Bhaskar Mitra, Nick Craswell
  
  Text REtrieval Conference (TREC) | December 2019
  
  Publication Preprint PDF PDF Access
- Investigating searchers’ mental models to inform search explanations
  
  Paul Thomas, Bodo von Billerbeck, Nick Craswell, Ryen W. White
  
  ACM Transactions on Information Systems | December 2019, Vol 38(1)
  
  Article 10
  
  DOI Publication
- Report on the First HIPstIR Workshop on the Future of Information Retrieval
  
  Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
  
  SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
  
  Publication PDF
- cwl_eval: An evaluation tool for information retrieval
  
  Leif Azzopardi, Paul Thomas, Alistair Moffat
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2019
  
  Publication
- The Emotion Profile of Web Search
  
  Gabriella Kazai, Paul Thomas, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2019
  
  Publication
- Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
  
  Bhaskar Mitra, Corby Rosset, David Hawking, Nick Craswell, Fernando Diaz, Emine Yilmaz
  
  July 2019
  
  Preprint Project Project
- Building economic models and measures of search (tutorial)
  
  Leif Azzopardi, Alistair Moffat, Paul Thomas, Guido Zuccon
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | June 2019
  
  Publication
- An Axiomatic Approach to Regularizing Neural Ranking Models
  
  Corby Rosset, Bhaskar Mitra, Chenyan Xiong, Nick Craswell, Xia Song, Saurabh Tiwary
  
  Proceedings of the 42nd international ACM SIGIR conference on Research & development in information retrieval | April 2019
  
  Publication Preprint PDF PDF Project Project
- An updated duet model for passage re-ranking
  
  Bhaskar Mitra, Nick Craswell
  
  March 2019
  
  Preprint
- Data sets for spoken conversational search
  
  Johanne Trippas, Paul Thomas
  
  Workshop on Barriers to Interactive IR Resources Re-use at the ACM SIGIR Conference on Human Information Interaction and Retrieval | March 2019
  
  Publication
- An Introduction to Neural Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Foundations and Trends® in Information Retrieval | December 2018, Vol 13(1): pp. 1-126
  
  Publication PDF PDF
- Better effectiveness metrics for SERPs, cards, and rankings
  
  Paul Thomas, Alistair Moffat, Peter Bailey, Falk Scholer, Nick Craswell
  
  Australasian Document Computing Symposium | December 2018
  
  Best paper award
  
  DOI Publication
- A Line in the Sand: Recommendation or Ad-hoc Retrieval?
  
  Surya Kallumadi, Bhaskar Mitra, Tereza Iofciu
  
  Proceedings of the Recommender Systems Challenge 2018 | July 2018
  
  Preprint PDF
- Measuring the Utility of Search Engine Result Pages: An Information Foraging Based Measure
  
  Leif Azzopardi, Paul Thomas, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2018
  
  Publication Publication Publication
- Cross Domain Regularization for Neural Ranking Models Using Adversarial Learning
  
  Daniel Cohen, Bhaskar Mitra, Katja Hofmann, W. Bruce Croft
  
  Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
  
  Best Short Research Paper
  
  Publication Preprint PDF
- Optimizing Query Evaluations Using Reinforcement Learning for Web Search
  
  Corby Rosset, Damien Jose, Gargi Ghosh, Bhaskar Mitra, Saurabh Tiwary
  
  Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
  
  Publication Preprint PDF Project Project
- Neural Networks for Information Retrieval (ECIR 2018)
  
  Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 40th European Conference on IR Research (ECIR) | March 2018
  
  Advances in Information Retrieval
  
  PDF PDF
- Style and alignment in information-seeking conversation
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell, Gloria Mark
  
  Proceedings of the Conference on Computer-Human Information Interaction and Retrieval | March 2018
  
  Publication Publication
- Neural Networks for Information Retrieval (WSDM 2018)
  
  Tom Kenter, Alexey Borisov, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining | January 2018
  
  Preprint
- Tasks, queries, and rankers in pre-retrieval performance prediction
  
  Paul Thomas, Falk Scholer, Peter Bailey, Alistair Moffat
  
  Proceedings of the Australasian Document Computing Symposium | December 2017
  
  Publication
- Neural Ranking Models with Multiple Document Fields
  
  Hamed Zamani, Bhaskar Mitra, Xia Song, Nick Craswell, Saurabh Tiwary
  
  Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM'18) | November 2017
  
  Publication Preprint PDF Project Project
- Multimodal analysis of vocal collaborative search: A public corpus and results
  
  Daniel McDuff, Paul Thomas, Mary Czerwinski, Nick Craswell
  
  Proceedings of the ACM International Conference on Multimodal Interaction | November 2017
  
  Publication Publication
- Neural information retrieval: introduction to the special issue
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Information Retrieval Journal : Special issue Neural | November 2017
  
  Publication PDF
- Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rĳke, Jiafeng Guo, Bhaskar Mitra
  
  SIGIR Forum | October 2017, Vol 52
  
  Publication PDF PDF
- Reply With: Proactive Recommendation of Email Attachments
  
  Christophe Van Gysel, Bhaskar Mitra, Matteo Venanzi, Roy Rosemarin, Grzegorz Kukla, Piotr Grudzien, Nicola Cancedda
  
  Proceedings of the 26th ACM International Conference on Information and Knowledge Management (CIKM) | October 2017
  
  Publication Preprint PDF
- Benchmark for Complex Answer Retrieval
  
  Federico Nanni, Bhaskar Mitra, Matt Magnusson, Laura Dietz
  
  Proceedings of 3rd ACM International Conference on the Theory of Information Retrieval | October 2017
  
  Publication Preprint PDF
- MISC: A data set of information-seeking conversations
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the 1st International Workshop on Conversational Approaches to Information Retrieval | August 2017
  
  Publication
- SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval | August 2017
  
  Publication PDF
- Natural search user interfaces for complex biomedical search: An eye tracking study
  
  Ying-Hsang Liu, Paul Thomas, Marijana Bacic, Tom Gedeon, Xindi Li
  
  Journal of the Australian Library and Information Association | August 2017, Vol 66(4): pp. 364-381
  
  Publication Publication
- Luandri: a Clean Lua Interface to the Indri Search Engine
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 40th international ACM SIGIR conference on Research & development in information retrieval | August 2017
  
  Publication Preprint PDF PDF
- Retrieval consistency in the presence of query variations
  
  Peter Bailey, Alistair Moffat, Falk Scholer, Paul Thomas
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | August 2017
  
  Publication Publication
- Toward Incorporation of Relevant Documents in word2vec
  
  Navid Rekabsaz, Bhaskar Mitra, Mihai Lupu, Allan Hanbury
  
  Proceedings of the SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR'17) | July 2017
  
  Preprint
- Neural Networks for Information Retrieval (SIGIR 2017)
  
  Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2017
  
  Publication Preprint PDF
- Incorporating user expectations and behavior into the measurement of search effectiveness
  
  Alistair Moffat, Peter Bailey, Falk Scholer, Paul Thomas
  
  ACM Transactions on Information Systems | June 2017, Vol 35(3)
  
  Publication Publication
- Neural Models for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  May 2017
  
  Preprint Video
- Learning to Match Using Local and Distributed Representations of Text for Web Search
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 26th International Conference on World Wide Web | April 2017
  
  Publication Preprint PDF PDF
- A Theoretical Framework for Conversational Search
  
  Filip Radlinski, Nick Craswell
  
  CHIIR 2017 | March 2017
  
  Publication
- What snippet size is needed in mobile web search?
  
  Jaewon Kim, Paul Thomas, Ramesh Sankaranarayana, Tom Gedeon
  
  Proceedings of the Conference on Computer-Human Information Interaction and Retrieval | March 2017
  
  Publication Publication
- Neural Text Embeddings for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM'17) | February 2017
  
  Publication PDF
- Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rĳke, Bhaskar Mitra
  
  SIGIR Forum | December 2016, Vol 50(2): pp. 96-103
  
  Publication PDF
- Query-biased summaries for tabular data
  
  Vincent Au, Paul Thomas, Gaya K Jayasinghe
  
  Proceedings of the Australasian Document Computing Symposium | December 2016
  
  Publication Publication Publication
- Understanding eye movements on mobile devices for better presentation of search results
  
  Jaewon Kim, Paul Thomas, Ramesh Sankaranarayana, Tom Gedeon, Hwan-Jin Yoon
  
  Journal of the Association for Information Science and Technology | November 2016, Vol 67(11): pp. 2607-2619
  
  Publication Publication
- Pagination versus scrolling in mobile web search
  
  Jaewon Kim, Paul Thomas, Ramesh Sankaranarayana, Tom Gedeon, Hwan-Jin Yoon
  
  Proceedings of the International ACM SIGIR Conference on on Information and Knowledge Management | October 2016
  
  Publication Publication
- Ingrams – a neuropsychological explanation for why people search
  
  Peter Bailey, Nick Craswell
  
  Proceedings of the 39th Annual ACM SIGIR Conference (SIGIR 2016) | July 2016
  
  Publication Publication
- Neu-IR: The SIGIR 2016 Workshop on Neural Information Retrieval
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2016
  
  Publication PDF
- UQV100: A Test Collection with Query Variability
  
  Peter Bailey, Alistair Moffat, Falk Scholer, Paul Thomas
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2016
  
  Publication Publication
- A Proposal for Evaluating Answer Distillation from Web Data
  
  Bhaskar Mitra, Grady Simon, Jianfeng Gao, Nick Craswell, Li Deng
  
  Proceedings of the Second WebQA Workshop (SIGIR 2016) | July 2016
  
  PDF PDF
- Towards Conversational Recommender Systems
  
  Konstantina Christakopoulou, Filip Radlinski, Katja Hofmann
  
  22nd ACM SigKDD Conference on Knowledge Discovery and Data Mining | June 2016
  
  Publication Project
- Query Expansion with Locally-Trained Word Embeddings
  
  Fernando Diaz, Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics | June 2016
  
  Publication Preprint PDF PDF
- Improving Document Ranking with Dual Word Embeddings
  
  Eric Nalisnick, Bhaskar Mitra, Nick Craswell, Rich Caruana
  
  WWW'16 | April 2016
  
  Publication PDF PDF Project
- Did You Say U2 or Youtube? Inferring Implicit Transcripts from Voice Search Logs
  
  Nick Craswell, Umut Ozertem, Milad Shokouhi
  
  Proceedings of the 25th International World Wide Web Conference (WWW2016) | April 2016
  
  Publication
- Where Can I Buy a Boulder? Searching for Offline Retail Locations
  
  Sandro Bauer, Filip Radlinski, Ryen W. White
  
  Proceedings of the 25th International World Wide Web Conference (WWW) (to appear) | April 2016
  
  Publication
- A Dual Embedding Space Model for Document Ranking
  
  Bhaskar Mitra, Eric Nalisnick, Nick Craswell, Rich Caruana
  
  February 2016
  
  This paper is an extended evaluation and analysis of the model proposed in a poster to appear in WWW'16, April 11 - 15, 2016, Montreal, Canada.
  
  Preprint PDF Access Project
- Query Auto-Completion for Rare Prefixes
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM) | October 2015
  
  Publication PDF PDF
- Exploring Session Context using Distributed Representations of Queries and Reformulations
  
  Bhaskar Mitra
  
  Proceedings of the 38th international ACM SIGIR conference on Research & development in information retrieval | August 2015
  
  Publication PDF PDF
- Large Scale Log Analysis of Individuals’ Domain Preferences in Web Search
  
  Sarah K. Tyler, Jaime Teevan, Peter Bailey, Sebastian de la Chica, Nikhil Dandekar
  
  MSR-TR-2015-048 | June 2015
  
  Publication
- An Eye-tracking Study of User Interactions with Query Auto Completion
  
  Katja Hofmann, Bhaskar Mitra, Filip Radlinski, Milad Shokouhi
  
  Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
  
  Publication PDF Video
- Relevance and Effort: An Analysis of Document Utility
  
  Emine Yilmaz, Manisha Verma, Nick Craswell, Filip Radlinski, Peter Bailey
  
  Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
  
  Publication
- On User Interactions with Query Auto-Completion
  
  Bhaskar Mitra, Milad Shokouhi, Filip Radlinski, Katja Hofmann
  
  Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR) | July 2014
  
  Publication PDF
- Beyond Clicks: Query Reformulation as a Predictor of Search Satisfaction
  
  Ahmed Awadallah, Xiaolin Shi, Nick Craswell, Bill Ramsey
  
  ACM International Conference on Information and Knowledge Management (CIKM) | October 2013
  
  Publication
- Epigenome-wide association studies without the need for cell-type composition
  
  James Zhou, Christoph Lippert, David Heckerman, Martin Aryee, Jennifer Listgarten
  
  Nature Methods | August 2013
  
  Publication Publication
- Optimized Interleaving for Online Retrieval Evaluation
  
  Filip Radlinski, Nick Craswell
  
  ACM International Conference on Web Search and Data Mining (WSDM) | February 2013
  
  ACM International Conference on Web Search and Data Mining (WSDM)
  
  Best Paper Award
  
  Publication
- Robust Query Rewriting using Anchor Data
  
  Nick Craswell, Bodo von Billerbeck, Dennis Fetterly, Marc Najork
  
  6th ACM International Conference on Web Search and Data Mining (WSDM) | February 2013
  
  Publication
- Modeling the Impact of Short- and Long-Term Behavior on Search Personalization
  
  Paul Bennett, Ryen W. White, Wei Chu, Susan Dumais, Peter Bailey, Fedor Borisyuk, Xiaoyuan Cui
  
  Proceedings of the 35th Annual ACM SIGIR Conference (SIGIR 2012) | August 2012
  
  Publication
- Large Scale Validation and Analysis of Interleaved Search Evaluation
  
  Olivier Chapelle, Thorsten Joachims, Filip Radlinski, Yisong Yue
  
  Transactions on Information Systems (TOIS) | February 2012, Vol 30
  
  Selected for ACM Notable Computing Books and Articles of 2012
  
  Publication
- Probabilistic Models for Personalizing Web Search
  
  David Sontag, Kevyn Collins-Thompson, Paul Bennett, Ryen W. White, Susan Dumais, Bodo von Billerbeck
  
  WSDM 2012: Fifth International Conference on Web Search and Data Miniing | February 2012
  
  Publication
- Microsoft Research at TREC 2011 Web Track
  
  Bodo von Billerbeck, Nick Craswell, Dennis Fetterly, Marc Najork
  
  Proc. of the 20th Text Retrieval Conference (TREC) | November 2011
  
  Publication
- Inferring and Using Location Metadata to Personalize Web Search
  
  Paul Bennett, Filip Radlinski, Ryen W. White, Emine Yilmaz
  
  Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR) | July 2011
  
  Publication
- The Power of Peers
  
  Nick Craswell, Dennis Fetterly, Marc Najork
  
  33rd European Conference on IR Research (ECIR) | April 2011
  
  33rd European Conference on IR Research (ECIR)
  
  Publication
- LambdaMerge: Merging the Results of Query Reformulations
  
  Daniel Sheldon, Milad Shokouhi, Martin Szummer, Nick Craswell
  
  Conf. Web search and data mining (WSDM) | February 2011
  
  Publication Publication
- Personalizing Web Search using Long Term Browsing History
  
  Nicolaas Matthijs, Filip Radlinski
  
  ACM International Conference on Web Search and Data Mining (WSDM) | February 2011
  
  Publication
- Microsoft Research at TREC 2010 Web Track
  
  Nick Craswell, Dennis Fetterly, Marc Najork
  
  Proc. of the 19th Text Retrieval Conference (TREC) | November 2010
  
  Proc. of the 19th Text Retrieval Conference (TREC)
  
  Publication
- Comparing the Sensitivity of Information Retrieval Metrics
  
  Filip Radlinski, Nick Craswell
  
  Proceedings of SIGIR | July 2010
  
  Publication
- Metrics for Assessing Sets of Subtopics
  
  Filip Radlinski, Martin Szummer, Nick Craswell
  
  SIGIR Conf. Research and Development in Information Retrieval | July 2010
  
  Publication
- Inferring Query Intent from Reformulations and Clicks
  
  Filip Radlinski, Martin Szummer, Nick Craswell
  
  Proc. 19th Annual International World Wide Web Conference (WWW '10). | April 2010
  
  Proc. 19th Annual International World Wide Web Conference (WWW '10).
  
  Publication
- Microsoft Research at TREC 2009: Web and Relevance Feedback Tracks
  
  Nick Craswell, Dennis Fetterly, Marc Najork, Stephen Robertson, Emine Yilmaz
  
  Proc. of the 18th Text Retrieval Conference (TREC) | November 2009
  
  Publication
- The Impact of Crawl Policy on Web Search Effectiveness
  
  Dennis Fetterly, Nick Craswell, Vishwa Vinay
  
  Proceedings of the 32nd annual international ACM SIGIR conference on Research and development in information retrieval | July 2009
  
  Publication
- Measuring the Search Effectiveness of a Breadth-First Crawl
  
  Dennis Fetterly, Nick Craswell, Vishwa Vinay
  
  Proceedings of the 31st European Conference on Information Retrieval (ECIR) | April 2009
  
  Proceedings of the 31st European Conference on Information Retrieval (ECIR)
  
  Publication
- A Two-Stage Model for Expert Search
  
  Yunbo Cao, Jingjing Liu, Shenghua Bao, Hang Li, Nick Craswell
  
  MSR-TR-2008-143 | October 2008
  
  Publication
- A Two-Stage Model for Expert Search
  
  Yunbo Cao, JJ (Jingjing) Liu, Shenghua Bao, Hang Li, Nick Craswell
  
  MSR-TR-2008-195 | October 2008
  
  Publication
- Efficient and Effective Link Analysis with Precomputed SALSA Maps
  
  Marc Najork, Nick Craswell
  
  17th ACM Conference on Information and Knowledge Management (CIKM) | October 2008
  
  17th ACM Conference on Information and Knowledge Management (CIKM)
  
  Publication
- A Decision Theoretic Framework for Ranking using Implicit Feedback
  
  Onno Zoeter, Mike Taylor, Ed Snelson, John Guiver, Nick Craswell, Martin Szummer
  
  SIGIR 2008 Workshop on Learning to Rank for Information Retrieval | July 2008
  
  Publication
- Behavioral Classification on the Click Graph
  
  Martin Szummer, Nick Craswell
  
  World Wide Web Conference | April 2008
  
  World Wide Web Conference
  
  Publication
- An experimental comparison of click position-bias models
  
  Nick Craswell, Mike Taylor
  
  WSDM '08: Proceedings of the international conference on Web search and web data mining | January 2008
  
  ISBN: 978-1-59593-927-9
  
  Publication Publication
- Overview of the INEX 2007 Entity Ranking Track
  
  Arjen P. de Vries, Anne-Marie Vercoustre, James A. Thom, Nick Craswell, Mounia Lalmas
  
  January 2008
  
  ISBN: 978-3-540-85901-7
  
  Publication Publication
- Relevance assessment: are judges exchangeable and does it matter
  
  Peter Bailey, Nick Craswell, Ian Soboroff, Paul Thomas, Arjen P. de Vries, Emine Yilmaz
  
  SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval | January 2008
  
  ISBN: 978-1-60558-164-4
  
  Publication Publication
- Random Walks on the Click Graph
  
  Nick Craswell, Martin Szummer
  
  Proceedings of SIGIR 2007 | July 2007
  
  ISBN: 978-1-59593-597-7
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_sigir07.pdf
  
  Publication Github
- The CSIRO Enterprise Search Test Collection
  
  Peter Bailey, Nick Craswell, Ian Soboroff, Arjen P. de Vries
  
  SIGIR Forum | January 2007, Vol 41(2): pp. 42-45
  
  Publication
- Optimisation methods for ranking functions with multiple parameters
  
  Mike Taylor, Hugo Zaragoza, Nick Craswell, Stephen Robertson, Chris J.C. Burges
  
  Proceedings of CIKM 2006 | January 2006
  
  Proceedings of CIKM 2006
  
  ISBN: 1-59593-433-2
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/taylor_cikm06.pdf
  
  Publication
- Quality and Relevance of Domain-specific Search: A Case Study in Mental Health
  
  Thanh Tang, Nick Craswell, David Hawking, Kathleen Griffiths, Helen Christensen
  
  Information Retrieval | January 2006, pp. 207-225
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/tang_ir06.pdf
  
  Publication
- Focused crawling for both topical relevance and quality of medical information
  
  Thanh Tang, David Hawking, Nick Craswell, Kathleen Griffiths
  
  Proceedings of CIKM 2005 | January 2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/tang_cikm05.pdf
  
  Publication
- Microsoft Cambridge at TREC-14: Enterprise Track
  
  Nick Craswell, Hugo Zaragoza, Stephen Robertson
  
  Proceedings of TREC-2005 | January 2005
  
  Proceedings of TREC-2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec05.pdf
  
  Publication
- Relevance Weighting for Query Independent Evidence
  
  Nick Craswell, Stephen Robertson, Hugo Zaragoza, Mike Taylor
  
  Proceedings of ACM SIGIR 2005 | January 2005
  
  ISBN: 1-59593-034-5
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_sigir05.pdf
  
  Publication
- The TREC terabyte retrieval track
  
  Charles Clarke, Nick Craswell, Ian Soboroff
  
  SIGIR Forum | January 2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/clarke_sigirforum05.pdf
  
  Publication
- RMIT University at TREC 2004
  
  Bodo von Billerbeck, Adam Cannane, Abhijit Chattaraj, Nicholas Lester, William Webber, Hugh E. Williams, John Yiannis, Justin Zobel
  
  Proceedings Text Retrieval Conference (TREC) | November 2004
  
  Publication
- Microsoft Cambridge at TREC 2004: Web and HARD track
  
  Hugo Zaragoza, Nick Craswell, Mike Taylor, Suchi Saria, Stephen Robertson
  
  Proceedings of TREC-2004 | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/zaragoza_trec04.pdf
  
  Publication Publication
- Testbed for Information Extraction from Deep Web
  
  Yasuhiro Yamada, Nick Craswell, Tetsuya Nakatoh, Sachio Hirokawa
  
  WWW2004 Poster Proceedings | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/yamada_www2004poster.pdf
  
  Publication
- Toward Better Weighting of Anchors
  
  David Hawking, Trystan Upstill, Nick Craswell
  
  Proceedings of ACM SIGIR 2004 (poster abstract) | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_sigir04poster.pdf
  
  Publication
- Predicting Fame and Fortune: PageRank or Indegree?
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  Proceedings of the Australasian Document Computing Symposium, ADCS2003 | December 2003
  
  Proceedings of the Australasian Document Computing Symposium, ADCS2003
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_adcs03.pdf
  
  Publication
- When query expansion fails
  
  Bodo von Billerbeck
  
  Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval | July 2003
  
  ISBN: 1-58113-646-3
  
  Publication
- Automated Discovery of Search Interfaces on the Web
  
  Jared Cope, Nick Craswell, David Hawking
  
  The Fourteenth Australasian Database Conference | January 2003
  
  The Fourteenth Australasian Database Conference
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/cope_adc03.pdf
  
  Publication
- Query expansion using associated queries
  
  Bodo von Billerbeck
  
  Proceedings of the twelfth international conference on information and knowledge management | January 2003
  
  ISBN: 1-58113-723-0
  
  Publication
- Query-independent evidence in home page finding
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  January 2003
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_tois03.pdf
  
  Publication
- Overview of the TREC-2002 Web Track
  
  Nick Craswell, David Hawking
  
  Proceedings of TREC-2002 | November 2002
  
  Proceedings of TREC-2002
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec02wt.pdf
  
  Publication
- Buying bestsellers online: A case study in Search & Searchability
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  7th Australasian Document Computing Symposium | January 2002
  
  7th Australasian Document Computing Symposium
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_adcs02.pdf
  
  Publication
- Which Search Engine is Best at Finding Online Services?
  
  David Hawking, Nick Craswell, Kathleen Griffiths
  
  Poster Proceedings of WWW10 | May 2001
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_www01poster.pdf
  
  Publication
- TREC10 Web and Interactive Tracks at CSIRO
  
  Nick Craswell, David Hawking, Ross Wilkinson, Mingfang Wu
  
  Proceedings of TREC-2001 | November 2001
  
  Proceedings of TREC-2001
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec01.pdf
  
  Publication
- Methods for Distributed Information Retrieval
  
  Nick Craswell
  
  January 2000
  
  January 2000
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_thesis00.pdf
  
  Publication
- ACSys TREC-7 experiments
  
  David Hawking, Nick Craswell, Paul Thistlewaite
  
  TREC-7, NIST special publication 500-242 | January 1998
  
  TREC-7, NIST special publication 500-242
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_trec98.pdf
  
  Publication
- Aglets: A good idea for spidering?
  
  Jason Haines, Brendan Humphreys, Chris Johnson Nick Craswell, Paul Thistlewaite, Nick Craswell
  
  4th IDEA Workshop Proceedings | January 1997
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_idea97.pdf
  
  Publication

- All claims are equal, but some claims are more equal than others: Importance-sensitive factuality evaluation of LLM generations
  
  Miriam Wanner, Leif Azzopardi, Paul Thomas, Soham Dan, Ben Van Durme, Nick Craswell
  
  October 2025
  
  preprint arXiv:2510.07083
  
  Publication
- Preaching to the ChoIR: Lessons IR should share with AI
  
  Gianluca Demartini, Claudia Hauff, Matthew Lease, Stefano Mizzaro, Kevin Roitero, Mark Sanderson, Falk Scholer, Chirag Shah, Damiano Spina, Paul Thomas, Arjen P de Vries, Guido Zuccon
  
  International Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR) | July 2025
  
  DOI
- Judging the Judges: A Collection of LLM-Generated Relevance Judgements
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  February 2025
  
  Preprint
- JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment
  
  Hossein A. Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra
  
  2025 The Web Conference | January 2025
  
  Publication Preprint PDF
- SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval
  
  Hossein A. Rahmani, Xi Wang, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas
  
  2025 The Web Conference | January 2025
  
  Publication PDF Preprint
- Proceedings of The First Workshop on Large Language Models for Evaluation in Information Retrieval (LLM4Eval 2024)
  
  Clemencia Siro, Mohammad Aliannejadi, Hossein A. Rahmani, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  September 2024
  
  Publication
- Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  August 2024
  
  Publication Preprint PDF
- Large Language Models Can Accurately Predict Searcher Preferences
  
  Paul Thomas, Seth Spielman, Nick Craswell, Bhaskar Mitra
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  An earlier version of this paper appeared as arXiv preprint arXiv:2309.10621v1 [cs.IR].
  
  Publication PDF PDF
- A Multi-objective Optimization Framework for Multi-stakeholder Fairness-aware Recommendation
  
  Haolun Wu, Chen Ma, Bhaskar Mitra, Fernando Diaz, Xue Liu
  
  ACM Transactions on Information Systems (TOIS) | August 2022
  
  Publication Preprint PDF
- Are We There Yet? A Decision Framework for Replacing Term-Based Retrieval with Dense Retrieval Systems
  
  Sebastian Hofstätter, Nick Craswell, Bhaskar Mitra, Hamed Zamani, Allan Hanbury
  
  June 2022
  
  Preprint
- Overview of the TREC 2021 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Text REtrieval Conference (TREC) | May 2022
  
  Publication PDF PDF Access
- Joint Multisided Exposure Fairness for Recommendation
  
  Bhaskar Mitra, Haolun Wu, Chen Ma, Fernando Diaz, Xue Liu
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF
- Inconsistent Ranking Assumptions in Medical Search and Their Downstream Consequences
  
  Daniel Cohen, Kevin Du, Bhaskar Mitra, Laura Mercurio, Navid Rekabsaz, Carsten Eickhoff
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF
- Fostering Coopetition While Plugging Leaks: The Design and Implementation of the MS MARCO Leaderboards
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF Access
- Bias-aware Fair Neural Ranking for Addressing Stereotypical Gender Biases
  
  Shirin SeyedSalehi, Amin Bigdeli, Negar Arabzadeh, Bhaskar Mitra, Morteza Zihayat, Ebrahim Bagheri
  
  Proceedings of the 25th International Conference on Extending Database Technology (EDBT) | February 2022
  
  Publication PDF PDF
- Exposing Query Identification for Search Transparency
  
  Ruohan Li, Jianxiang Li, Bhaskar Mitra, Fernando Diaz, Asia Biega
  
  2022 The Web Conference | January 2022
  
  Publication Preprint PDF PDF
- Less is Less: When Are Snippets Insufficient for Human vs Machine Relevance Estimation?
  
  Gabriella Kazai, Bhaskar Mitra, Anlei Dong, Nick Craswell, Linjun Yang
  
  European Conference on Information Retrieval | November 2021
  
  Publication Preprint PDF PDF
- Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness
  
  Nicola Neophytou, Bhaskar Mitra, Catherine Stinson
  
  European Conference on Information Retrieval | November 2021
  
  Publication Preprint PDF PDF
- MS MARCO Chameleons: Challenging the MS MARCO Leaderboard with Extremely Obstinate Queries
  
  Negar Arabzadeh, Bhaskar Mitra, Ebrahim Bagheri
  
  Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM) | August 2021
  
  Publication PDF PDF Access
- Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval (Dissertation Abstract)
  
  Bhaskar Mitra
  
  SIGIR Forum | July 2021, Vol 55(1)
  
  Publication PDF PDF
- Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
  
  Sebastian Hofstatter, Bhaskar Mitra, Hamed Zamani, Nick Craswell, Allan Hanbury
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
  
  Publication Preprint PDF PDF
- Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Models
  
  Daniel Cohen, Bhaskar Mitra, Oleg Lesota, Navid Rekabsaz, Carsten Eickhoff
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
  
  Publication Preprint PDF
- TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees, Ian Soboroff
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF
- MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | March 2021
  
  Publication Preprint PDF Access
- Overview of the TREC 2020 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos
  
  Text REtrieval Conference (TREC) | February 2021
  
  Publication Preprint PDF PDF Access
- Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval
  
  Bhaskar Mitra
  
  December 2020
  
  PhD Thesis: University College London | December 2020
  
  Publication Preprint PDF PDF
- Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Text REtrieval Conference (TREC) | November 2020
  
  Publication Preprint PDF PDF Access
- Semantic Product Search for Matching Structured Product Catalogs in E-Commerce
  
  Jason Ingyu Choi, Surya Kallumadi, Bhaskar Mitra, Eugene Agichtein, Faizan Javed
  
  August 2020
  
  Preprint
- Conformer-Kernel with Query Term Independence for Document Retrieval
  
  Bhaskar Mitra, Sebastian Hofstätter, Hamed Zamani, Nick Craswell
  
  July 2020
  
  Preprint Github
- Evaluating Stochastic Rankings with Expected Exposure
  
  Fernando Diaz, Bhaskar Mitra, Michael D. Ekstrand, Asia J. Biega, Ben Carterette
  
  Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | July 2020
  
  Best Long Research Paper Nominee
  
  Publication Preprint PDF
- MIMICS: A Large-Scale Data Collection for Search Clarification
  
  Hamed Zamani, Gord Lueck, Everest Chen, Rodolfo Quispe, Flint Luu, Nick Craswell
  
  June 2020
- ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search
  
  Nick Craswell, Daniel Campos, Bhaskar Mitra, Emine Yilmaz, Bodo von Billerbeck
  
  Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | June 2020
  
  Publication Preprint PDF Access
- Analyzing and Learning from User Interactions for Search Clarification
  
  Hamed Zamani, Bhaskar Mitra, Everest Chen, Gord Lueck, Fernando Diaz, Paul Bennett, Nick Craswell, Susan Dumais
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF PDF
- Local Self-Attention over Long Text for Efficient Document Retrieval
  
  Sebastian Hofstätter, Hamed Zamani, Bhaskar Mitra, Nick Craswell, Allan Hanbury
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF
- On the Reliability of Test Collections for Evaluating Systems of Different Types
  
  Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Daniel Campos
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | April 2020
  
  Publication Preprint PDF PDF Access
- Overview of the TREC 2019 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees
  
  Text REtrieval Conference (TREC) | March 2020
  
  Publication PDF PDF Access
- Duet at TREC 2019 Deep Learning Track
  
  Bhaskar Mitra, Nick Craswell
  
  Text REtrieval Conference (TREC) | December 2019
  
  Publication Preprint PDF PDF Access
- Report on the First HIPstIR Workshop on the Future of Information Retrieval
  
  Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
  
  SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
  
  Publication PDF
- Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
  
  Bhaskar Mitra, Corby Rosset, David Hawking, Nick Craswell, Fernando Diaz, Emine Yilmaz
  
  July 2019
  
  Preprint Project Project
- An Axiomatic Approach to Regularizing Neural Ranking Models
  
  Corby Rosset, Bhaskar Mitra, Chenyan Xiong, Nick Craswell, Xia Song, Saurabh Tiwary
  
  Proceedings of the 42nd international ACM SIGIR conference on Research & development in information retrieval | April 2019
  
  Publication Preprint PDF PDF Project Project
- An updated duet model for passage re-ranking
  
  Bhaskar Mitra, Nick Craswell
  
  March 2019
  
  Preprint
- An Introduction to Neural Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Foundations and Trends® in Information Retrieval | December 2018, Vol 13(1): pp. 1-126
  
  Publication PDF PDF
- A Line in the Sand: Recommendation or Ad-hoc Retrieval?
  
  Surya Kallumadi, Bhaskar Mitra, Tereza Iofciu
  
  Proceedings of the Recommender Systems Challenge 2018 | July 2018
  
  Preprint PDF
- Cross Domain Regularization for Neural Ranking Models Using Adversarial Learning
  
  Daniel Cohen, Bhaskar Mitra, Katja Hofmann, W. Bruce Croft
  
  Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
  
  Best Short Research Paper
  
  Publication Preprint PDF
- Optimizing Query Evaluations Using Reinforcement Learning for Web Search
  
  Corby Rosset, Damien Jose, Gargi Ghosh, Bhaskar Mitra, Saurabh Tiwary
  
  Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
  
  Publication Preprint PDF Project Project
- Neural Networks for Information Retrieval (ECIR 2018)
  
  Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 40th European Conference on IR Research (ECIR) | March 2018
  
  Advances in Information Retrieval
  
  PDF PDF
- Neural Networks for Information Retrieval (WSDM 2018)
  
  Tom Kenter, Alexey Borisov, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining | January 2018
  
  Preprint
- Neural Ranking Models with Multiple Document Fields
  
  Hamed Zamani, Bhaskar Mitra, Xia Song, Nick Craswell, Saurabh Tiwary
  
  Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM'18) | November 2017
  
  Publication Preprint PDF Project Project
- Neural information retrieval: introduction to the special issue
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Information Retrieval Journal : Special issue Neural | November 2017
  
  Publication PDF
- Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rĳke, Jiafeng Guo, Bhaskar Mitra
  
  SIGIR Forum | October 2017, Vol 52
  
  Publication PDF PDF
- Reply With: Proactive Recommendation of Email Attachments
  
  Christophe Van Gysel, Bhaskar Mitra, Matteo Venanzi, Roy Rosemarin, Grzegorz Kukla, Piotr Grudzien, Nicola Cancedda
  
  Proceedings of the 26th ACM International Conference on Information and Knowledge Management (CIKM) | October 2017
  
  Publication Preprint PDF
- Benchmark for Complex Answer Retrieval
  
  Federico Nanni, Bhaskar Mitra, Matt Magnusson, Laura Dietz
  
  Proceedings of 3rd ACM International Conference on the Theory of Information Retrieval | October 2017
  
  Publication Preprint PDF
- SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval | August 2017
  
  Publication PDF
- Luandri: a Clean Lua Interface to the Indri Search Engine
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 40th international ACM SIGIR conference on Research & development in information retrieval | August 2017
  
  Publication Preprint PDF PDF
- Toward Incorporation of Relevant Documents in word2vec
  
  Navid Rekabsaz, Bhaskar Mitra, Mihai Lupu, Allan Hanbury
  
  Proceedings of the SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR'17) | July 2017
  
  Preprint
- Neural Networks for Information Retrieval (SIGIR 2017)
  
  Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2017
  
  Publication Preprint PDF
- Neural Models for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  May 2017
  
  Preprint Video
- Learning to Match Using Local and Distributed Representations of Text for Web Search
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 26th International Conference on World Wide Web | April 2017
  
  Publication Preprint PDF PDF
- Neural Text Embeddings for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM'17) | February 2017
  
  Publication PDF
- Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rĳke, Bhaskar Mitra
  
  SIGIR Forum | December 2016, Vol 50(2): pp. 96-103
  
  Publication PDF
- Neu-IR: The SIGIR 2016 Workshop on Neural Information Retrieval
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2016
  
  Publication PDF
- A Proposal for Evaluating Answer Distillation from Web Data
  
  Bhaskar Mitra, Grady Simon, Jianfeng Gao, Nick Craswell, Li Deng
  
  Proceedings of the Second WebQA Workshop (SIGIR 2016) | July 2016
  
  PDF PDF
- Query Expansion with Locally-Trained Word Embeddings
  
  Fernando Diaz, Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics | June 2016
  
  Publication Preprint PDF PDF
- Improving Document Ranking with Dual Word Embeddings
  
  Eric Nalisnick, Bhaskar Mitra, Nick Craswell, Rich Caruana
  
  WWW'16 | April 2016
  
  Publication PDF PDF Project
- A Dual Embedding Space Model for Document Ranking
  
  Bhaskar Mitra, Eric Nalisnick, Nick Craswell, Rich Caruana
  
  February 2016
  
  This paper is an extended evaluation and analysis of the model proposed in a poster to appear in WWW'16, April 11 - 15, 2016, Montreal, Canada.
  
  Preprint PDF Access Project
- Query Auto-Completion for Rare Prefixes
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM) | October 2015
  
  Publication PDF PDF
- Exploring Session Context using Distributed Representations of Queries and Reformulations
  
  Bhaskar Mitra
  
  Proceedings of the 38th international ACM SIGIR conference on Research & development in information retrieval | August 2015
  
  Publication PDF PDF
- An Eye-tracking Study of User Interactions with Query Auto Completion
  
  Katja Hofmann, Bhaskar Mitra, Filip Radlinski, Milad Shokouhi
  
  Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
  
  Publication PDF Video
- Epigenome-wide association studies without the need for cell-type composition
  
  James Zhou, Christoph Lippert, David Heckerman, Martin Aryee, Jennifer Listgarten
  
  Nature Methods | August 2013
  
  Publication Publication
- LambdaMerge: Merging the Results of Query Reformulations
  
  Daniel Sheldon, Milad Shokouhi, Martin Szummer, Nick Craswell
  
  Conf. Web search and data mining (WSDM) | February 2011
  
  Publication Publication
- A Decision Theoretic Framework for Ranking using Implicit Feedback
  
  Onno Zoeter, Mike Taylor, Ed Snelson, John Guiver, Nick Craswell, Martin Szummer
  
  SIGIR 2008 Workshop on Learning to Rank for Information Retrieval | July 2008
  
  Publication
- Random Walks on the Click Graph
  
  Nick Craswell, Martin Szummer
  
  Proceedings of SIGIR 2007 | July 2007
  
  ISBN: 978-1-59593-597-7
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_sigir07.pdf
  
  Publication Github
- The CSIRO Enterprise Search Test Collection
  
  Peter Bailey, Nick Craswell, Ian Soboroff, Arjen P. de Vries
  
  SIGIR Forum | January 2007, Vol 41(2): pp. 42-45
  
  Publication
- Quality and Relevance of Domain-specific Search: A Case Study in Mental Health
  
  Thanh Tang, Nick Craswell, David Hawking, Kathleen Griffiths, Helen Christensen
  
  Information Retrieval | January 2006, pp. 207-225
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/tang_ir06.pdf
  
  Publication
- Query-independent evidence in home page finding
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  January 2003
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_tois03.pdf
  
  Publication
- Which Search Engine is Best at Finding Online Services?
  
  David Hawking, Nick Craswell, Kathleen Griffiths
  
  Poster Proceedings of WWW10 | May 2001
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_www01poster.pdf
  
  Publication
- Preaching to the ChoIR: Lessons IR should share with AI
  
  Gianluca Demartini, Claudia Hauff, Matthew Lease, Stefano Mizzaro, Kevin Roitero, Mark Sanderson, Falk Scholer, Chirag Shah, Damiano Spina, Paul Thomas, Arjen P de Vries, Guido Zuccon
  
  International Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR) | July 2025
  
  DOI
- Demographically-inspired query variants using an LLM
  
  Marwah Alaofi, Nicola Ferro, Paul Thomas, Falk Scholer, Mark Sanderson
  
  International Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR) | July 2025
  
  DOI Publication
- System comparison using automated generation of relevance judgements in multiple languages
  
  Paul Thomas, Douglas W Oard, Eugene Yang, Dawn Lawrie, James Mayfield
  
  2025 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2025
  
  Publication
- Report from the 4th Strategic Workshop on Information Retrieval in Lorne (SWIRL 2025)
  
  Johanne R Trippas, J. Shane Culpepper, Mohammad Aliannejadi, James Allan, Enrique Amigó, Jaime Arguello, Leif Azzopardi, Peter Bailey, Jamie Callan, Rob Capra, Nick Craswell, Bruce Croft, Jeff Dalton, Gianluca Demartini, Laura Dietz, Zhicheng Dou, Carsten Eickhoff, Michael Ekstrand, Nicola Ferro, Norbert Fuhr, Dorota Glowacka, Faegheh Hasibi, Danula Hettiachchi, Rosie Jones, Jaap Kamps, Noriko Kando, Sarvnaz Karimi, Makoto P. Kato, Bevan Koopman, Yiqun Liu, Chenglong Ma, Joel Mackenzie, Maria Maistro, Jiaxin Mao, Dana McKay, Bhaskar Mitra, Stefano Mizzaro, Alistair Moffat, Josiane Mothe, Iadh Ounis, Lida Rashidi, Yongli Ren, Mark Sanderson, Rodrygo Santos, Falk Scholer, Chirag A Shah, Laurianne Sitbon, Ian Soboroff, Damiano Spina, Paul Thomas, Julián Urbano, Arjen P. de Vries, Ryen W. White, Abby Yuan, Hamed Zamani, Oleg Zendel, Min Zhang, Shengyao Zhuang, Justin Zobel, Guido Zuccon
  
  ACM SIGIR Forum | June 2025, Vol 59(1): pp. 1-68
  
  DOI PDF
- LLM4Eval@WSDM 2025: Large Language Model for Evaluation in Information Retrieval
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L A Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  2025 Web Search and Data Mining | March 2025
  
  Publication
- Judging the Judges: A Collection of LLM-Generated Relevance Judgements
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  February 2025
  
  Preprint
- JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment
  
  Hossein A. Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra
  
  2025 The Web Conference | January 2025
  
  Publication Preprint PDF
- SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval
  
  Hossein A. Rahmani, Xi Wang, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas
  
  2025 The Web Conference | January 2025
  
  Publication PDF Preprint
- LLMs can be Fooled into Labelling a Document as Relevant (Best café near me; this paper is perfectly relevant)
  
  Marwah Alaofi, Paul Thomas, Falk Scholer, Mark Sanderson
  
  International ACM SIGIR Conference on Information Retrieval in the Asia Pacific | December 2024
  
  Best paper nomination
  
  PDF DOI
- Proceedings of The First Workshop on Large Language Models for Evaluation in Information Retrieval (LLM4Eval 2024)
  
  Clemencia Siro, Mohammad Aliannejadi, Hossein A. Rahmani, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  September 2024
  
  Publication
- LLMJudge: LLMs for Relevance Judgments
  
  Hossein A Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas, Charles L A Clarke, Mohammad Aliannejadi, Clemencia Siro, Guglielmo Faggioli
  
  August 2024
  
  Publication Preprint
- Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  August 2024
  
  Publication Preprint PDF
- LLM4Eval: Large Language Model for Evaluation in IR
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  Workshop description
  
  Publication PDF
- What Matters in a Measure? A Perspective from Large-Scale Search Evaluation
  
  Paul Thomas, Gabriella Kazai, Nick Craswell, Seth Spielman
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  Best paper nomination
  
  DOI PDF
- Large Language Models Can Accurately Predict Searcher Preferences
  
  Paul Thomas, Seth Spielman, Nick Craswell, Bhaskar Mitra
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  An earlier version of this paper appeared as arXiv preprint arXiv:2309.10621v1 [cs.IR].
  
  Publication PDF PDF
- Enhancing human annotation: Leveraging large language models and efficient batch processing
  
  Oleg Zendel, J Shane Culpepper, Falk Scholer, Paul Thomas
  
  Conference on Human Information Interaction and Retrieval | March 2024
  
  DOI PDF
- Can generative LLMs create query variants for test collections? An exploratory study
  
  Marwah Alaofi, Luke Gallagher, Mark Sanderson, Falk Scholer, Paul Thomas
  
  2023 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2023
  
  PDF DOI
- Taking Search to Task
  
  Chirag Shah, Ryen W. White, Paul Thomas, Bhaskar Mitra, Shawon Sarkar, Nicholas Belkin
  
  Proceedings of the 8th ACM SIGIR Conference On Human Information Interaction And Retrieval | January 2023
  
  Publication Preprint PDF
- A Multi-objective Optimization Framework for Multi-stakeholder Fairness-aware Recommendation
  
  Haolun Wu, Chen Ma, Bhaskar Mitra, Fernando Diaz, Xue Liu
  
  ACM Transactions on Information Systems (TOIS) | August 2022
  
  Publication Preprint PDF
- A flexible framework for offline effectiveness metrics
  
  Alistair Moffat, Joel Mackenzie, Paul Thomas, Leif Azzopardi
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2022
  
  Publication DOI
- Are We There Yet? A Decision Framework for Replacing Term-Based Retrieval with Dense Retrieval Systems
  
  Sebastian Hofstätter, Nick Craswell, Bhaskar Mitra, Hamed Zamani, Allan Hanbury
  
  June 2022
  
  Preprint
- Overview of the TREC 2021 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Text REtrieval Conference (TREC) | May 2022
  
  Publication PDF PDF Access
- Joint Multisided Exposure Fairness for Recommendation
  
  Bhaskar Mitra, Haolun Wu, Chen Ma, Fernando Diaz, Xue Liu
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF
- Inconsistent Ranking Assumptions in Medical Search and Their Downstream Consequences
  
  Daniel Cohen, Kevin Du, Bhaskar Mitra, Laura Mercurio, Navid Rekabsaz, Carsten Eickhoff
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF
- Fostering Coopetition While Plugging Leaks: The Design and Implementation of the MS MARCO Leaderboards
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF Access
- Search interfaces for biomedical searching: How do gaze, user perception, search behaviour and search performance relate?
  
  Ying-Hsang Liu, Paul Thomas, Tom Gedeon, Nicolay Rusnachenko
  
  ACM SIGIR Conference on Human Information Interaction and Retrieval | March 2022
  
  DOI
- The crowd is made of people: Observations from large-scale crowd labelling
  
  Paul Thomas, Gabriella Kazai, Ryen W. White, Nick Craswell
  
  ACM SIGIR Conference on Human Information Interaction and Retrieval | March 2022
  
  Publication DOI
- Report on the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021)
  
  Chirag Shah, Torsten Suel, Fernando Diaz, Bhaskar Mitra, Bárbara Poblete, Hussein Suleman, Suzan Verberne
  
  SIGIR Forum | February 2022, Vol 55(2)
  
  Publication PDF
- Bias-aware Fair Neural Ranking for Addressing Stereotypical Gender Biases
  
  Shirin SeyedSalehi, Amin Bigdeli, Negar Arabzadeh, Bhaskar Mitra, Morteza Zihayat, Ebrahim Bagheri
  
  Proceedings of the 25th International Conference on Extending Database Technology (EDBT) | February 2022
  
  Publication PDF PDF
- Exposing Query Identification for Search Transparency
  
  Ruohan Li, Jianxiang Li, Bhaskar Mitra, Fernando Diaz, Asia Biega
  
  2022 The Web Conference | January 2022
  
  Publication Preprint PDF PDF
- Less is Less: When Are Snippets Insufficient for Human vs Machine Relevance Estimation?
  
  Gabriella Kazai, Bhaskar Mitra, Anlei Dong, Nick Craswell, Linjun Yang
  
  European Conference on Information Retrieval | November 2021
  
  Publication Preprint PDF PDF
- Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness
  
  Nicola Neophytou, Bhaskar Mitra, Catherine Stinson
  
  European Conference on Information Retrieval | November 2021
  
  Publication Preprint PDF PDF
- Analysing mixed initiatives and search strategies during conversational search
  
  Mohammad Aliannejadi, Leif Azzopardi, Hamed Zamani, Evangelos Kanoulas, Paul Thomas, Nick Craswell
  
  ACM International Conference on Information and Knowledge Management | November 2021
  
  DOI
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  ACM Transactions on Information Systems | August 2021, Vol 39(4)
  
  Article 39
  
  Publication
- MS MARCO Chameleons: Challenging the MS MARCO Leaderboard with Extremely Obstinate Queries
  
  Negar Arabzadeh, Bhaskar Mitra, Ebrahim Bagheri
  
  Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM) | August 2021
  
  Publication PDF PDF Access
- Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval (Dissertation Abstract)
  
  Bhaskar Mitra
  
  SIGIR Forum | July 2021, Vol 55(1)
  
  Publication PDF PDF
- Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
  
  Sebastian Hofstatter, Bhaskar Mitra, Hamed Zamani, Nick Craswell, Allan Hanbury
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
  
  Publication Preprint PDF PDF
- Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Models
  
  Daniel Cohen, Bhaskar Mitra, Oleg Lesota, Navid Rekabsaz, Carsten Eickhoff
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
  
  Publication Preprint PDF
- TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees, Ian Soboroff
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF
- MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- Proceedings of the 2021 Conference on Human Information Interaction and Retrieval
  
  Falk Scholer, Paul Thomas, David Elsweiler, Hideo Joho, Noriko Kando, Catherine Smith
  
  March 2021
  
  Publication
- Social competence in conversational retrieval
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  Future Conversations | March 2021
  
  Publication
- Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | March 2021
  
  Publication Preprint PDF Access
- Overview of the TREC 2020 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos
  
  Text REtrieval Conference (TREC) | February 2021
  
  Publication Preprint PDF PDF Access
- Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification
  
  Jaime Arguello, Adam Ferguson, Emery Fine, Bhaskar Mitra, Hamed Zamani, Fernando Diaz
  
  Proceedings of the 6th international ACM SIGIR Conference on Human Information Interaction and Retrieval | January 2021
  
  Publication Preprint PDF PDF Github Access
- Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval
  
  Bhaskar Mitra
  
  December 2020
  
  PhD Thesis: University College London | December 2020
  
  Publication Preprint PDF PDF
- Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Text REtrieval Conference (TREC) | November 2020
  
  Publication Preprint PDF PDF Access
- Simulating information retrieval test collections
  
  David Hawking, Bodo von Billerbeck, Paul Thomas, Nick Craswell
  
  Published by Morgan & Claypool | September 2020
  
  Editor(s): Gary Marchionini
  
  ISBN: 9781681739571
  
  Synthesis Lectures on Information Concepts, Retrieval, and Services
  
  Publication DOI
- Semantic Product Search for Matching Structured Product Catalogs in E-Commerce
  
  Jason Ingyu Choi, Surya Kallumadi, Bhaskar Mitra, Eugene Agichtein, Faizan Javed
  
  August 2020
  
  Preprint
- Expressions of style in information-seeking conversation with an embodied agent
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2020
  
  PDF DOI
- Conformer-Kernel with Query Term Independence for Document Retrieval
  
  Bhaskar Mitra, Sebastian Hofstätter, Hamed Zamani, Nick Craswell
  
  July 2020
  
  Preprint Github
- Evaluating Stochastic Rankings with Expected Exposure
  
  Fernando Diaz, Bhaskar Mitra, Michael D. Ekstrand, Asia J. Biega, Ben Carterette
  
  Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | July 2020
  
  Best Long Research Paper Nominee
  
  Publication Preprint PDF
- MIMICS: A Large-Scale Data Collection for Search Clarification
  
  Hamed Zamani, Gord Lueck, Everest Chen, Rodolfo Quispe, Flint Luu, Nick Craswell
  
  June 2020
- ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search
  
  Nick Craswell, Daniel Campos, Bhaskar Mitra, Emine Yilmaz, Bodo von Billerbeck
  
  Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | June 2020
  
  Publication Preprint PDF Access
- Analyzing and Learning from User Interactions for Search Clarification
  
  Hamed Zamani, Bhaskar Mitra, Everest Chen, Gord Lueck, Fernando Diaz, Paul Bennett, Nick Craswell, Susan Dumais
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF PDF
- Local Self-Attention over Long Text for Efficient Document Retrieval
  
  Sebastian Hofstätter, Hamed Zamani, Bhaskar Mitra, Nick Craswell, Allan Hanbury
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF
- On the Reliability of Test Collections for Evaluating Systems of Different Types
  
  Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Daniel Campos
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | April 2020
  
  Publication Preprint PDF PDF Access
- Overview of the TREC 2019 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees
  
  Text REtrieval Conference (TREC) | March 2020
  
  Publication PDF PDF Access
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  International Workshop on Conversational Approaches to Information Retrieval | March 2020
  
  An extended version of this paper also appears as Thomas et al., ACM Trans. Info. Sys. 39(2), 2021: https://dl.acm.org/doi/pdf/10.1145/3439869.
  
  Publication
- Data-driven evaluation metrics for heterogeneous search engine result pages
  
  Leif Azzopardi, Ryen W. White, Paul Thomas, Nick Craswell
  
  Proceedings of the Conference on Human Information Interaction and Retrieval | March 2020
  
  DOI Publication PDF
- Towards a model for spoken conversational search
  
  Johanne R Trippas, Damiano Spina, Paul Thomas, Mark Sanderson, Hideo Joho, Lawrence Cavedon
  
  Information Processing and Management | March 2020, Vol 57(2)
- Duet at TREC 2019 Deep Learning Track
  
  Bhaskar Mitra, Nick Craswell
  
  Text REtrieval Conference (TREC) | December 2019
  
  Publication Preprint PDF PDF Access
- Investigating searchers’ mental models to inform search explanations
  
  Paul Thomas, Bodo von Billerbeck, Nick Craswell, Ryen W. White
  
  ACM Transactions on Information Systems | December 2019, Vol 38(1)
  
  Article 10
  
  DOI Publication
- Report on the First HIPstIR Workshop on the Future of Information Retrieval
  
  Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
  
  SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
  
  Publication PDF
- cwl_eval: An evaluation tool for information retrieval
  
  Leif Azzopardi, Paul Thomas, Alistair Moffat
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2019
  
  Publication
- The Emotion Profile of Web Search
  
  Gabriella Kazai, Paul Thomas, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2019
  
  Publication
- Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
  
  Bhaskar Mitra, Corby Rosset, David Hawking, Nick Craswell, Fernando Diaz, Emine Yilmaz
  
  July 2019
  
  Preprint Project Project
- Building economic models and measures of search (tutorial)
  
  Leif Azzopardi, Alistair Moffat, Paul Thomas, Guido Zuccon
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | June 2019
  
  Publication
- An Axiomatic Approach to Regularizing Neural Ranking Models
  
  Corby Rosset, Bhaskar Mitra, Chenyan Xiong, Nick Craswell, Xia Song, Saurabh Tiwary
  
  Proceedings of the 42nd international ACM SIGIR conference on Research & development in information retrieval | April 2019
  
  Publication Preprint PDF PDF Project Project
- An updated duet model for passage re-ranking
  
  Bhaskar Mitra, Nick Craswell
  
  March 2019
  
  Preprint
- Data sets for spoken conversational search
  
  Johanne Trippas, Paul Thomas
  
  Workshop on Barriers to Interactive IR Resources Re-use at the ACM SIGIR Conference on Human Information Interaction and Retrieval | March 2019
  
  Publication
- An Introduction to Neural Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Foundations and Trends® in Information Retrieval | December 2018, Vol 13(1): pp. 1-126
  
  Publication PDF PDF
- Better effectiveness metrics for SERPs, cards, and rankings
  
  Paul Thomas, Alistair Moffat, Peter Bailey, Falk Scholer, Nick Craswell
  
  Australasian Document Computing Symposium | December 2018
  
  Best paper award
  
  DOI Publication
- A Line in the Sand: Recommendation or Ad-hoc Retrieval?
  
  Surya Kallumadi, Bhaskar Mitra, Tereza Iofciu
  
  Proceedings of the Recommender Systems Challenge 2018 | July 2018
  
  Preprint PDF
- Measuring the Utility of Search Engine Result Pages: An Information Foraging Based Measure
  
  Leif Azzopardi, Paul Thomas, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2018
  
  Publication Publication Publication
- Cross Domain Regularization for Neural Ranking Models Using Adversarial Learning
  
  Daniel Cohen, Bhaskar Mitra, Katja Hofmann, W. Bruce Croft
  
  Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
  
  Best Short Research Paper
  
  Publication Preprint PDF
- Optimizing Query Evaluations Using Reinforcement Learning for Web Search
  
  Corby Rosset, Damien Jose, Gargi Ghosh, Bhaskar Mitra, Saurabh Tiwary
  
  Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
  
  Publication Preprint PDF Project Project
- Neural Networks for Information Retrieval (ECIR 2018)
  
  Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 40th European Conference on IR Research (ECIR) | March 2018
  
  Advances in Information Retrieval
  
  PDF PDF
- Style and alignment in information-seeking conversation
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell, Gloria Mark
  
  Proceedings of the Conference on Computer-Human Information Interaction and Retrieval | March 2018
  
  Publication Publication
- Neural Networks for Information Retrieval (WSDM 2018)
  
  Tom Kenter, Alexey Borisov, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining | January 2018
  
  Preprint
- Tasks, queries, and rankers in pre-retrieval performance prediction
  
  Paul Thomas, Falk Scholer, Peter Bailey, Alistair Moffat
  
  Proceedings of the Australasian Document Computing Symposium | December 2017
  
  Publication
- Neural Ranking Models with Multiple Document Fields
  
  Hamed Zamani, Bhaskar Mitra, Xia Song, Nick Craswell, Saurabh Tiwary
  
  Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM'18) | November 2017
  
  Publication Preprint PDF Project Project
- Multimodal analysis of vocal collaborative search: A public corpus and results
  
  Daniel McDuff, Paul Thomas, Mary Czerwinski, Nick Craswell
  
  Proceedings of the ACM International Conference on Multimodal Interaction | November 2017
  
  Publication Publication
- Neural information retrieval: introduction to the special issue
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Information Retrieval Journal : Special issue Neural | November 2017
  
  Publication PDF
- Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rĳke, Jiafeng Guo, Bhaskar Mitra
  
  SIGIR Forum | October 2017, Vol 52
  
  Publication PDF PDF
- Reply With: Proactive Recommendation of Email Attachments
  
  Christophe Van Gysel, Bhaskar Mitra, Matteo Venanzi, Roy Rosemarin, Grzegorz Kukla, Piotr Grudzien, Nicola Cancedda
  
  Proceedings of the 26th ACM International Conference on Information and Knowledge Management (CIKM) | October 2017
  
  Publication Preprint PDF
- Benchmark for Complex Answer Retrieval
  
  Federico Nanni, Bhaskar Mitra, Matt Magnusson, Laura Dietz
  
  Proceedings of 3rd ACM International Conference on the Theory of Information Retrieval | October 2017
  
  Publication Preprint PDF
- SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval | August 2017
  
  Publication PDF
- MISC: A data set of information-seeking conversations
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the 1st International Workshop on Conversational Approaches to Information Retrieval | August 2017
  
  Publication
- Natural search user interfaces for complex biomedical search: An eye tracking study
  
  Ying-Hsang Liu, Paul Thomas, Marijana Bacic, Tom Gedeon, Xindi Li
  
  Journal of the Australian Library and Information Association | August 2017, Vol 66(4): pp. 364-381
  
  Publication Publication
- Retrieval consistency in the presence of query variations
  
  Peter Bailey, Alistair Moffat, Falk Scholer, Paul Thomas
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | August 2017
  
  Publication Publication
- Luandri: a Clean Lua Interface to the Indri Search Engine
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 40th international ACM SIGIR conference on Research & development in information retrieval | August 2017
  
  Publication Preprint PDF PDF
- Toward Incorporation of Relevant Documents in word2vec
  
  Navid Rekabsaz, Bhaskar Mitra, Mihai Lupu, Allan Hanbury
  
  Proceedings of the SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR'17) | July 2017
  
  Preprint
- Neural Networks for Information Retrieval (SIGIR 2017)
  
  Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2017
  
  Publication Preprint PDF
- Incorporating user expectations and behavior into the measurement of search effectiveness
  
  Alistair Moffat, Peter Bailey, Falk Scholer, Paul Thomas
  
  ACM Transactions on Information Systems | June 2017, Vol 35(3)
  
  Publication Publication
- Neural Models for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  May 2017
  
  Preprint Video
- Learning to Match Using Local and Distributed Representations of Text for Web Search
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 26th International Conference on World Wide Web | April 2017
  
  Publication Preprint PDF PDF
- What snippet size is needed in mobile web search?
  
  Jaewon Kim, Paul Thomas, Ramesh Sankaranarayana, Tom Gedeon
  
  Proceedings of the Conference on Computer-Human Information Interaction and Retrieval | March 2017
  
  Publication Publication
- A Theoretical Framework for Conversational Search
  
  Filip Radlinski, Nick Craswell
  
  CHIIR 2017 | March 2017
  
  Publication
- Neural Text Embeddings for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM'17) | February 2017
  
  Publication PDF
- Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rĳke, Bhaskar Mitra
  
  SIGIR Forum | December 2016, Vol 50(2): pp. 96-103
  
  Publication PDF
- Query-biased summaries for tabular data
  
  Vincent Au, Paul Thomas, Gaya K Jayasinghe
  
  Proceedings of the Australasian Document Computing Symposium | December 2016
  
  Publication Publication Publication
- Understanding eye movements on mobile devices for better presentation of search results
  
  Jaewon Kim, Paul Thomas, Ramesh Sankaranarayana, Tom Gedeon, Hwan-Jin Yoon
  
  Journal of the Association for Information Science and Technology | November 2016, Vol 67(11): pp. 2607-2619
  
  Publication Publication
- Pagination versus scrolling in mobile web search
  
  Jaewon Kim, Paul Thomas, Ramesh Sankaranarayana, Tom Gedeon, Hwan-Jin Yoon
  
  Proceedings of the International ACM SIGIR Conference on on Information and Knowledge Management | October 2016
  
  Publication Publication
- UQV100: A Test Collection with Query Variability
  
  Peter Bailey, Alistair Moffat, Falk Scholer, Paul Thomas
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2016
  
  Publication Publication
- Neu-IR: The SIGIR 2016 Workshop on Neural Information Retrieval
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2016
  
  Publication PDF
- Ingrams – a neuropsychological explanation for why people search
  
  Peter Bailey, Nick Craswell
  
  Proceedings of the 39th Annual ACM SIGIR Conference (SIGIR 2016) | July 2016
  
  Publication Publication
- A Proposal for Evaluating Answer Distillation from Web Data
  
  Bhaskar Mitra, Grady Simon, Jianfeng Gao, Nick Craswell, Li Deng
  
  Proceedings of the Second WebQA Workshop (SIGIR 2016) | July 2016
  
  PDF PDF
- Towards Conversational Recommender Systems
  
  Konstantina Christakopoulou, Filip Radlinski, Katja Hofmann
  
  22nd ACM SigKDD Conference on Knowledge Discovery and Data Mining | June 2016
  
  Publication Project
- Query Expansion with Locally-Trained Word Embeddings
  
  Fernando Diaz, Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics | June 2016
  
  Publication Preprint PDF PDF
- Improving Document Ranking with Dual Word Embeddings
  
  Eric Nalisnick, Bhaskar Mitra, Nick Craswell, Rich Caruana
  
  WWW'16 | April 2016
  
  Publication PDF PDF Project
- Where Can I Buy a Boulder? Searching for Offline Retail Locations
  
  Sandro Bauer, Filip Radlinski, Ryen W. White
  
  Proceedings of the 25th International World Wide Web Conference (WWW) (to appear) | April 2016
  
  Publication
- Did You Say U2 or Youtube? Inferring Implicit Transcripts from Voice Search Logs
  
  Nick Craswell, Umut Ozertem, Milad Shokouhi
  
  Proceedings of the 25th International World Wide Web Conference (WWW2016) | April 2016
  
  Publication
- A Dual Embedding Space Model for Document Ranking
  
  Bhaskar Mitra, Eric Nalisnick, Nick Craswell, Rich Caruana
  
  February 2016
  
  This paper is an extended evaluation and analysis of the model proposed in a poster to appear in WWW'16, April 11 - 15, 2016, Montreal, Canada.
  
  Preprint PDF Access Project
- Query Auto-Completion for Rare Prefixes
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM) | October 2015
  
  Publication PDF PDF
- Exploring Session Context using Distributed Representations of Queries and Reformulations
  
  Bhaskar Mitra
  
  Proceedings of the 38th international ACM SIGIR conference on Research & development in information retrieval | August 2015
  
  Publication PDF PDF
- Large Scale Log Analysis of Individuals’ Domain Preferences in Web Search
  
  Sarah K. Tyler, Jaime Teevan, Peter Bailey, Sebastian de la Chica, Nikhil Dandekar
  
  MSR-TR-2015-048 | June 2015
  
  Publication
- An Eye-tracking Study of User Interactions with Query Auto Completion
  
  Katja Hofmann, Bhaskar Mitra, Filip Radlinski, Milad Shokouhi
  
  Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
  
  Publication PDF Video
- Relevance and Effort: An Analysis of Document Utility
  
  Emine Yilmaz, Manisha Verma, Nick Craswell, Filip Radlinski, Peter Bailey
  
  Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
  
  Publication
- On User Interactions with Query Auto-Completion
  
  Bhaskar Mitra, Milad Shokouhi, Filip Radlinski, Katja Hofmann
  
  Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR) | July 2014
  
  Publication PDF
- Beyond Clicks: Query Reformulation as a Predictor of Search Satisfaction
  
  Ahmed Awadallah, Xiaolin Shi, Nick Craswell, Bill Ramsey
  
  ACM International Conference on Information and Knowledge Management (CIKM) | October 2013
  
  Publication
- Optimized Interleaving for Online Retrieval Evaluation
  
  Filip Radlinski, Nick Craswell
  
  ACM International Conference on Web Search and Data Mining (WSDM) | February 2013
  
  ACM International Conference on Web Search and Data Mining (WSDM)
  
  Best Paper Award
  
  Publication
- Robust Query Rewriting using Anchor Data
  
  Nick Craswell, Bodo von Billerbeck, Dennis Fetterly, Marc Najork
  
  6th ACM International Conference on Web Search and Data Mining (WSDM) | February 2013
  
  Publication
- Modeling the Impact of Short- and Long-Term Behavior on Search Personalization
  
  Paul Bennett, Ryen W. White, Wei Chu, Susan Dumais, Peter Bailey, Fedor Borisyuk, Xiaoyuan Cui
  
  Proceedings of the 35th Annual ACM SIGIR Conference (SIGIR 2012) | August 2012
  
  Publication
- Probabilistic Models for Personalizing Web Search
  
  David Sontag, Kevyn Collins-Thompson, Paul Bennett, Ryen W. White, Susan Dumais, Bodo von Billerbeck
  
  WSDM 2012: Fifth International Conference on Web Search and Data Miniing | February 2012
  
  Publication
- Large Scale Validation and Analysis of Interleaved Search Evaluation
  
  Olivier Chapelle, Thorsten Joachims, Filip Radlinski, Yisong Yue
  
  Transactions on Information Systems (TOIS) | February 2012, Vol 30
  
  Selected for ACM Notable Computing Books and Articles of 2012
  
  Publication
- Microsoft Research at TREC 2011 Web Track
  
  Bodo von Billerbeck, Nick Craswell, Dennis Fetterly, Marc Najork
  
  Proc. of the 20th Text Retrieval Conference (TREC) | November 2011
  
  Publication
- Inferring and Using Location Metadata to Personalize Web Search
  
  Paul Bennett, Filip Radlinski, Ryen W. White, Emine Yilmaz
  
  Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR) | July 2011
  
  Publication
- The Power of Peers
  
  Nick Craswell, Dennis Fetterly, Marc Najork
  
  33rd European Conference on IR Research (ECIR) | April 2011
  
  33rd European Conference on IR Research (ECIR)
  
  Publication
- Personalizing Web Search using Long Term Browsing History
  
  Nicolaas Matthijs, Filip Radlinski
  
  ACM International Conference on Web Search and Data Mining (WSDM) | February 2011
  
  Publication
- LambdaMerge: Merging the Results of Query Reformulations
  
  Daniel Sheldon, Milad Shokouhi, Martin Szummer, Nick Craswell
  
  Conf. Web search and data mining (WSDM) | February 2011
  
  Publication Publication
- Microsoft Research at TREC 2010 Web Track
  
  Nick Craswell, Dennis Fetterly, Marc Najork
  
  Proc. of the 19th Text Retrieval Conference (TREC) | November 2010
  
  Proc. of the 19th Text Retrieval Conference (TREC)
  
  Publication
- Metrics for Assessing Sets of Subtopics
  
  Filip Radlinski, Martin Szummer, Nick Craswell
  
  SIGIR Conf. Research and Development in Information Retrieval | July 2010
  
  Publication
- Comparing the Sensitivity of Information Retrieval Metrics
  
  Filip Radlinski, Nick Craswell
  
  Proceedings of SIGIR | July 2010
  
  Publication
- Inferring Query Intent from Reformulations and Clicks
  
  Filip Radlinski, Martin Szummer, Nick Craswell
  
  Proc. 19th Annual International World Wide Web Conference (WWW '10). | April 2010
  
  Proc. 19th Annual International World Wide Web Conference (WWW '10).
  
  Publication
- Microsoft Research at TREC 2009: Web and Relevance Feedback Tracks
  
  Nick Craswell, Dennis Fetterly, Marc Najork, Stephen Robertson, Emine Yilmaz
  
  Proc. of the 18th Text Retrieval Conference (TREC) | November 2009
  
  Publication
- Measuring the Search Effectiveness of a Breadth-First Crawl
  
  Dennis Fetterly, Nick Craswell, Vishwa Vinay
  
  Proceedings of the 31st European Conference on Information Retrieval (ECIR) | April 2009
  
  Proceedings of the 31st European Conference on Information Retrieval (ECIR)
  
  Publication
- Efficient and Effective Link Analysis with Precomputed SALSA Maps
  
  Marc Najork, Nick Craswell
  
  17th ACM Conference on Information and Knowledge Management (CIKM) | October 2008
  
  17th ACM Conference on Information and Knowledge Management (CIKM)
  
  Publication
- A Decision Theoretic Framework for Ranking using Implicit Feedback
  
  Onno Zoeter, Mike Taylor, Ed Snelson, John Guiver, Nick Craswell, Martin Szummer
  
  SIGIR 2008 Workshop on Learning to Rank for Information Retrieval | July 2008
  
  Publication
- Relevance assessment: are judges exchangeable and does it matter
  
  Peter Bailey, Nick Craswell, Ian Soboroff, Paul Thomas, Arjen P. de Vries, Emine Yilmaz
  
  SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval | January 2008
  
  ISBN: 978-1-60558-164-4
  
  Publication Publication
- Overview of the INEX 2007 Entity Ranking Track
  
  Arjen P. de Vries, Anne-Marie Vercoustre, James A. Thom, Nick Craswell, Mounia Lalmas
  
  January 2008
  
  ISBN: 978-3-540-85901-7
  
  Publication Publication
- An experimental comparison of click position-bias models
  
  Nick Craswell, Mike Taylor
  
  WSDM '08: Proceedings of the international conference on Web search and web data mining | January 2008
  
  ISBN: 978-1-59593-927-9
  
  Publication Publication
- Relevance Weighting for Query Independent Evidence
  
  Nick Craswell, Stephen Robertson, Hugo Zaragoza, Mike Taylor
  
  Proceedings of ACM SIGIR 2005 | January 2005
  
  ISBN: 1-59593-034-5
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_sigir05.pdf
  
  Publication
- RMIT University at TREC 2004
  
  Bodo von Billerbeck, Adam Cannane, Abhijit Chattaraj, Nicholas Lester, William Webber, Hugh E. Williams, John Yiannis, Justin Zobel
  
  Proceedings Text Retrieval Conference (TREC) | November 2004
  
  Publication
- Predicting Fame and Fortune: PageRank or Indegree?
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  Proceedings of the Australasian Document Computing Symposium, ADCS2003 | December 2003
  
  Proceedings of the Australasian Document Computing Symposium, ADCS2003
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_adcs03.pdf
  
  Publication
- Automated Discovery of Search Interfaces on the Web
  
  Jared Cope, Nick Craswell, David Hawking
  
  The Fourteenth Australasian Database Conference | January 2003
  
  The Fourteenth Australasian Database Conference
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/cope_adc03.pdf
  
  Publication
- Query expansion using associated queries
  
  Bodo von Billerbeck
  
  Proceedings of the twelfth international conference on information and knowledge management | January 2003
  
  ISBN: 1-58113-723-0
  
  Publication
- Overview of the TREC-2002 Web Track
  
  Nick Craswell, David Hawking
  
  Proceedings of TREC-2002 | November 2002
  
  Proceedings of TREC-2002
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec02wt.pdf
  
  Publication
- Buying bestsellers online: A case study in Search & Searchability
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  7th Australasian Document Computing Symposium | January 2002
  
  7th Australasian Document Computing Symposium
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_adcs02.pdf
  
  Publication
- Search interfaces for biomedical searching: How do gaze, user perception, search behaviour and search performance relate?
  
  Ying-Hsang Liu, Paul Thomas, Tom Gedeon, Nicolay Rusnachenko
  
  ACM SIGIR Conference on Human Information Interaction and Retrieval | March 2022
  
  DOI
- The crowd is made of people: Observations from large-scale crowd labelling
  
  Paul Thomas, Gabriella Kazai, Ryen W. White, Nick Craswell
  
  ACM SIGIR Conference on Human Information Interaction and Retrieval | March 2022
  
  Publication DOI
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  ACM Transactions on Information Systems | August 2021, Vol 39(4)
  
  Article 39
  
  Publication
- Proceedings of the 2021 Conference on Human Information Interaction and Retrieval
  
  Falk Scholer, Paul Thomas, David Elsweiler, Hideo Joho, Noriko Kando, Catherine Smith
  
  March 2021
  
  Publication
- Social competence in conversational retrieval
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  Future Conversations | March 2021
  
  Publication
- Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification
  
  Jaime Arguello, Adam Ferguson, Emery Fine, Bhaskar Mitra, Hamed Zamani, Fernando Diaz
  
  Proceedings of the 6th international ACM SIGIR Conference on Human Information Interaction and Retrieval | January 2021
  
  Publication Preprint PDF PDF Github Access
- Expressions of style in information-seeking conversation with an embodied agent
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2020
  
  PDF DOI
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  International Workshop on Conversational Approaches to Information Retrieval | March 2020
  
  An extended version of this paper also appears as Thomas et al., ACM Trans. Info. Sys. 39(2), 2021: https://dl.acm.org/doi/pdf/10.1145/3439869.
  
  Publication
- Style and alignment in information-seeking conversation
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell, Gloria Mark
  
  Proceedings of the Conference on Computer-Human Information Interaction and Retrieval | March 2018
  
  Publication Publication
- Multimodal analysis of vocal collaborative search: A public corpus and results
  
  Daniel McDuff, Paul Thomas, Mary Czerwinski, Nick Craswell
  
  Proceedings of the ACM International Conference on Multimodal Interaction | November 2017
  
  Publication Publication
- MISC: A data set of information-seeking conversations
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the 1st International Workshop on Conversational Approaches to Information Retrieval | August 2017
  
  Publication
- Query Auto-Completion for Rare Prefixes
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM) | October 2015
  
  Publication PDF PDF
- Large Scale Log Analysis of Individuals’ Domain Preferences in Web Search
  
  Sarah K. Tyler, Jaime Teevan, Peter Bailey, Sebastian de la Chica, Nikhil Dandekar
  
  MSR-TR-2015-048 | June 2015
  
  Publication
- An Eye-tracking Study of User Interactions with Query Auto Completion
  
  Katja Hofmann, Bhaskar Mitra, Filip Radlinski, Milad Shokouhi
  
  Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
  
  Publication PDF Video
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  International Workshop on Conversational Approaches to Information Retrieval | March 2020
  
  An extended version of this paper also appears as Thomas et al., ACM Trans. Info. Sys. 39(2), 2021: https://dl.acm.org/doi/pdf/10.1145/3439869.
  
  Publication
- A Two-Stage Model for Expert Search
  
  Yunbo Cao, JJ (Jingjing) Liu, Shenghua Bao, Hang Li, Nick Craswell
  
  MSR-TR-2008-195 | October 2008
  
  Publication
- Methods for Distributed Information Retrieval
  
  Nick Craswell
  
  January 2000
  
  January 2000
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_thesis00.pdf
  
  Publication
- Aglets: A good idea for spidering?
  
  Jason Haines, Brendan Humphreys, Chris Johnson Nick Craswell, Paul Thistlewaite, Nick Craswell
  
  4th IDEA Workshop Proceedings | January 1997
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_idea97.pdf
  
  Publication
- Large Scale Log Analysis of Individuals’ Domain Preferences in Web Search
  
  Sarah K. Tyler, Jaime Teevan, Peter Bailey, Sebastian de la Chica, Nikhil Dandekar
  
  MSR-TR-2015-048 | June 2015
  
  Publication
- A Two-Stage Model for Expert Search
  
  Yunbo Cao, Jingjing Liu, Shenghua Bao, Hang Li, Nick Craswell
  
  MSR-TR-2008-143 | October 2008
  
  Publication
- Testbed for Information Extraction from Deep Web
  
  Yasuhiro Yamada, Nick Craswell, Tetsuya Nakatoh, Sachio Hirokawa
  
  WWW2004 Poster Proceedings | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/yamada_www2004poster.pdf
  
  Publication
- Epigenome-wide association studies without the need for cell-type composition
  
  James Zhou, Christoph Lippert, David Heckerman, Martin Aryee, Jennifer Listgarten
  
  Nature Methods | August 2013
  
  Publication Publication
- The Impact of Crawl Policy on Web Search Effectiveness
  
  Dennis Fetterly, Nick Craswell, Vishwa Vinay
  
  Proceedings of the 32nd annual international ACM SIGIR conference on Research and development in information retrieval | July 2009
  
  Publication
- The TREC terabyte retrieval track
  
  Charles Clarke, Nick Craswell, Ian Soboroff
  
  SIGIR Forum | January 2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/clarke_sigirforum05.pdf
  
  Publication
- Toward Better Weighting of Anchors
  
  David Hawking, Trystan Upstill, Nick Craswell
  
  Proceedings of ACM SIGIR 2004 (poster abstract) | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_sigir04poster.pdf
  
  Publication
- When query expansion fails
  
  Bodo von Billerbeck
  
  Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval | July 2003
  
  ISBN: 1-58113-646-3
  
  Publication
- TREC10 Web and Interactive Tracks at CSIRO
  
  Nick Craswell, David Hawking, Ross Wilkinson, Mingfang Wu
  
  Proceedings of TREC-2001 | November 2001
  
  Proceedings of TREC-2001
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec01.pdf
  
  Publication
- Aglets: A good idea for spidering?
  
  Jason Haines, Brendan Humphreys, Chris Johnson Nick Craswell, Paul Thistlewaite, Nick Craswell
  
  4th IDEA Workshop Proceedings | January 1997
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_idea97.pdf
  
  Publication
- Behavioral Classification on the Click Graph
  
  Martin Szummer, Nick Craswell
  
  World Wide Web Conference | April 2008
  
  World Wide Web Conference
  
  Publication
- Optimisation methods for ranking functions with multiple parameters
  
  Mike Taylor, Hugo Zaragoza, Nick Craswell, Stephen Robertson, Chris J.C. Burges
  
  Proceedings of CIKM 2006 | January 2006
  
  Proceedings of CIKM 2006
  
  ISBN: 1-59593-433-2
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/taylor_cikm06.pdf
  
  Publication
- ACSys TREC-7 experiments
  
  David Hawking, Nick Craswell, Paul Thistlewaite
  
  TREC-7, NIST special publication 500-242 | January 1998
  
  TREC-7, NIST special publication 500-242
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_trec98.pdf
  
  Publication
- Focused crawling for both topical relevance and quality of medical information
  
  Thanh Tang, David Hawking, Nick Craswell, Kathleen Griffiths
  
  Proceedings of CIKM 2005 | January 2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/tang_cikm05.pdf
  
  Publication
- Microsoft Cambridge at TREC-14: Enterprise Track
  
  Nick Craswell, Hugo Zaragoza, Stephen Robertson
  
  Proceedings of TREC-2005 | January 2005
  
  Proceedings of TREC-2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec05.pdf
  
  Publication
- Microsoft Cambridge at TREC 2004: Web and HARD track
  
  Hugo Zaragoza, Nick Craswell, Mike Taylor, Suchi Saria, Stephen Robertson
  
  Proceedings of TREC-2004 | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/zaragoza_trec04.pdf
  
  Publication Publication

- All claims are equal, but some claims are more equal than others: Importance-sensitive factuality evaluation of LLM generations
  
  Miriam Wanner, Leif Azzopardi, Paul Thomas, Soham Dan, Ben Van Durme, Nick Craswell
  
  October 2025
  
  preprint arXiv:2510.07083
  
  Publication
- Proceedings of The First Workshop on Large Language Models for Evaluation in Information Retrieval (LLM4Eval 2024)
  
  Clemencia Siro, Mohammad Aliannejadi, Hossein A. Rahmani, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  September 2024
  
  Publication
- LLMJudge: LLMs for Relevance Judgments
  
  Hossein A Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas, Charles L A Clarke, Mohammad Aliannejadi, Clemencia Siro, Guglielmo Faggioli
  
  August 2024
  
  Publication Preprint
- Proceedings of the 2021 Conference on Human Information Interaction and Retrieval
  
  Falk Scholer, Paul Thomas, David Elsweiler, Hideo Joho, Noriko Kando, Catherine Smith
  
  March 2021
  
  Publication
- Neural Models for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  May 2017
  
  Preprint Video
- A Dual Embedding Space Model for Document Ranking
  
  Bhaskar Mitra, Eric Nalisnick, Nick Craswell, Rich Caruana
  
  February 2016
  
  This paper is an extended evaluation and analysis of the model proposed in a poster to appear in WWW'16, April 11 - 15, 2016, Montreal, Canada.
  
  Preprint PDF Access Project
- Report from the 4th Strategic Workshop on Information Retrieval in Lorne (SWIRL 2025)
  
  Johanne R Trippas, J. Shane Culpepper, Mohammad Aliannejadi, James Allan, Enrique Amigó, Jaime Arguello, Leif Azzopardi, Peter Bailey, Jamie Callan, Rob Capra, Nick Craswell, Bruce Croft, Jeff Dalton, Gianluca Demartini, Laura Dietz, Zhicheng Dou, Carsten Eickhoff, Michael Ekstrand, Nicola Ferro, Norbert Fuhr, Dorota Glowacka, Faegheh Hasibi, Danula Hettiachchi, Rosie Jones, Jaap Kamps, Noriko Kando, Sarvnaz Karimi, Makoto P. Kato, Bevan Koopman, Yiqun Liu, Chenglong Ma, Joel Mackenzie, Maria Maistro, Jiaxin Mao, Dana McKay, Bhaskar Mitra, Stefano Mizzaro, Alistair Moffat, Josiane Mothe, Iadh Ounis, Lida Rashidi, Yongli Ren, Mark Sanderson, Rodrygo Santos, Falk Scholer, Chirag A Shah, Laurianne Sitbon, Ian Soboroff, Damiano Spina, Paul Thomas, Julián Urbano, Arjen P. de Vries, Ryen W. White, Abby Yuan, Hamed Zamani, Oleg Zendel, Min Zhang, Shengyao Zhuang, Justin Zobel, Guido Zuccon
  
  ACM SIGIR Forum | June 2025, Vol 59(1): pp. 1-68
  
  DOI PDF
- A Multi-objective Optimization Framework for Multi-stakeholder Fairness-aware Recommendation
  
  Haolun Wu, Chen Ma, Bhaskar Mitra, Fernando Diaz, Xue Liu
  
  ACM Transactions on Information Systems (TOIS) | August 2022
  
  Publication Preprint PDF
- Report on the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021)
  
  Chirag Shah, Torsten Suel, Fernando Diaz, Bhaskar Mitra, Bárbara Poblete, Hussein Suleman, Suzan Verberne
  
  SIGIR Forum | February 2022, Vol 55(2)
  
  Publication PDF
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  ACM Transactions on Information Systems | August 2021, Vol 39(4)
  
  Article 39
  
  Publication
- Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval (Dissertation Abstract)
  
  Bhaskar Mitra
  
  SIGIR Forum | July 2021, Vol 55(1)
  
  Publication PDF PDF
- Towards a model for spoken conversational search
  
  Johanne R Trippas, Damiano Spina, Paul Thomas, Mark Sanderson, Hideo Joho, Lawrence Cavedon
  
  Information Processing and Management | March 2020, Vol 57(2)
- Investigating searchers’ mental models to inform search explanations
  
  Paul Thomas, Bodo von Billerbeck, Nick Craswell, Ryen W. White
  
  ACM Transactions on Information Systems | December 2019, Vol 38(1)
  
  Article 10
  
  DOI Publication
- Report on the First HIPstIR Workshop on the Future of Information Retrieval
  
  Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
  
  SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
  
  Publication PDF
- An Introduction to Neural Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Foundations and Trends® in Information Retrieval | December 2018, Vol 13(1): pp. 1-126
  
  Publication PDF PDF
- Neural information retrieval: introduction to the special issue
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Information Retrieval Journal : Special issue Neural | November 2017
  
  Publication PDF
- Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rĳke, Jiafeng Guo, Bhaskar Mitra
  
  SIGIR Forum | October 2017, Vol 52
  
  Publication PDF PDF
- Natural search user interfaces for complex biomedical search: An eye tracking study
  
  Ying-Hsang Liu, Paul Thomas, Marijana Bacic, Tom Gedeon, Xindi Li
  
  Journal of the Australian Library and Information Association | August 2017, Vol 66(4): pp. 364-381
  
  Publication Publication
- Incorporating user expectations and behavior into the measurement of search effectiveness
  
  Alistair Moffat, Peter Bailey, Falk Scholer, Paul Thomas
  
  ACM Transactions on Information Systems | June 2017, Vol 35(3)
  
  Publication Publication
- Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rĳke, Bhaskar Mitra
  
  SIGIR Forum | December 2016, Vol 50(2): pp. 96-103
  
  Publication PDF
- Understanding eye movements on mobile devices for better presentation of search results
  
  Jaewon Kim, Paul Thomas, Ramesh Sankaranarayana, Tom Gedeon, Hwan-Jin Yoon
  
  Journal of the Association for Information Science and Technology | November 2016, Vol 67(11): pp. 2607-2619
  
  Publication Publication
- Epigenome-wide association studies without the need for cell-type composition
  
  James Zhou, Christoph Lippert, David Heckerman, Martin Aryee, Jennifer Listgarten
  
  Nature Methods | August 2013
  
  Publication Publication
- Large Scale Validation and Analysis of Interleaved Search Evaluation
  
  Olivier Chapelle, Thorsten Joachims, Filip Radlinski, Yisong Yue
  
  Transactions on Information Systems (TOIS) | February 2012, Vol 30
  
  Selected for ACM Notable Computing Books and Articles of 2012
  
  Publication
- The CSIRO Enterprise Search Test Collection
  
  Peter Bailey, Nick Craswell, Ian Soboroff, Arjen P. de Vries
  
  SIGIR Forum | January 2007, Vol 41(2): pp. 42-45
  
  Publication
- Quality and Relevance of Domain-specific Search: A Case Study in Mental Health
  
  Thanh Tang, Nick Craswell, David Hawking, Kathleen Griffiths, Helen Christensen
  
  Information Retrieval | January 2006, pp. 207-225
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/tang_ir06.pdf
  
  Publication
- Which Search Engine is Best at Finding Online Services?
  
  David Hawking, Nick Craswell, Kathleen Griffiths
  
  Poster Proceedings of WWW10 | May 2001
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_www01poster.pdf
  
  Publication
- ACSys TREC-7 experiments
  
  David Hawking, Nick Craswell, Paul Thistlewaite
  
  TREC-7, NIST special publication 500-242 | January 1998
  
  TREC-7, NIST special publication 500-242
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_trec98.pdf
  
  Publication
- Aglets: A good idea for spidering?
  
  Jason Haines, Brendan Humphreys, Chris Johnson Nick Craswell, Paul Thistlewaite, Nick Craswell
  
  4th IDEA Workshop Proceedings | January 1997
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_idea97.pdf
  
  Publication
- Preaching to the ChoIR: Lessons IR should share with AI
  
  Gianluca Demartini, Claudia Hauff, Matthew Lease, Stefano Mizzaro, Kevin Roitero, Mark Sanderson, Falk Scholer, Chirag Shah, Damiano Spina, Paul Thomas, Arjen P de Vries, Guido Zuccon
  
  International Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR) | July 2025
  
  DOI
- Demographically-inspired query variants using an LLM
  
  Marwah Alaofi, Nicola Ferro, Paul Thomas, Falk Scholer, Mark Sanderson
  
  International Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR) | July 2025
  
  DOI Publication
- System comparison using automated generation of relevance judgements in multiple languages
  
  Paul Thomas, Douglas W Oard, Eugene Yang, Dawn Lawrie, James Mayfield
  
  2025 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2025
  
  Publication
- LLM4Eval@WSDM 2025: Large Language Model for Evaluation in Information Retrieval
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L A Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  2025 Web Search and Data Mining | March 2025
  
  Publication
- JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment
  
  Hossein A. Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra
  
  2025 The Web Conference | January 2025
  
  Publication Preprint PDF
- SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval
  
  Hossein A. Rahmani, Xi Wang, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas
  
  2025 The Web Conference | January 2025
  
  Publication PDF Preprint
- LLMs can be Fooled into Labelling a Document as Relevant (Best café near me; this paper is perfectly relevant)
  
  Marwah Alaofi, Paul Thomas, Falk Scholer, Mark Sanderson
  
  International ACM SIGIR Conference on Information Retrieval in the Asia Pacific | December 2024
  
  Best paper nomination
  
  PDF DOI
- LLM4Eval: Large Language Model for Evaluation in IR
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  Workshop description
  
  Publication PDF
- What Matters in a Measure? A Perspective from Large-Scale Search Evaluation
  
  Paul Thomas, Gabriella Kazai, Nick Craswell, Seth Spielman
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  Best paper nomination
  
  DOI PDF
- Large Language Models Can Accurately Predict Searcher Preferences
  
  Paul Thomas, Seth Spielman, Nick Craswell, Bhaskar Mitra
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  An earlier version of this paper appeared as arXiv preprint arXiv:2309.10621v1 [cs.IR].
  
  Publication PDF PDF
- Enhancing human annotation: Leveraging large language models and efficient batch processing
  
  Oleg Zendel, J Shane Culpepper, Falk Scholer, Paul Thomas
  
  Conference on Human Information Interaction and Retrieval | March 2024
  
  DOI PDF
- Can generative LLMs create query variants for test collections? An exploratory study
  
  Marwah Alaofi, Luke Gallagher, Mark Sanderson, Falk Scholer, Paul Thomas
  
  2023 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2023
  
  PDF DOI
- Taking Search to Task
  
  Chirag Shah, Ryen W. White, Paul Thomas, Bhaskar Mitra, Shawon Sarkar, Nicholas Belkin
  
  Proceedings of the 8th ACM SIGIR Conference On Human Information Interaction And Retrieval | January 2023
  
  Publication Preprint PDF
- A flexible framework for offline effectiveness metrics
  
  Alistair Moffat, Joel Mackenzie, Paul Thomas, Leif Azzopardi
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2022
  
  Publication DOI
- Overview of the TREC 2021 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Text REtrieval Conference (TREC) | May 2022
  
  Publication PDF PDF Access
- Joint Multisided Exposure Fairness for Recommendation
  
  Bhaskar Mitra, Haolun Wu, Chen Ma, Fernando Diaz, Xue Liu
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF
- Inconsistent Ranking Assumptions in Medical Search and Their Downstream Consequences
  
  Daniel Cohen, Kevin Du, Bhaskar Mitra, Laura Mercurio, Navid Rekabsaz, Carsten Eickhoff
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF
- Fostering Coopetition While Plugging Leaks: The Design and Implementation of the MS MARCO Leaderboards
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF Access
- Search interfaces for biomedical searching: How do gaze, user perception, search behaviour and search performance relate?
  
  Ying-Hsang Liu, Paul Thomas, Tom Gedeon, Nicolay Rusnachenko
  
  ACM SIGIR Conference on Human Information Interaction and Retrieval | March 2022
  
  DOI
- The crowd is made of people: Observations from large-scale crowd labelling
  
  Paul Thomas, Gabriella Kazai, Ryen W. White, Nick Craswell
  
  ACM SIGIR Conference on Human Information Interaction and Retrieval | March 2022
  
  Publication DOI
- Bias-aware Fair Neural Ranking for Addressing Stereotypical Gender Biases
  
  Shirin SeyedSalehi, Amin Bigdeli, Negar Arabzadeh, Bhaskar Mitra, Morteza Zihayat, Ebrahim Bagheri
  
  Proceedings of the 25th International Conference on Extending Database Technology (EDBT) | February 2022
  
  Publication PDF PDF
- Exposing Query Identification for Search Transparency
  
  Ruohan Li, Jianxiang Li, Bhaskar Mitra, Fernando Diaz, Asia Biega
  
  2022 The Web Conference | January 2022
  
  Publication Preprint PDF PDF
- Less is Less: When Are Snippets Insufficient for Human vs Machine Relevance Estimation?
  
  Gabriella Kazai, Bhaskar Mitra, Anlei Dong, Nick Craswell, Linjun Yang
  
  European Conference on Information Retrieval | November 2021
  
  Publication Preprint PDF PDF
- Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness
  
  Nicola Neophytou, Bhaskar Mitra, Catherine Stinson
  
  European Conference on Information Retrieval | November 2021
  
  Publication Preprint PDF PDF
- Analysing mixed initiatives and search strategies during conversational search
  
  Mohammad Aliannejadi, Leif Azzopardi, Hamed Zamani, Evangelos Kanoulas, Paul Thomas, Nick Craswell
  
  ACM International Conference on Information and Knowledge Management | November 2021
  
  DOI
- MS MARCO Chameleons: Challenging the MS MARCO Leaderboard with Extremely Obstinate Queries
  
  Negar Arabzadeh, Bhaskar Mitra, Ebrahim Bagheri
  
  Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM) | August 2021
  
  Publication PDF PDF Access
- Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
  
  Sebastian Hofstatter, Bhaskar Mitra, Hamed Zamani, Nick Craswell, Allan Hanbury
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
  
  Publication Preprint PDF PDF
- Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Models
  
  Daniel Cohen, Bhaskar Mitra, Oleg Lesota, Navid Rekabsaz, Carsten Eickhoff
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
  
  Publication Preprint PDF
- TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees, Ian Soboroff
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF
- MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- Social competence in conversational retrieval
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  Future Conversations | March 2021
  
  Publication
- Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | March 2021
  
  Publication Preprint PDF Access
- Overview of the TREC 2020 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos
  
  Text REtrieval Conference (TREC) | February 2021
  
  Publication Preprint PDF PDF Access
- Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification
  
  Jaime Arguello, Adam Ferguson, Emery Fine, Bhaskar Mitra, Hamed Zamani, Fernando Diaz
  
  Proceedings of the 6th international ACM SIGIR Conference on Human Information Interaction and Retrieval | January 2021
  
  Publication Preprint PDF PDF Github Access
- Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Text REtrieval Conference (TREC) | November 2020
  
  Publication Preprint PDF PDF Access
- Expressions of style in information-seeking conversation with an embodied agent
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2020
  
  PDF DOI
- Evaluating Stochastic Rankings with Expected Exposure
  
  Fernando Diaz, Bhaskar Mitra, Michael D. Ekstrand, Asia J. Biega, Ben Carterette
  
  Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | July 2020
  
  Best Long Research Paper Nominee
  
  Publication Preprint PDF
- ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search
  
  Nick Craswell, Daniel Campos, Bhaskar Mitra, Emine Yilmaz, Bodo von Billerbeck
  
  Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | June 2020
  
  Publication Preprint PDF Access
- Analyzing and Learning from User Interactions for Search Clarification
  
  Hamed Zamani, Bhaskar Mitra, Everest Chen, Gord Lueck, Fernando Diaz, Paul Bennett, Nick Craswell, Susan Dumais
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF PDF
- Local Self-Attention over Long Text for Efficient Document Retrieval
  
  Sebastian Hofstätter, Hamed Zamani, Bhaskar Mitra, Nick Craswell, Allan Hanbury
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF
- On the Reliability of Test Collections for Evaluating Systems of Different Types
  
  Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Daniel Campos
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | April 2020
  
  Publication Preprint PDF PDF Access
- Overview of the TREC 2019 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees
  
  Text REtrieval Conference (TREC) | March 2020
  
  Publication PDF PDF Access
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  International Workshop on Conversational Approaches to Information Retrieval | March 2020
  
  An extended version of this paper also appears as Thomas et al., ACM Trans. Info. Sys. 39(2), 2021: https://dl.acm.org/doi/pdf/10.1145/3439869.
  
  Publication
- Data-driven evaluation metrics for heterogeneous search engine result pages
  
  Leif Azzopardi, Ryen W. White, Paul Thomas, Nick Craswell
  
  Proceedings of the Conference on Human Information Interaction and Retrieval | March 2020
  
  DOI Publication PDF
- Duet at TREC 2019 Deep Learning Track
  
  Bhaskar Mitra, Nick Craswell
  
  Text REtrieval Conference (TREC) | December 2019
  
  Publication Preprint PDF PDF Access
- cwl_eval: An evaluation tool for information retrieval
  
  Leif Azzopardi, Paul Thomas, Alistair Moffat
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2019
  
  Publication
- The Emotion Profile of Web Search
  
  Gabriella Kazai, Paul Thomas, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2019
  
  Publication
- Building economic models and measures of search (tutorial)
  
  Leif Azzopardi, Alistair Moffat, Paul Thomas, Guido Zuccon
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | June 2019
  
  Publication
- An Axiomatic Approach to Regularizing Neural Ranking Models
  
  Corby Rosset, Bhaskar Mitra, Chenyan Xiong, Nick Craswell, Xia Song, Saurabh Tiwary
  
  Proceedings of the 42nd international ACM SIGIR conference on Research & development in information retrieval | April 2019
  
  Publication Preprint PDF PDF Project Project
- Data sets for spoken conversational search
  
  Johanne Trippas, Paul Thomas
  
  Workshop on Barriers to Interactive IR Resources Re-use at the ACM SIGIR Conference on Human Information Interaction and Retrieval | March 2019
  
  Publication
- Better effectiveness metrics for SERPs, cards, and rankings
  
  Paul Thomas, Alistair Moffat, Peter Bailey, Falk Scholer, Nick Craswell
  
  Australasian Document Computing Symposium | December 2018
  
  Best paper award
  
  DOI Publication
- A Line in the Sand: Recommendation or Ad-hoc Retrieval?
  
  Surya Kallumadi, Bhaskar Mitra, Tereza Iofciu
  
  Proceedings of the Recommender Systems Challenge 2018 | July 2018
  
  Preprint PDF
- Measuring the Utility of Search Engine Result Pages: An Information Foraging Based Measure
  
  Leif Azzopardi, Paul Thomas, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2018
  
  Publication Publication Publication
- Cross Domain Regularization for Neural Ranking Models Using Adversarial Learning
  
  Daniel Cohen, Bhaskar Mitra, Katja Hofmann, W. Bruce Croft
  
  Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
  
  Best Short Research Paper
  
  Publication Preprint PDF
- Optimizing Query Evaluations Using Reinforcement Learning for Web Search
  
  Corby Rosset, Damien Jose, Gargi Ghosh, Bhaskar Mitra, Saurabh Tiwary
  
  Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
  
  Publication Preprint PDF Project Project
- Neural Networks for Information Retrieval (ECIR 2018)
  
  Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 40th European Conference on IR Research (ECIR) | March 2018
  
  Advances in Information Retrieval
  
  PDF PDF
- Style and alignment in information-seeking conversation
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell, Gloria Mark
  
  Proceedings of the Conference on Computer-Human Information Interaction and Retrieval | March 2018
  
  Publication Publication
- Neural Networks for Information Retrieval (WSDM 2018)
  
  Tom Kenter, Alexey Borisov, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining | January 2018
  
  Preprint
- Tasks, queries, and rankers in pre-retrieval performance prediction
  
  Paul Thomas, Falk Scholer, Peter Bailey, Alistair Moffat
  
  Proceedings of the Australasian Document Computing Symposium | December 2017
  
  Publication
- Neural Ranking Models with Multiple Document Fields
  
  Hamed Zamani, Bhaskar Mitra, Xia Song, Nick Craswell, Saurabh Tiwary
  
  Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM'18) | November 2017
  
  Publication Preprint PDF Project Project
- Multimodal analysis of vocal collaborative search: A public corpus and results
  
  Daniel McDuff, Paul Thomas, Mary Czerwinski, Nick Craswell
  
  Proceedings of the ACM International Conference on Multimodal Interaction | November 2017
  
  Publication Publication
- Reply With: Proactive Recommendation of Email Attachments
  
  Christophe Van Gysel, Bhaskar Mitra, Matteo Venanzi, Roy Rosemarin, Grzegorz Kukla, Piotr Grudzien, Nicola Cancedda
  
  Proceedings of the 26th ACM International Conference on Information and Knowledge Management (CIKM) | October 2017
  
  Publication Preprint PDF
- Benchmark for Complex Answer Retrieval
  
  Federico Nanni, Bhaskar Mitra, Matt Magnusson, Laura Dietz
  
  Proceedings of 3rd ACM International Conference on the Theory of Information Retrieval | October 2017
  
  Publication Preprint PDF
- SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval | August 2017
  
  Publication PDF
- MISC: A data set of information-seeking conversations
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the 1st International Workshop on Conversational Approaches to Information Retrieval | August 2017
  
  Publication
- Retrieval consistency in the presence of query variations
  
  Peter Bailey, Alistair Moffat, Falk Scholer, Paul Thomas
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | August 2017
  
  Publication Publication
- Luandri: a Clean Lua Interface to the Indri Search Engine
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 40th international ACM SIGIR conference on Research & development in information retrieval | August 2017
  
  Publication Preprint PDF PDF
- Toward Incorporation of Relevant Documents in word2vec
  
  Navid Rekabsaz, Bhaskar Mitra, Mihai Lupu, Allan Hanbury
  
  Proceedings of the SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR'17) | July 2017
  
  Preprint
- Neural Networks for Information Retrieval (SIGIR 2017)
  
  Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2017
  
  Publication Preprint PDF
- Learning to Match Using Local and Distributed Representations of Text for Web Search
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 26th International Conference on World Wide Web | April 2017
  
  Publication Preprint PDF PDF
- What snippet size is needed in mobile web search?
  
  Jaewon Kim, Paul Thomas, Ramesh Sankaranarayana, Tom Gedeon
  
  Proceedings of the Conference on Computer-Human Information Interaction and Retrieval | March 2017
  
  Publication Publication
- A Theoretical Framework for Conversational Search
  
  Filip Radlinski, Nick Craswell
  
  CHIIR 2017 | March 2017
  
  Publication
- Neural Text Embeddings for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM'17) | February 2017
  
  Publication PDF
- Query-biased summaries for tabular data
  
  Vincent Au, Paul Thomas, Gaya K Jayasinghe
  
  Proceedings of the Australasian Document Computing Symposium | December 2016
  
  Publication Publication Publication
- Pagination versus scrolling in mobile web search
  
  Jaewon Kim, Paul Thomas, Ramesh Sankaranarayana, Tom Gedeon, Hwan-Jin Yoon
  
  Proceedings of the International ACM SIGIR Conference on on Information and Knowledge Management | October 2016
  
  Publication Publication
- UQV100: A Test Collection with Query Variability
  
  Peter Bailey, Alistair Moffat, Falk Scholer, Paul Thomas
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2016
  
  Publication Publication
- Neu-IR: The SIGIR 2016 Workshop on Neural Information Retrieval
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2016
  
  Publication PDF
- Ingrams – a neuropsychological explanation for why people search
  
  Peter Bailey, Nick Craswell
  
  Proceedings of the 39th Annual ACM SIGIR Conference (SIGIR 2016) | July 2016
  
  Publication Publication
- A Proposal for Evaluating Answer Distillation from Web Data
  
  Bhaskar Mitra, Grady Simon, Jianfeng Gao, Nick Craswell, Li Deng
  
  Proceedings of the Second WebQA Workshop (SIGIR 2016) | July 2016
  
  PDF PDF
- Towards Conversational Recommender Systems
  
  Konstantina Christakopoulou, Filip Radlinski, Katja Hofmann
  
  22nd ACM SigKDD Conference on Knowledge Discovery and Data Mining | June 2016
  
  Publication Project
- Query Expansion with Locally-Trained Word Embeddings
  
  Fernando Diaz, Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics | June 2016
  
  Publication Preprint PDF PDF
- Improving Document Ranking with Dual Word Embeddings
  
  Eric Nalisnick, Bhaskar Mitra, Nick Craswell, Rich Caruana
  
  WWW'16 | April 2016
  
  Publication PDF PDF Project
- Where Can I Buy a Boulder? Searching for Offline Retail Locations
  
  Sandro Bauer, Filip Radlinski, Ryen W. White
  
  Proceedings of the 25th International World Wide Web Conference (WWW) (to appear) | April 2016
  
  Publication
- Did You Say U2 or Youtube? Inferring Implicit Transcripts from Voice Search Logs
  
  Nick Craswell, Umut Ozertem, Milad Shokouhi
  
  Proceedings of the 25th International World Wide Web Conference (WWW2016) | April 2016
  
  Publication
- Query Auto-Completion for Rare Prefixes
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM) | October 2015
  
  Publication PDF PDF
- Exploring Session Context using Distributed Representations of Queries and Reformulations
  
  Bhaskar Mitra
  
  Proceedings of the 38th international ACM SIGIR conference on Research & development in information retrieval | August 2015
  
  Publication PDF PDF
- An Eye-tracking Study of User Interactions with Query Auto Completion
  
  Katja Hofmann, Bhaskar Mitra, Filip Radlinski, Milad Shokouhi
  
  Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
  
  Publication PDF Video
- Relevance and Effort: An Analysis of Document Utility
  
  Emine Yilmaz, Manisha Verma, Nick Craswell, Filip Radlinski, Peter Bailey
  
  Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
  
  Publication
- On User Interactions with Query Auto-Completion
  
  Bhaskar Mitra, Milad Shokouhi, Filip Radlinski, Katja Hofmann
  
  Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR) | July 2014
  
  Publication PDF
- Beyond Clicks: Query Reformulation as a Predictor of Search Satisfaction
  
  Ahmed Awadallah, Xiaolin Shi, Nick Craswell, Bill Ramsey
  
  ACM International Conference on Information and Knowledge Management (CIKM) | October 2013
  
  Publication
- Optimized Interleaving for Online Retrieval Evaluation
  
  Filip Radlinski, Nick Craswell
  
  ACM International Conference on Web Search and Data Mining (WSDM) | February 2013
  
  ACM International Conference on Web Search and Data Mining (WSDM)
  
  Best Paper Award
  
  Publication
- Robust Query Rewriting using Anchor Data
  
  Nick Craswell, Bodo von Billerbeck, Dennis Fetterly, Marc Najork
  
  6th ACM International Conference on Web Search and Data Mining (WSDM) | February 2013
  
  Publication
- Modeling the Impact of Short- and Long-Term Behavior on Search Personalization
  
  Paul Bennett, Ryen W. White, Wei Chu, Susan Dumais, Peter Bailey, Fedor Borisyuk, Xiaoyuan Cui
  
  Proceedings of the 35th Annual ACM SIGIR Conference (SIGIR 2012) | August 2012
  
  Publication
- Probabilistic Models for Personalizing Web Search
  
  David Sontag, Kevyn Collins-Thompson, Paul Bennett, Ryen W. White, Susan Dumais, Bodo von Billerbeck
  
  WSDM 2012: Fifth International Conference on Web Search and Data Miniing | February 2012
  
  Publication
- Microsoft Research at TREC 2011 Web Track
  
  Bodo von Billerbeck, Nick Craswell, Dennis Fetterly, Marc Najork
  
  Proc. of the 20th Text Retrieval Conference (TREC) | November 2011
  
  Publication
- Inferring and Using Location Metadata to Personalize Web Search
  
  Paul Bennett, Filip Radlinski, Ryen W. White, Emine Yilmaz
  
  Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR) | July 2011
  
  Publication
- The Power of Peers
  
  Nick Craswell, Dennis Fetterly, Marc Najork
  
  33rd European Conference on IR Research (ECIR) | April 2011
  
  33rd European Conference on IR Research (ECIR)
  
  Publication
- Personalizing Web Search using Long Term Browsing History
  
  Nicolaas Matthijs, Filip Radlinski
  
  ACM International Conference on Web Search and Data Mining (WSDM) | February 2011
  
  Publication
- LambdaMerge: Merging the Results of Query Reformulations
  
  Daniel Sheldon, Milad Shokouhi, Martin Szummer, Nick Craswell
  
  Conf. Web search and data mining (WSDM) | February 2011
  
  Publication Publication
- Microsoft Research at TREC 2010 Web Track
  
  Nick Craswell, Dennis Fetterly, Marc Najork
  
  Proc. of the 19th Text Retrieval Conference (TREC) | November 2010
  
  Proc. of the 19th Text Retrieval Conference (TREC)
  
  Publication
- Metrics for Assessing Sets of Subtopics
  
  Filip Radlinski, Martin Szummer, Nick Craswell
  
  SIGIR Conf. Research and Development in Information Retrieval | July 2010
  
  Publication
- Comparing the Sensitivity of Information Retrieval Metrics
  
  Filip Radlinski, Nick Craswell
  
  Proceedings of SIGIR | July 2010
  
  Publication
- Inferring Query Intent from Reformulations and Clicks
  
  Filip Radlinski, Martin Szummer, Nick Craswell
  
  Proc. 19th Annual International World Wide Web Conference (WWW '10). | April 2010
  
  Proc. 19th Annual International World Wide Web Conference (WWW '10).
  
  Publication
- Microsoft Research at TREC 2009: Web and Relevance Feedback Tracks
  
  Nick Craswell, Dennis Fetterly, Marc Najork, Stephen Robertson, Emine Yilmaz
  
  Proc. of the 18th Text Retrieval Conference (TREC) | November 2009
  
  Publication
- The Impact of Crawl Policy on Web Search Effectiveness
  
  Dennis Fetterly, Nick Craswell, Vishwa Vinay
  
  Proceedings of the 32nd annual international ACM SIGIR conference on Research and development in information retrieval | July 2009
  
  Publication
- Measuring the Search Effectiveness of a Breadth-First Crawl
  
  Dennis Fetterly, Nick Craswell, Vishwa Vinay
  
  Proceedings of the 31st European Conference on Information Retrieval (ECIR) | April 2009
  
  Proceedings of the 31st European Conference on Information Retrieval (ECIR)
  
  Publication
- Efficient and Effective Link Analysis with Precomputed SALSA Maps
  
  Marc Najork, Nick Craswell
  
  17th ACM Conference on Information and Knowledge Management (CIKM) | October 2008
  
  17th ACM Conference on Information and Knowledge Management (CIKM)
  
  Publication
- A Decision Theoretic Framework for Ranking using Implicit Feedback
  
  Onno Zoeter, Mike Taylor, Ed Snelson, John Guiver, Nick Craswell, Martin Szummer
  
  SIGIR 2008 Workshop on Learning to Rank for Information Retrieval | July 2008
  
  Publication
- Behavioral Classification on the Click Graph
  
  Martin Szummer, Nick Craswell
  
  World Wide Web Conference | April 2008
  
  World Wide Web Conference
  
  Publication
- Relevance assessment: are judges exchangeable and does it matter
  
  Peter Bailey, Nick Craswell, Ian Soboroff, Paul Thomas, Arjen P. de Vries, Emine Yilmaz
  
  SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval | January 2008
  
  ISBN: 978-1-60558-164-4
  
  Publication Publication
- Overview of the INEX 2007 Entity Ranking Track
  
  Arjen P. de Vries, Anne-Marie Vercoustre, James A. Thom, Nick Craswell, Mounia Lalmas
  
  January 2008
  
  ISBN: 978-3-540-85901-7
  
  Publication Publication
- An experimental comparison of click position-bias models
  
  Nick Craswell, Mike Taylor
  
  WSDM '08: Proceedings of the international conference on Web search and web data mining | January 2008
  
  ISBN: 978-1-59593-927-9
  
  Publication Publication
- Random Walks on the Click Graph
  
  Nick Craswell, Martin Szummer
  
  Proceedings of SIGIR 2007 | July 2007
  
  ISBN: 978-1-59593-597-7
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_sigir07.pdf
  
  Publication Github
- Optimisation methods for ranking functions with multiple parameters
  
  Mike Taylor, Hugo Zaragoza, Nick Craswell, Stephen Robertson, Chris J.C. Burges
  
  Proceedings of CIKM 2006 | January 2006
  
  Proceedings of CIKM 2006
  
  ISBN: 1-59593-433-2
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/taylor_cikm06.pdf
  
  Publication
- Relevance Weighting for Query Independent Evidence
  
  Nick Craswell, Stephen Robertson, Hugo Zaragoza, Mike Taylor
  
  Proceedings of ACM SIGIR 2005 | January 2005
  
  ISBN: 1-59593-034-5
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_sigir05.pdf
  
  Publication
- Focused crawling for both topical relevance and quality of medical information
  
  Thanh Tang, David Hawking, Nick Craswell, Kathleen Griffiths
  
  Proceedings of CIKM 2005 | January 2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/tang_cikm05.pdf
  
  Publication
- Microsoft Cambridge at TREC-14: Enterprise Track
  
  Nick Craswell, Hugo Zaragoza, Stephen Robertson
  
  Proceedings of TREC-2005 | January 2005
  
  Proceedings of TREC-2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec05.pdf
  
  Publication
- The TREC terabyte retrieval track
  
  Charles Clarke, Nick Craswell, Ian Soboroff
  
  SIGIR Forum | January 2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/clarke_sigirforum05.pdf
  
  Publication
- RMIT University at TREC 2004
  
  Bodo von Billerbeck, Adam Cannane, Abhijit Chattaraj, Nicholas Lester, William Webber, Hugh E. Williams, John Yiannis, Justin Zobel
  
  Proceedings Text Retrieval Conference (TREC) | November 2004
  
  Publication
- Microsoft Cambridge at TREC 2004: Web and HARD track
  
  Hugo Zaragoza, Nick Craswell, Mike Taylor, Suchi Saria, Stephen Robertson
  
  Proceedings of TREC-2004 | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/zaragoza_trec04.pdf
  
  Publication Publication
- Testbed for Information Extraction from Deep Web
  
  Yasuhiro Yamada, Nick Craswell, Tetsuya Nakatoh, Sachio Hirokawa
  
  WWW2004 Poster Proceedings | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/yamada_www2004poster.pdf
  
  Publication
- Toward Better Weighting of Anchors
  
  David Hawking, Trystan Upstill, Nick Craswell
  
  Proceedings of ACM SIGIR 2004 (poster abstract) | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_sigir04poster.pdf
  
  Publication
- Predicting Fame and Fortune: PageRank or Indegree?
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  Proceedings of the Australasian Document Computing Symposium, ADCS2003 | December 2003
  
  Proceedings of the Australasian Document Computing Symposium, ADCS2003
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_adcs03.pdf
  
  Publication
- When query expansion fails
  
  Bodo von Billerbeck
  
  Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval | July 2003
  
  ISBN: 1-58113-646-3
  
  Publication
- Query-independent evidence in home page finding
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  January 2003
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_tois03.pdf
  
  Publication
- Automated Discovery of Search Interfaces on the Web
  
  Jared Cope, Nick Craswell, David Hawking
  
  The Fourteenth Australasian Database Conference | January 2003
  
  The Fourteenth Australasian Database Conference
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/cope_adc03.pdf
  
  Publication
- Query expansion using associated queries
  
  Bodo von Billerbeck
  
  Proceedings of the twelfth international conference on information and knowledge management | January 2003
  
  ISBN: 1-58113-723-0
  
  Publication
- Overview of the TREC-2002 Web Track
  
  Nick Craswell, David Hawking
  
  Proceedings of TREC-2002 | November 2002
  
  Proceedings of TREC-2002
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec02wt.pdf
  
  Publication
- Buying bestsellers online: A case study in Search & Searchability
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  7th Australasian Document Computing Symposium | January 2002
  
  7th Australasian Document Computing Symposium
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_adcs02.pdf
  
  Publication
- TREC10 Web and Interactive Tracks at CSIRO
  
  Nick Craswell, David Hawking, Ross Wilkinson, Mingfang Wu
  
  Proceedings of TREC-2001 | November 2001
  
  Proceedings of TREC-2001
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec01.pdf
  
  Publication
- Judging the Judges: A Collection of LLM-Generated Relevance Judgements
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  February 2025
  
  Preprint
- Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  August 2024
  
  Publication Preprint PDF
- Are We There Yet? A Decision Framework for Replacing Term-Based Retrieval with Dense Retrieval Systems
  
  Sebastian Hofstätter, Nick Craswell, Bhaskar Mitra, Hamed Zamani, Allan Hanbury
  
  June 2022
  
  Preprint
- Semantic Product Search for Matching Structured Product Catalogs in E-Commerce
  
  Jason Ingyu Choi, Surya Kallumadi, Bhaskar Mitra, Eugene Agichtein, Faizan Javed
  
  August 2020
  
  Preprint
- Conformer-Kernel with Query Term Independence for Document Retrieval
  
  Bhaskar Mitra, Sebastian Hofstätter, Hamed Zamani, Nick Craswell
  
  July 2020
  
  Preprint Github
- MIMICS: A Large-Scale Data Collection for Search Clarification
  
  Hamed Zamani, Gord Lueck, Everest Chen, Rodolfo Quispe, Flint Luu, Nick Craswell
  
  June 2020
- Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
  
  Bhaskar Mitra, Corby Rosset, David Hawking, Nick Craswell, Fernando Diaz, Emine Yilmaz
  
  July 2019
  
  Preprint Project Project
- An updated duet model for passage re-ranking
  
  Bhaskar Mitra, Nick Craswell
  
  March 2019
  
  Preprint
- Methods for Distributed Information Retrieval
  
  Nick Craswell
  
  January 2000
  
  January 2000
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_thesis00.pdf
  
  Publication
- Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval
  
  Bhaskar Mitra
  
  December 2020
  
  PhD Thesis: University College London | December 2020
  
  Publication Preprint PDF PDF
- Simulating information retrieval test collections
  
  David Hawking, Bodo von Billerbeck, Paul Thomas, Nick Craswell
  
  Published by Morgan & Claypool | September 2020
  
  Editor(s): Gary Marchionini
  
  ISBN: 9781681739571
  
  Synthesis Lectures on Information Concepts, Retrieval, and Services
  
  Publication DOI
- Large Scale Log Analysis of Individuals’ Domain Preferences in Web Search
  
  Sarah K. Tyler, Jaime Teevan, Peter Bailey, Sebastian de la Chica, Nikhil Dandekar
  
  MSR-TR-2015-048 | June 2015
  
  Publication
- A Two-Stage Model for Expert Search
  
  Yunbo Cao, Jingjing Liu, Shenghua Bao, Hang Li, Nick Craswell
  
  MSR-TR-2008-143 | October 2008
  
  Publication
- A Two-Stage Model for Expert Search
  
  Yunbo Cao, JJ (Jingjing) Liu, Shenghua Bao, Hang Li, Nick Craswell
  
  MSR-TR-2008-195 | October 2008
  
  Publication

People

Nick Craswell

Principal Architect

Learn more

Paul Thomas

Senior applied scientist

Learn more

Web and AI Sciences

Publications

All claims are equal, but some claims are more equal than others: Importance-sensitive factuality evaluation of LLM generations

Demographically-inspired query variants using an LLM

Preaching to the ChoIR: Lessons IR should share with AI

System comparison using automated generation of relevance judgements in multiple languages

Report from the 4th Strategic Workshop on Information Retrieval in Lorne (SWIRL 2025)

LLM4Eval@WSDM 2025: Large Language Model for Evaluation in Information Retrieval

Judging the Judges: A Collection of LLM-Generated Relevance Judgements

JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment

SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval

LLMs can be Fooled into Labelling a Document as Relevant (Best café near me; this paper is perfectly relevant)

Proceedings of The First Workshop on Large Language Models for Evaluation in Information Retrieval (LLM4Eval 2024)

LLMJudge: LLMs for Relevance Judgments

Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024

Large Language Models Can Accurately Predict Searcher Preferences

LLM4Eval: Large Language Model for Evaluation in IR

What Matters in a Measure? A Perspective from Large-Scale Search Evaluation

Enhancing human annotation: Leveraging large language models and efficient batch processing

Can generative LLMs create query variants for test collections? An exploratory study

Taking Search to Task

A Multi-objective Optimization Framework for Multi-stakeholder Fairness-aware Recommendation

A flexible framework for offline effectiveness metrics

Are We There Yet? A Decision Framework for Replacing Term-Based Retrieval with Dense Retrieval Systems

Overview of the TREC 2021 deep learning track

Fostering Coopetition While Plugging Leaks: The Design and Implementation of the MS MARCO Leaderboards

Inconsistent Ranking Assumptions in Medical Search and Their Downstream Consequences

Joint Multisided Exposure Fairness for Recommendation

Search interfaces for biomedical searching: How do gaze, user perception, search behaviour and search performance relate?

The crowd is made of people: Observations from large-scale crowd labelling

Report on the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021)

Bias-aware Fair Neural Ranking for Addressing Stereotypical Gender Biases

Exposing Query Identification for Search Transparency

Less is Less: When Are Snippets Insufficient for Human vs Machine Relevance Estimation?

Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness

Analysing mixed initiatives and search strategies during conversational search

MS MARCO Chameleons: Challenging the MS MARCO Leaderboard with Extremely Obstinate Queries

Theories of conversation for conversational IR

Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval (Dissertation Abstract)

Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking

Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Models

Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence

MS MARCO: Benchmarking Ranking Models in the Large-Data Regime

TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime

Proceedings of the 2021 Conference on Human Information Interaction and Retrieval

Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard

Social competence in conversational retrieval

Overview of the TREC 2020 deep learning track

Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification

Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval

Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track

Simulating information retrieval test collections

Semantic Product Search for Matching Structured Product Catalogs in E-Commerce

Conformer-Kernel with Query Term Independence for Document Retrieval

Evaluating Stochastic Rankings with Expected Exposure

Expressions of style in information-seeking conversation with an embodied agent

MIMICS: A Large-Scale Data Collection for Search Clarification

ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search

Analyzing and Learning from User Interactions for Search Clarification

Local Self-Attention over Long Text for Efficient Document Retrieval

On the Reliability of Test Collections for Evaluating Systems of Different Types

Data-driven evaluation metrics for heterogeneous search engine result pages

Overview of the TREC 2019 deep learning track

Theories of conversation for conversational IR

Towards a model for spoken conversational search

Duet at TREC 2019 Deep Learning Track

Investigating searchers’ mental models to inform search explanations

Report on the First HIPstIR Workshop on the Future of Information Retrieval

cwl_eval: An evaluation tool for information retrieval

The Emotion Profile of Web Search

Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks

Building economic models and measures of search (tutorial)

An Axiomatic Approach to Regularizing Neural Ranking Models

An updated duet model for passage re-ranking

Data sets for spoken conversational search

An Introduction to Neural Information Retrieval

Better effectiveness metrics for SERPs, cards, and rankings

A Line in the Sand: Recommendation or Ad-hoc Retrieval?

Measuring the Utility of Search Engine Result Pages: An Information Foraging Based Measure

Cross Domain Regularization for Neural Ranking Models Using Adversarial Learning