Nick Craswell

Principal Architect

Publications

View by:

- Evaluation Validity in Information Retrieval
  
  Paul Thomas, Nick Craswell, Mark Sanderson, Seth Spielman, Robert Sim, Ryen W. White
  
  2026 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2026
  
  DOI
- All claims are equal, but some claims are more equal than others: Importance-sensitive factuality evaluation of LLM generations
  
  Miriam Wanner, Leif Azzopardi, Paul Thomas, Soham Dan, Ben Van Durme, Nick Craswell
  
  October 2025
  
  preprint arXiv:2510.07083
  
  Publication
- Report from the 4th Strategic Workshop on Information Retrieval in Lorne (SWIRL 2025)
  
  Johanne R Trippas, J. Shane Culpepper, Mohammad Aliannejadi, James Allan, Enrique Amigó, Jaime Arguello, Leif Azzopardi, Peter Bailey, Jamie Callan, Rob Capra, Nick Craswell, Bruce Croft, Jeff Dalton, Gianluca Demartini, Laura Dietz, Zhicheng Dou, Carsten Eickhoff, Michael Ekstrand, Nicola Ferro, Norbert Fuhr, Dorota Glowacka, Faegheh Hasibi, Danula Hettiachchi, Rosie Jones, Jaap Kamps, Noriko Kando, Sarvnaz Karimi, Makoto P. Kato, Bevan Koopman, Yiqun Liu, Chenglong Ma, Joel Mackenzie, Maria Maistro, Jiaxin Mao, Dana McKay, Bhaskar Mitra, Stefano Mizzaro, Alistair Moffat, Josiane Mothe, Iadh Ounis, Lida Rashidi, Yongli Ren, Mark Sanderson, Rodrygo Santos, Falk Scholer, Chirag A Shah, Laurianne Sitbon, Ian Soboroff, Damiano Spina, Paul Thomas, Julián Urbano, Arjen P. de Vries, Ryen W. White, Abby Yuan, Hamed Zamani, Oleg Zendel, Min Zhang, Shengyao Zhuang, Justin Zobel, Guido Zuccon
  
  ACM SIGIR Forum | June 2025, Vol 59(1): pp. 1-68
  
  DOI PDF
- LLM4Eval@WSDM 2025: Large Language Model for Evaluation in Information Retrieval
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L A Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  2025 Web Search and Data Mining | March 2025
  
  Publication
- Judging the Judges: A Collection of LLM-Generated Relevance Judgements
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  February 2025
  
  Preprint
- JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment
  
  Hossein A. Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra
  
  2025 The Web Conference | January 2025
  
  Publication Preprint PDF
- SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval
  
  Hossein A. Rahmani, Xi Wang, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas
  
  2025 The Web Conference | January 2025
  
  Publication PDF Preprint
- Proceedings of The First Workshop on Large Language Models for Evaluation in Information Retrieval (LLM4Eval 2024)
  
  Clemencia Siro, Mohammad Aliannejadi, Hossein A. Rahmani, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  September 2024
  
  Publication
- LLMJudge: LLMs for Relevance Judgments
  
  Hossein A Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas, Charles L A Clarke, Mohammad Aliannejadi, Clemencia Siro, Guglielmo Faggioli
  
  August 2024
  
  Publication Preprint
- Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  August 2024
  
  Publication Preprint PDF
- Large Language Models Can Accurately Predict Searcher Preferences
  
  Paul Thomas, Seth Spielman, Nick Craswell, Bhaskar Mitra
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  An earlier version of this paper appeared as arXiv preprint arXiv:2309.10621v1 [cs.IR].
  
  Publication PDF PDF
- LLM4Eval: Large Language Model for Evaluation in IR
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  Workshop description
  
  Publication PDF
- What Matters in a Measure? A Perspective from Large-Scale Search Evaluation
  
  Paul Thomas, Gabriella Kazai, Nick Craswell, Seth Spielman
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  Best paper nomination
  
  DOI PDF
- Synthetic Test Collections for Retrieval Evaluation
  
  Hossein A. Rahmani, Nick Craswell, Emine Yilmaz, Bhaskar Mitra, Daniel Campos
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | May 2024
  
  Publication Preprint PDF
- MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels
  
  Qi Chen, Xiubo Geng, Corby Rosset, Carolyn Buractaon, Jingwen Lu, Tao Shen, Kun Zhou, Chenyan Xiong, Yeyun Gong, Paul Bennett, Nick Craswell, Xing Xie, Fan Yang, Bryan Tower, Zheng Liu, Mingqin Li, Chuanjie Liu, Jason (Zengzhong) Li, Rangan Majumder, Jennifer Neville, Harsha Simhadri, Manik Varma, Yujing Wang, Linjun Yang, Mao Yang, Ce Zhang
  
  2024 The Web Conference | May 2024
  
  Publication PDF Access
- Towards Group-aware Search Success
  
  Haolun Wu, Bhaskar Mitra, Nick Craswell
  
  International Conference on the Theory of Information Retrieval | April 2024
  
  Publication Preprint PDF
- Overview of the TREC 2023 Deep Learning Track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Hossein A. Rahmani, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
  
  Text REtrieval Conference (TREC) | February 2024
  
  Publication PDF Access
- Patterns of gender-specializing query reformulation
  
  Amifa Raj, Bhaskar Mitra, Nick Craswell, Michael D. Ekstrand
  
  2023 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2023
  
  Publication Preprint PDF PDF
- Overview of the TREC 2022 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
  
  Text REtrieval Conference (TREC) | March 2023
  
  Publication PDF Access
- Neural Approaches to Conversational Information Retrieval
  
  Jianfeng Gao, Chenyan Xiong, Paul Bennett, Nick Craswell
  
  Published by Springer | March 2023, Vol 44
  
  ISBN: 978-3-031-23079-0
  
  The Information Retrieval Series.
  
  Preprint
- Zero-shot Clarifying Question Generation for Conversational Search
  
  Zhenduo Wang, Yuancheng Tu, Nick Craswell, Corby Rosset, Ming Wu, Qingyao Ai
  
  2023 The Web Conference | March 2023
  
  Publication
- Are We There Yet? A Decision Framework for Replacing Term-Based Retrieval with Dense Retrieval Systems
  
  Sebastian Hofstätter, Nick Craswell, Bhaskar Mitra, Hamed Zamani, Allan Hanbury
  
  June 2022
  
  Preprint
- Overview of the TREC 2021 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Text REtrieval Conference (TREC) | May 2022
  
  Publication PDF PDF Access
- Fostering Coopetition While Plugging Leaks: The Design and Implementation of the MS MARCO Leaderboards
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF Access
- The crowd is made of people: Observations from large-scale crowd labelling
  
  Paul Thomas, Gabriella Kazai, Ryen W. White, Nick Craswell
  
  ACM SIGIR Conference on Human Information Interaction and Retrieval | March 2022
  
  Publication DOI
- Summarize with Caution: Comparing Global Feature Attributions
  
  Alex Okeson, Rich Caruana, Nick Craswell, Kori Inkpen, Scott M. Lundberg, Harsha Nori, Hanna Wallach, Jennifer Wortman Vaughan
  
  Bulletin of the IEEE Computer Society Technical Committee on Data Engineering | December 2021
  
  Publication Project
- Less is Less: When Are Snippets Insufficient for Human vs Machine Relevance Estimation?
  
  Gabriella Kazai, Bhaskar Mitra, Anlei Dong, Nick Craswell, Linjun Yang
  
  European Conference on Information Retrieval | November 2021
  
  Publication Preprint PDF PDF
- Analysing mixed initiatives and search strategies during conversational search
  
  Mohammad Aliannejadi, Leif Azzopardi, Hamed Zamani, Evangelos Kanoulas, Paul Thomas, Nick Craswell
  
  ACM International Conference on Information and Knowledge Management | November 2021
  
  DOI
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  ACM Transactions on Information Systems | August 2021, Vol 39(4)
  
  Article 39
  
  Publication
- Do affective cues validate behavioural metrics for search?
  
  Daniel McDuff, Paul Thomas, Kael Rowan, Nick Craswell, Mary Czerwinski
  
  SIGIR 2021 | July 2021
  
  PDF DOI
- Report on the future conversations workshop at CHIIR 2021
  
  Damiano Spina, Johanne R Trippas, Paul Thomas, Hideo Joho, Katriina Byström, Leigh Clark, Nick Craswell, Mary Czerwinski, David Elsweiler, Alexander Frummet, Souvick Ghosh, Johannes Kiesel, Irene Lopatovska, Daniel McDuff, Selina Meyer, Ahmed Mourad, Paul Owoicho, Sachin Pathiyan Cherumanal, Daniel Russell, Laurianne Sitbon
  
  SIGIR Forum | June 2021, Vol 55(1): pp. 1-22
  
  Publication PDF
- Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
  
  Sebastian Hofstatter, Bhaskar Mitra, Hamed Zamani, Nick Craswell, Allan Hanbury
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
  
  Publication Preprint PDF PDF
- Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF
- MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees, Ian Soboroff
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | March 2021
  
  Publication Preprint PDF Access
- Social competence in conversational retrieval
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  Future Conversations | March 2021
  
  Publication
- Overview of the TREC 2020 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos
  
  Text REtrieval Conference (TREC) | February 2021
  
  Publication Preprint PDF PDF Access
- Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Text REtrieval Conference (TREC) | November 2020
  
  Publication Preprint PDF PDF Access
- Simulating information retrieval test collections
  
  David Hawking, Bodo von Billerbeck, Paul Thomas, Nick Craswell
  
  Published by Morgan & Claypool | September 2020
  
  Editor(s): Gary Marchionini
  
  ISBN: 9781681739571
  
  Synthesis Lectures on Information Concepts, Retrieval, and Services
  
  Publication DOI
- Conformer-Kernel with Query Term Independence for Document Retrieval
  
  Bhaskar Mitra, Sebastian Hofstätter, Hamed Zamani, Nick Craswell
  
  July 2020
  
  Preprint Github
- Expressions of style in information-seeking conversation with an embodied agent
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2020
  
  PDF DOI
- MIMICS: A Large-Scale Data Collection for Search Clarification
  
  Hamed Zamani, Gord Lueck, Everest Chen, Rodolfo Quispe, Flint Luu, Nick Craswell
  
  June 2020
- ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search
  
  Nick Craswell, Daniel Campos, Bhaskar Mitra, Emine Yilmaz, Bodo von Billerbeck
  
  Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | June 2020
  
  Publication Preprint PDF Access
- Analyzing and Learning from User Interactions for Search Clarification
  
  Hamed Zamani, Bhaskar Mitra, Everest Chen, Gord Lueck, Fernando Diaz, Paul Bennett, Nick Craswell, Susan Dumais
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF PDF
- Local Self-Attention over Long Text for Efficient Document Retrieval
  
  Sebastian Hofstätter, Hamed Zamani, Bhaskar Mitra, Nick Craswell, Allan Hanbury
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF
- Generating Clarifying Questions for Information Retrieval
  
  Hamed Zamani, Susan Dumais, Nick Craswell, Paul Bennett, Gord Lueck
  
  The Web Conference 2020 (formerly WWW conference) | April 2020
  
  Publication
- Leading Conversational Search by Suggesting Useful Questions
  
  Corby Rosset, Chenyan Xiong, Xia Song, Daniel Campos, Nick Craswell, Saurabh Tiwary, Paul Bennett
  
  The Web Conference 2020 (formerly WWW conference) | April 2020
  
  Publication Publication
- On the Reliability of Test Collections for Evaluating Systems of Different Types
  
  Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Daniel Campos
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | April 2020
  
  Publication Preprint PDF PDF Access
- Data-driven evaluation metrics for heterogeneous search engine result pages
  
  Leif Azzopardi, Ryen W. White, Paul Thomas, Nick Craswell
  
  Proceedings of the Conference on Human Information Interaction and Retrieval | March 2020
  
  DOI Publication PDF
- Overview of the TREC 2019 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees
  
  Text REtrieval Conference (TREC) | March 2020
  
  Publication PDF PDF Access
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  International Workshop on Conversational Approaches to Information Retrieval | March 2020
  
  An extended version of this paper also appears as Thomas et al., ACM Trans. Info. Sys. 39(2), 2021: https://dl.acm.org/doi/pdf/10.1145/3439869.
  
  Publication
- Duet at TREC 2019 Deep Learning Track
  
  Bhaskar Mitra, Nick Craswell
  
  Text REtrieval Conference (TREC) | December 2019
  
  Publication Preprint PDF PDF Access
- Investigating searchers’ mental models to inform search explanations
  
  Paul Thomas, Bodo von Billerbeck, Nick Craswell, Ryen W. White
  
  ACM Transactions on Information Systems | December 2019, Vol 38(1)
  
  Article 10
  
  DOI Publication
- Generic Intent Representation in Web Search
  
  Hongfei Zhang, Xia Song, Chenyan Xiong, Corby Rosset, Paul Bennett, Nick Craswell, Saurabh Tiwary
  
  The 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019) | July 2019
  
  Publication Project Project
- The Emotion Profile of Web Search
  
  Gabriella Kazai, Paul Thomas, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2019
  
  Publication
- Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
  
  Bhaskar Mitra, Corby Rosset, David Hawking, Nick Craswell, Fernando Diaz, Emine Yilmaz
  
  July 2019
  
  Preprint Project Project
- An Axiomatic Approach to Regularizing Neural Ranking Models
  
  Corby Rosset, Bhaskar Mitra, Chenyan Xiong, Nick Craswell, Xia Song, Saurabh Tiwary
  
  Proceedings of the 42nd international ACM SIGIR conference on Research & development in information retrieval | April 2019
  
  Publication Preprint PDF PDF Project Project
- An updated duet model for passage re-ranking
  
  Bhaskar Mitra, Nick Craswell
  
  March 2019
  
  Preprint
- An Introduction to Neural Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Foundations and Trends® in Information Retrieval | December 2018, Vol 13(1): pp. 1-126
  
  Publication PDF PDF
- Better effectiveness metrics for SERPs, cards, and rankings
  
  Paul Thomas, Alistair Moffat, Peter Bailey, Falk Scholer, Nick Craswell
  
  Australasian Document Computing Symposium | December 2018
  
  Best paper award
  
  DOI Publication
- Measuring the Utility of Search Engine Result Pages: An Information Foraging Based Measure
  
  Leif Azzopardi, Paul Thomas, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2018
  
  Publication Publication Publication
- Style and alignment in information-seeking conversation
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell, Gloria Mark
  
  Proceedings of the Conference on Computer-Human Information Interaction and Retrieval | March 2018
  
  Publication Publication
- Neural Ranking Models with Multiple Document Fields
  
  Hamed Zamani, Bhaskar Mitra, Xia Song, Nick Craswell, Saurabh Tiwary
  
  Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM'18) | November 2017
  
  Publication Preprint PDF Project Project
- Multimodal analysis of vocal collaborative search: A public corpus and results
  
  Daniel McDuff, Paul Thomas, Mary Czerwinski, Nick Craswell
  
  Proceedings of the ACM International Conference on Multimodal Interaction | November 2017
  
  Publication Publication
- Neural information retrieval: introduction to the special issue
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Information Retrieval Journal : Special issue Neural | November 2017
  
  Publication PDF
- Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rĳke, Jiafeng Guo, Bhaskar Mitra
  
  SIGIR Forum | October 2017, Vol 52
  
  Publication PDF PDF
- MISC: A data set of information-seeking conversations
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the 1st International Workshop on Conversational Approaches to Information Retrieval | August 2017
  
  Publication
- SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval | August 2017
  
  Publication PDF
- Luandri: a Clean Lua Interface to the Indri Search Engine
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 40th international ACM SIGIR conference on Research & development in information retrieval | August 2017
  
  Publication Preprint PDF PDF
- Neural Models for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  May 2017
  
  Preprint Video
- Learning to Match Using Local and Distributed Representations of Text for Web Search
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 26th International Conference on World Wide Web | April 2017
  
  Publication Preprint PDF PDF
- A Theoretical Framework for Conversational Search
  
  Filip Radlinski, Nick Craswell
  
  CHIIR 2017 | March 2017
  
  Publication
- Neural Text Embeddings for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM'17) | February 2017
  
  Publication PDF
- Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rĳke, Bhaskar Mitra
  
  SIGIR Forum | December 2016, Vol 50(2): pp. 96-103
  
  Publication PDF
- Ingrams – a neuropsychological explanation for why people search
  
  Peter Bailey, Nick Craswell
  
  Proceedings of the 39th Annual ACM SIGIR Conference (SIGIR 2016) | July 2016
  
  Publication Publication
- Neu-IR: The SIGIR 2016 Workshop on Neural Information Retrieval
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2016
  
  Publication PDF
- A Proposal for Evaluating Answer Distillation from Web Data
  
  Bhaskar Mitra, Grady Simon, Jianfeng Gao, Nick Craswell, Li Deng
  
  Proceedings of the Second WebQA Workshop (SIGIR 2016) | July 2016
  
  PDF PDF
- Query Expansion with Locally-Trained Word Embeddings
  
  Fernando Diaz, Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics | June 2016
  
  Publication Preprint PDF PDF
- Explicit In Situ User Feedback for Web Search Results
  
  Jin Young Kim, Jaime Teevan, Nick Craswell
  
  Proceedings of the 39th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2016), Pisa, Italy | June 2016
  
  Publication
- Improving Document Ranking with Dual Word Embeddings
  
  Eric Nalisnick, Bhaskar Mitra, Nick Craswell, Rich Caruana
  
  WWW'16 | April 2016
  
  Publication PDF PDF Project
- Did You Say U2 or Youtube? Inferring Implicit Transcripts from Voice Search Logs
  
  Nick Craswell, Umut Ozertem, Milad Shokouhi
  
  Proceedings of the 25th International World Wide Web Conference (WWW2016) | April 2016
  
  Publication
- A Dual Embedding Space Model for Document Ranking
  
  Bhaskar Mitra, Eric Nalisnick, Nick Craswell, Rich Caruana
  
  February 2016
  
  This paper is an extended evaluation and analysis of the model proposed in a poster to appear in WWW'16, April 11 - 15, 2016, Montreal, Canada.
  
  Preprint PDF Access Project
- Query Auto-Completion for Rare Prefixes
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM) | October 2015
  
  Publication PDF PDF
- Relevance and Effort: An Analysis of Document Utility
  
  Emine Yilmaz, Manisha Verma, Nick Craswell, Filip Radlinski, Peter Bailey
  
  Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
  
  Publication
- Beyond Clicks: Query Reformulation as a Predictor of Search Satisfaction
  
  Ahmed Awadallah, Xiaolin Shi, Nick Craswell, Bill Ramsey
  
  ACM International Conference on Information and Knowledge Management (CIKM) | October 2013
  
  Publication
- Optimized Interleaving for Online Retrieval Evaluation
  
  Filip Radlinski, Nick Craswell
  
  ACM International Conference on Web Search and Data Mining (WSDM) | February 2013
  
  ACM International Conference on Web Search and Data Mining (WSDM)
  
  Best Paper Award
  
  Publication
- Robust Query Rewriting using Anchor Data
  
  Nick Craswell, Bodo von Billerbeck, Dennis Fetterly, Marc Najork
  
  6th ACM International Conference on Web Search and Data Mining (WSDM) | February 2013
  
  Publication
- Microsoft Research at TREC 2011 Web Track
  
  Bodo von Billerbeck, Nick Craswell, Dennis Fetterly, Marc Najork
  
  Proc. of the 20th Text Retrieval Conference (TREC) | November 2011
  
  Publication
- The Power of Peers
  
  Nick Craswell, Dennis Fetterly, Marc Najork
  
  33rd European Conference on IR Research (ECIR) | April 2011
  
  33rd European Conference on IR Research (ECIR)
  
  Publication
- LambdaMerge: Merging the Results of Query Reformulations
  
  Daniel Sheldon, Milad Shokouhi, Martin Szummer, Nick Craswell
  
  Conf. Web search and data mining (WSDM) | February 2011
  
  Publication Publication
- Microsoft Research at TREC 2010 Web Track
  
  Nick Craswell, Dennis Fetterly, Marc Najork
  
  Proc. of the 19th Text Retrieval Conference (TREC) | November 2010
  
  Proc. of the 19th Text Retrieval Conference (TREC)
  
  Publication
- Evaluating Search Systems Using Result Page Context
  
  Peter Bailey, Nick Craswell, Ryen W. White, Liwei Chen, Ashwin Satyanarayana, S. M. M. Tahaghoghi
  
  3rd Symposium on Information Interaction in Context (IIiX 2010), New Brunswick, NJ, USA | August 2010
  
  Best Paper Finalist
  
  Publication
- Evaluating Whole-Page Relevance
  
  Peter Bailey, Nick Craswell, Ryen W. White, Liwei Chen, Ashwin Satyanarayana, S.M.M. Tahaghoghi
  
  33rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2010), Geneva, Switzerland | July 2010
  
  Publication
- Comparing the Sensitivity of Information Retrieval Metrics
  
  Filip Radlinski, Nick Craswell
  
  Proceedings of SIGIR | July 2010
  
  Publication
- Metrics for Assessing Sets of Subtopics
  
  Filip Radlinski, Martin Szummer, Nick Craswell
  
  SIGIR Conf. Research and Development in Information Retrieval | July 2010
  
  Publication
- Inferring Query Intent from Reformulations and Clicks
  
  Filip Radlinski, Martin Szummer, Nick Craswell
  
  Proc. 19th Annual International World Wide Web Conference (WWW '10). | April 2010
  
  Proc. 19th Annual International World Wide Web Conference (WWW '10).
  
  Publication
- Microsoft Research at TREC 2009: Web and Relevance Feedback Tracks
  
  Nick Craswell, Dennis Fetterly, Marc Najork, Stephen Robertson, Emine Yilmaz
  
  Proc. of the 18th Text Retrieval Conference (TREC) | November 2009
  
  Publication
- The Impact of Crawl Policy on Web Search Effectiveness
  
  Dennis Fetterly, Nick Craswell, Vishwa Vinay
  
  Proceedings of the 32nd annual international ACM SIGIR conference on Research and development in information retrieval | July 2009
  
  Publication
- Measuring the Search Effectiveness of a Breadth-First Crawl
  
  Dennis Fetterly, Nick Craswell, Vishwa Vinay
  
  Proceedings of the 31st European Conference on Information Retrieval (ECIR) | April 2009
  
  Proceedings of the 31st European Conference on Information Retrieval (ECIR)
  
  Publication
- ACM Proceedings of the 2009 Workshop on Web Search Click Data
  
  Nick Craswell, Rosie Jones, Georges Dupret, Evelyne Viegas
  
  February 2009
  
  ISBN: 978-1-60558-434-8
  
  Publication
- Wikipedia Pages as Entry Points for Book Search
  
  Gabriella Kazai, Nick Craswell
  
  February 2009
  
  Proceedings of the Second ACM International Conference on Web Search and Data Mining (WSDM'09)
- A Two-Stage Model for Expert Search
  
  Yunbo Cao, Jingjing Liu, Shenghua Bao, Hang Li, Nick Craswell
  
  MSR-TR-2008-143 | October 2008
  
  Publication
- A Two-Stage Model for Expert Search
  
  Yunbo Cao, JJ (Jingjing) Liu, Shenghua Bao, Hang Li, Nick Craswell
  
  MSR-TR-2008-195 | October 2008
  
  Publication
- Efficient and Effective Link Analysis with Precomputed SALSA Maps
  
  Marc Najork, Nick Craswell
  
  17th ACM Conference on Information and Knowledge Management (CIKM) | October 2008
  
  17th ACM Conference on Information and Knowledge Management (CIKM)
  
  Publication
- A Decision Theoretic Framework for Ranking using Implicit Feedback
  
  Onno Zoeter, Mike Taylor, Ed Snelson, John Guiver, Nick Craswell, Martin Szummer
  
  SIGIR 2008 Workshop on Learning to Rank for Information Retrieval | July 2008
  
  Publication
- Behavioral Classification on the Click Graph
  
  Martin Szummer, Nick Craswell
  
  World Wide Web Conference | April 2008
  
  World Wide Web Conference
  
  Publication
- An experimental comparison of click position-bias models
  
  Nick Craswell, Mike Taylor
  
  WSDM '08: Proceedings of the international conference on Web search and web data mining | January 2008
  
  ISBN: 978-1-59593-927-9
  
  Publication Publication
- Overview of the INEX 2007 Entity Ranking Track
  
  Arjen P. de Vries, Anne-Marie Vercoustre, James A. Thom, Nick Craswell, Mounia Lalmas
  
  January 2008
  
  ISBN: 978-3-540-85901-7
  
  Publication Publication
- Relevance assessment: are judges exchangeable and does it matter
  
  Peter Bailey, Nick Craswell, Ian Soboroff, Paul Thomas, Arjen P. de Vries, Emine Yilmaz
  
  SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval | January 2008
  
  ISBN: 978-1-60558-164-4
  
  Publication Publication
- Random Walks on the Click Graph
  
  Nick Craswell, Martin Szummer
  
  Proceedings of SIGIR 2007 | July 2007
  
  ISBN: 978-1-59593-597-7
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_sigir07.pdf
  
  Publication Github
- Searching Documents Based on Relevance and Type
  
  Jun Xu, Yunbo Cao, Hang Li, Nick Craswell, Yalou Huang
  
  ECIR 2007 | January 2007
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/xu_ecir07.pdf
  
  Publication
- The CSIRO Enterprise Search Test Collection
  
  Peter Bailey, Nick Craswell, Ian Soboroff, Arjen P. de Vries
  
  SIGIR Forum | January 2007, Vol 41(2): pp. 42-45
  
  Publication
- Optimisation methods for ranking functions with multiple parameters
  
  Mike Taylor, Hugo Zaragoza, Nick Craswell, Stephen Robertson, Chris J.C. Burges
  
  Proceedings of CIKM 2006 | January 2006
  
  Proceedings of CIKM 2006
  
  ISBN: 1-59593-433-2
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/taylor_cikm06.pdf
  
  Publication
- Quality and Relevance of Domain-specific Search: A Case Study in Mental Health
  
  Thanh Tang, Nick Craswell, David Hawking, Kathleen Griffiths, Helen Christensen
  
  Information Retrieval | January 2006, pp. 207-225
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/tang_ir06.pdf
  
  Publication
- Focused crawling for both topical relevance and quality of medical information
  
  Thanh Tang, David Hawking, Nick Craswell, Kathleen Griffiths
  
  Proceedings of CIKM 2005 | January 2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/tang_cikm05.pdf
  
  Publication
- Microsoft Cambridge at TREC-14: Enterprise Track
  
  Nick Craswell, Hugo Zaragoza, Stephen Robertson
  
  Proceedings of TREC-2005 | January 2005
  
  Proceedings of TREC-2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec05.pdf
  
  Publication
- Relevance Weighting for Query Independent Evidence
  
  Nick Craswell, Stephen Robertson, Hugo Zaragoza, Mike Taylor
  
  Proceedings of ACM SIGIR 2005 | January 2005
  
  ISBN: 1-59593-034-5
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_sigir05.pdf
  
  Publication
- The TREC terabyte retrieval track
  
  Charles Clarke, Nick Craswell, Ian Soboroff
  
  SIGIR Forum | January 2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/clarke_sigirforum05.pdf
  
  Publication
- Microsoft Cambridge at TREC 2004: Web and HARD track
  
  Hugo Zaragoza, Nick Craswell, Mike Taylor, Suchi Saria, Stephen Robertson
  
  Proceedings of TREC-2004 | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/zaragoza_trec04.pdf
  
  Publication Publication
- Testbed for Information Extraction from Deep Web
  
  Yasuhiro Yamada, Nick Craswell, Tetsuya Nakatoh, Sachio Hirokawa
  
  WWW2004 Poster Proceedings | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/yamada_www2004poster.pdf
  
  Publication
- Toward Better Weighting of Anchors
  
  David Hawking, Trystan Upstill, Nick Craswell
  
  Proceedings of ACM SIGIR 2004 (poster abstract) | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_sigir04poster.pdf
  
  Publication
- Predicting Fame and Fortune: PageRank or Indegree?
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  Proceedings of the Australasian Document Computing Symposium, ADCS2003 | December 2003
  
  Proceedings of the Australasian Document Computing Symposium, ADCS2003
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_adcs03.pdf
  
  Publication
- Automated Discovery of Search Interfaces on the Web
  
  Jared Cope, Nick Craswell, David Hawking
  
  The Fourteenth Australasian Database Conference | January 2003
  
  The Fourteenth Australasian Database Conference
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/cope_adc03.pdf
  
  Publication
- Query-independent evidence in home page finding
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  January 2003
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_tois03.pdf
  
  Publication
- Overview of the TREC-2002 Web Track
  
  Nick Craswell, David Hawking
  
  Proceedings of TREC-2002 | November 2002
  
  Proceedings of TREC-2002
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec02wt.pdf
  
  Publication
- Buying bestsellers online: A case study in Search & Searchability
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  7th Australasian Document Computing Symposium | January 2002
  
  7th Australasian Document Computing Symposium
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_adcs02.pdf
  
  Publication
- Which Search Engine is Best at Finding Online Services?
  
  David Hawking, Nick Craswell, Kathleen Griffiths
  
  Poster Proceedings of WWW10 | May 2001
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_www01poster.pdf
  
  Publication
- TREC10 Web and Interactive Tracks at CSIRO
  
  Nick Craswell, David Hawking, Ross Wilkinson, Mingfang Wu
  
  Proceedings of TREC-2001 | November 2001
  
  Proceedings of TREC-2001
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec01.pdf
  
  Publication
- Methods for Distributed Information Retrieval
  
  Nick Craswell
  
  January 2000
  
  January 2000
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_thesis00.pdf
  
  Publication
- ACSys TREC-7 experiments
  
  David Hawking, Nick Craswell, Paul Thistlewaite
  
  TREC-7, NIST special publication 500-242 | January 1998
  
  TREC-7, NIST special publication 500-242
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_trec98.pdf
  
  Publication
- Aglets: A good idea for spidering?
  
  Jason Haines, Brendan Humphreys, Chris Johnson Nick Craswell, Paul Thistlewaite, Nick Craswell
  
  4th IDEA Workshop Proceedings | January 1997
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_idea97.pdf
  
  Publication

- Evaluation Validity in Information Retrieval
  
  Paul Thomas, Nick Craswell, Mark Sanderson, Seth Spielman, Robert Sim, Ryen W. White
  
  2026 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2026
  
  DOI
- All claims are equal, but some claims are more equal than others: Importance-sensitive factuality evaluation of LLM generations
  
  Miriam Wanner, Leif Azzopardi, Paul Thomas, Soham Dan, Ben Van Durme, Nick Craswell
  
  October 2025
  
  preprint arXiv:2510.07083
  
  Publication
- Judging the Judges: A Collection of LLM-Generated Relevance Judgements
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  February 2025
  
  Preprint
- JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment
  
  Hossein A. Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra
  
  2025 The Web Conference | January 2025
  
  Publication Preprint PDF
- SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval
  
  Hossein A. Rahmani, Xi Wang, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas
  
  2025 The Web Conference | January 2025
  
  Publication PDF Preprint
- Proceedings of The First Workshop on Large Language Models for Evaluation in Information Retrieval (LLM4Eval 2024)
  
  Clemencia Siro, Mohammad Aliannejadi, Hossein A. Rahmani, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  September 2024
  
  Publication
- Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  August 2024
  
  Publication Preprint PDF
- Large Language Models Can Accurately Predict Searcher Preferences
  
  Paul Thomas, Seth Spielman, Nick Craswell, Bhaskar Mitra
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  An earlier version of this paper appeared as arXiv preprint arXiv:2309.10621v1 [cs.IR].
  
  Publication PDF PDF
- Synthetic Test Collections for Retrieval Evaluation
  
  Hossein A. Rahmani, Nick Craswell, Emine Yilmaz, Bhaskar Mitra, Daniel Campos
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | May 2024
  
  Publication Preprint PDF
- Towards Group-aware Search Success
  
  Haolun Wu, Bhaskar Mitra, Nick Craswell
  
  International Conference on the Theory of Information Retrieval | April 2024
  
  Publication Preprint PDF
- Overview of the TREC 2023 Deep Learning Track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Hossein A. Rahmani, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
  
  Text REtrieval Conference (TREC) | February 2024
  
  Publication PDF Access
- Overview of the TREC 2022 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
  
  Text REtrieval Conference (TREC) | March 2023
  
  Publication PDF Access
- Are We There Yet? A Decision Framework for Replacing Term-Based Retrieval with Dense Retrieval Systems
  
  Sebastian Hofstätter, Nick Craswell, Bhaskar Mitra, Hamed Zamani, Allan Hanbury
  
  June 2022
  
  Preprint
- Overview of the TREC 2021 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Text REtrieval Conference (TREC) | May 2022
  
  Publication PDF PDF Access
- Fostering Coopetition While Plugging Leaks: The Design and Implementation of the MS MARCO Leaderboards
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF Access
- Summarize with Caution: Comparing Global Feature Attributions
  
  Alex Okeson, Rich Caruana, Nick Craswell, Kori Inkpen, Scott M. Lundberg, Harsha Nori, Hanna Wallach, Jennifer Wortman Vaughan
  
  Bulletin of the IEEE Computer Society Technical Committee on Data Engineering | December 2021
  
  Publication Project
- Less is Less: When Are Snippets Insufficient for Human vs Machine Relevance Estimation?
  
  Gabriella Kazai, Bhaskar Mitra, Anlei Dong, Nick Craswell, Linjun Yang
  
  European Conference on Information Retrieval | November 2021
  
  Publication Preprint PDF PDF
- Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
  
  Sebastian Hofstatter, Bhaskar Mitra, Hamed Zamani, Nick Craswell, Allan Hanbury
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
  
  Publication Preprint PDF PDF
- TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees, Ian Soboroff
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF
- MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | March 2021
  
  Publication Preprint PDF Access
- Overview of the TREC 2020 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos
  
  Text REtrieval Conference (TREC) | February 2021
  
  Publication Preprint PDF PDF Access
- Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Text REtrieval Conference (TREC) | November 2020
  
  Publication Preprint PDF PDF Access
- Conformer-Kernel with Query Term Independence for Document Retrieval
  
  Bhaskar Mitra, Sebastian Hofstätter, Hamed Zamani, Nick Craswell
  
  July 2020
  
  Preprint Github
- MIMICS: A Large-Scale Data Collection for Search Clarification
  
  Hamed Zamani, Gord Lueck, Everest Chen, Rodolfo Quispe, Flint Luu, Nick Craswell
  
  June 2020
- ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search
  
  Nick Craswell, Daniel Campos, Bhaskar Mitra, Emine Yilmaz, Bodo von Billerbeck
  
  Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | June 2020
  
  Publication Preprint PDF Access
- Analyzing and Learning from User Interactions for Search Clarification
  
  Hamed Zamani, Bhaskar Mitra, Everest Chen, Gord Lueck, Fernando Diaz, Paul Bennett, Nick Craswell, Susan Dumais
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF PDF
- Local Self-Attention over Long Text for Efficient Document Retrieval
  
  Sebastian Hofstätter, Hamed Zamani, Bhaskar Mitra, Nick Craswell, Allan Hanbury
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF
- Generating Clarifying Questions for Information Retrieval
  
  Hamed Zamani, Susan Dumais, Nick Craswell, Paul Bennett, Gord Lueck
  
  The Web Conference 2020 (formerly WWW conference) | April 2020
  
  Publication
- Leading Conversational Search by Suggesting Useful Questions
  
  Corby Rosset, Chenyan Xiong, Xia Song, Daniel Campos, Nick Craswell, Saurabh Tiwary, Paul Bennett
  
  The Web Conference 2020 (formerly WWW conference) | April 2020
  
  Publication Publication
- On the Reliability of Test Collections for Evaluating Systems of Different Types
  
  Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Daniel Campos
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | April 2020
  
  Publication Preprint PDF PDF Access
- Overview of the TREC 2019 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees
  
  Text REtrieval Conference (TREC) | March 2020
  
  Publication PDF PDF Access
- Duet at TREC 2019 Deep Learning Track
  
  Bhaskar Mitra, Nick Craswell
  
  Text REtrieval Conference (TREC) | December 2019
  
  Publication Preprint PDF PDF Access
- Generic Intent Representation in Web Search
  
  Hongfei Zhang, Xia Song, Chenyan Xiong, Corby Rosset, Paul Bennett, Nick Craswell, Saurabh Tiwary
  
  The 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019) | July 2019
  
  Publication Project Project
- Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
  
  Bhaskar Mitra, Corby Rosset, David Hawking, Nick Craswell, Fernando Diaz, Emine Yilmaz
  
  July 2019
  
  Preprint Project Project
- An Axiomatic Approach to Regularizing Neural Ranking Models
  
  Corby Rosset, Bhaskar Mitra, Chenyan Xiong, Nick Craswell, Xia Song, Saurabh Tiwary
  
  Proceedings of the 42nd international ACM SIGIR conference on Research & development in information retrieval | April 2019
  
  Publication Preprint PDF PDF Project Project
- An updated duet model for passage re-ranking
  
  Bhaskar Mitra, Nick Craswell
  
  March 2019
  
  Preprint
- An Introduction to Neural Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Foundations and Trends® in Information Retrieval | December 2018, Vol 13(1): pp. 1-126
  
  Publication PDF PDF
- Neural Ranking Models with Multiple Document Fields
  
  Hamed Zamani, Bhaskar Mitra, Xia Song, Nick Craswell, Saurabh Tiwary
  
  Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM'18) | November 2017
  
  Publication Preprint PDF Project Project
- Neural information retrieval: introduction to the special issue
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Information Retrieval Journal : Special issue Neural | November 2017
  
  Publication PDF
- Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rĳke, Jiafeng Guo, Bhaskar Mitra
  
  SIGIR Forum | October 2017, Vol 52
  
  Publication PDF PDF
- SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval | August 2017
  
  Publication PDF
- Luandri: a Clean Lua Interface to the Indri Search Engine
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 40th international ACM SIGIR conference on Research & development in information retrieval | August 2017
  
  Publication Preprint PDF PDF
- Neural Models for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  May 2017
  
  Preprint Video
- Learning to Match Using Local and Distributed Representations of Text for Web Search
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 26th International Conference on World Wide Web | April 2017
  
  Publication Preprint PDF PDF
- Neural Text Embeddings for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM'17) | February 2017
  
  Publication PDF
- Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rĳke, Bhaskar Mitra
  
  SIGIR Forum | December 2016, Vol 50(2): pp. 96-103
  
  Publication PDF
- Neu-IR: The SIGIR 2016 Workshop on Neural Information Retrieval
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2016
  
  Publication PDF
- A Proposal for Evaluating Answer Distillation from Web Data
  
  Bhaskar Mitra, Grady Simon, Jianfeng Gao, Nick Craswell, Li Deng
  
  Proceedings of the Second WebQA Workshop (SIGIR 2016) | July 2016
  
  PDF PDF
- Query Expansion with Locally-Trained Word Embeddings
  
  Fernando Diaz, Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics | June 2016
  
  Publication Preprint PDF PDF
- Improving Document Ranking with Dual Word Embeddings
  
  Eric Nalisnick, Bhaskar Mitra, Nick Craswell, Rich Caruana
  
  WWW'16 | April 2016
  
  Publication PDF PDF Project
- A Dual Embedding Space Model for Document Ranking
  
  Bhaskar Mitra, Eric Nalisnick, Nick Craswell, Rich Caruana
  
  February 2016
  
  This paper is an extended evaluation and analysis of the model proposed in a poster to appear in WWW'16, April 11 - 15, 2016, Montreal, Canada.
  
  Preprint PDF Access Project
- Query Auto-Completion for Rare Prefixes
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM) | October 2015
  
  Publication PDF PDF
- LambdaMerge: Merging the Results of Query Reformulations
  
  Daniel Sheldon, Milad Shokouhi, Martin Szummer, Nick Craswell
  
  Conf. Web search and data mining (WSDM) | February 2011
  
  Publication Publication
- A Decision Theoretic Framework for Ranking using Implicit Feedback
  
  Onno Zoeter, Mike Taylor, Ed Snelson, John Guiver, Nick Craswell, Martin Szummer
  
  SIGIR 2008 Workshop on Learning to Rank for Information Retrieval | July 2008
  
  Publication
- Random Walks on the Click Graph
  
  Nick Craswell, Martin Szummer
  
  Proceedings of SIGIR 2007 | July 2007
  
  ISBN: 978-1-59593-597-7
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_sigir07.pdf
  
  Publication Github
- Searching Documents Based on Relevance and Type
  
  Jun Xu, Yunbo Cao, Hang Li, Nick Craswell, Yalou Huang
  
  ECIR 2007 | January 2007
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/xu_ecir07.pdf
  
  Publication
- The CSIRO Enterprise Search Test Collection
  
  Peter Bailey, Nick Craswell, Ian Soboroff, Arjen P. de Vries
  
  SIGIR Forum | January 2007, Vol 41(2): pp. 42-45
  
  Publication
- Quality and Relevance of Domain-specific Search: A Case Study in Mental Health
  
  Thanh Tang, Nick Craswell, David Hawking, Kathleen Griffiths, Helen Christensen
  
  Information Retrieval | January 2006, pp. 207-225
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/tang_ir06.pdf
  
  Publication
- Query-independent evidence in home page finding
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  January 2003
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_tois03.pdf
  
  Publication
- Evaluation Validity in Information Retrieval
  
  Paul Thomas, Nick Craswell, Mark Sanderson, Seth Spielman, Robert Sim, Ryen W. White
  
  2026 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2026
  
  DOI
- Zero-shot Clarifying Question Generation for Conversational Search
  
  Zhenduo Wang, Yuancheng Tu, Nick Craswell, Corby Rosset, Ming Wu, Qingyao Ai
  
  2023 The Web Conference | March 2023
  
  Publication
- Neural Approaches to Conversational Information Retrieval
  
  Jianfeng Gao, Chenyan Xiong, Paul Bennett, Nick Craswell
  
  Published by Springer | March 2023, Vol 44
  
  ISBN: 978-3-031-23079-0
  
  The Information Retrieval Series.
  
  Preprint
- Report on the future conversations workshop at CHIIR 2021
  
  Damiano Spina, Johanne R Trippas, Paul Thomas, Hideo Joho, Katriina Byström, Leigh Clark, Nick Craswell, Mary Czerwinski, David Elsweiler, Alexander Frummet, Souvick Ghosh, Johannes Kiesel, Irene Lopatovska, Daniel McDuff, Selina Meyer, Ahmed Mourad, Paul Owoicho, Sachin Pathiyan Cherumanal, Daniel Russell, Laurianne Sitbon
  
  SIGIR Forum | June 2021, Vol 55(1): pp. 1-22
  
  Publication PDF
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  International Workshop on Conversational Approaches to Information Retrieval | March 2020
  
  An extended version of this paper also appears as Thomas et al., ACM Trans. Info. Sys. 39(2), 2021: https://dl.acm.org/doi/pdf/10.1145/3439869.
  
  Publication
- A Two-Stage Model for Expert Search
  
  Yunbo Cao, JJ (Jingjing) Liu, Shenghua Bao, Hang Li, Nick Craswell
  
  MSR-TR-2008-195 | October 2008
  
  Publication
- Which Search Engine is Best at Finding Online Services?
  
  David Hawking, Nick Craswell, Kathleen Griffiths
  
  Poster Proceedings of WWW10 | May 2001
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_www01poster.pdf
  
  Publication
- Evaluation Validity in Information Retrieval
  
  Paul Thomas, Nick Craswell, Mark Sanderson, Seth Spielman, Robert Sim, Ryen W. White
  
  2026 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2026
  
  DOI
- Report from the 4th Strategic Workshop on Information Retrieval in Lorne (SWIRL 2025)
  
  Johanne R Trippas, J. Shane Culpepper, Mohammad Aliannejadi, James Allan, Enrique Amigó, Jaime Arguello, Leif Azzopardi, Peter Bailey, Jamie Callan, Rob Capra, Nick Craswell, Bruce Croft, Jeff Dalton, Gianluca Demartini, Laura Dietz, Zhicheng Dou, Carsten Eickhoff, Michael Ekstrand, Nicola Ferro, Norbert Fuhr, Dorota Glowacka, Faegheh Hasibi, Danula Hettiachchi, Rosie Jones, Jaap Kamps, Noriko Kando, Sarvnaz Karimi, Makoto P. Kato, Bevan Koopman, Yiqun Liu, Chenglong Ma, Joel Mackenzie, Maria Maistro, Jiaxin Mao, Dana McKay, Bhaskar Mitra, Stefano Mizzaro, Alistair Moffat, Josiane Mothe, Iadh Ounis, Lida Rashidi, Yongli Ren, Mark Sanderson, Rodrygo Santos, Falk Scholer, Chirag A Shah, Laurianne Sitbon, Ian Soboroff, Damiano Spina, Paul Thomas, Julián Urbano, Arjen P. de Vries, Ryen W. White, Abby Yuan, Hamed Zamani, Oleg Zendel, Min Zhang, Shengyao Zhuang, Justin Zobel, Guido Zuccon
  
  ACM SIGIR Forum | June 2025, Vol 59(1): pp. 1-68
  
  DOI PDF
- LLM4Eval@WSDM 2025: Large Language Model for Evaluation in Information Retrieval
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L A Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  2025 Web Search and Data Mining | March 2025
  
  Publication
- Judging the Judges: A Collection of LLM-Generated Relevance Judgements
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  February 2025
  
  Preprint
- JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment
  
  Hossein A. Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra
  
  2025 The Web Conference | January 2025
  
  Publication Preprint PDF
- SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval
  
  Hossein A. Rahmani, Xi Wang, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas
  
  2025 The Web Conference | January 2025
  
  Publication PDF Preprint
- Proceedings of The First Workshop on Large Language Models for Evaluation in Information Retrieval (LLM4Eval 2024)
  
  Clemencia Siro, Mohammad Aliannejadi, Hossein A. Rahmani, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  September 2024
  
  Publication
- LLMJudge: LLMs for Relevance Judgments
  
  Hossein A Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas, Charles L A Clarke, Mohammad Aliannejadi, Clemencia Siro, Guglielmo Faggioli
  
  August 2024
  
  Publication Preprint
- Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  August 2024
  
  Publication Preprint PDF
- LLM4Eval: Large Language Model for Evaluation in IR
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  Workshop description
  
  Publication PDF
- What Matters in a Measure? A Perspective from Large-Scale Search Evaluation
  
  Paul Thomas, Gabriella Kazai, Nick Craswell, Seth Spielman
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  Best paper nomination
  
  DOI PDF
- Large Language Models Can Accurately Predict Searcher Preferences
  
  Paul Thomas, Seth Spielman, Nick Craswell, Bhaskar Mitra
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  An earlier version of this paper appeared as arXiv preprint arXiv:2309.10621v1 [cs.IR].
  
  Publication PDF PDF
- Synthetic Test Collections for Retrieval Evaluation
  
  Hossein A. Rahmani, Nick Craswell, Emine Yilmaz, Bhaskar Mitra, Daniel Campos
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | May 2024
  
  Publication Preprint PDF
- MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels
  
  Qi Chen, Xiubo Geng, Corby Rosset, Carolyn Buractaon, Jingwen Lu, Tao Shen, Kun Zhou, Chenyan Xiong, Yeyun Gong, Paul Bennett, Nick Craswell, Xing Xie, Fan Yang, Bryan Tower, Zheng Liu, Mingqin Li, Chuanjie Liu, Jason (Zengzhong) Li, Rangan Majumder, Jennifer Neville, Harsha Simhadri, Manik Varma, Yujing Wang, Linjun Yang, Mao Yang, Ce Zhang
  
  2024 The Web Conference | May 2024
  
  Publication PDF Access
- Towards Group-aware Search Success
  
  Haolun Wu, Bhaskar Mitra, Nick Craswell
  
  International Conference on the Theory of Information Retrieval | April 2024
  
  Publication Preprint PDF
- Overview of the TREC 2023 Deep Learning Track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Hossein A. Rahmani, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
  
  Text REtrieval Conference (TREC) | February 2024
  
  Publication PDF Access
- Patterns of gender-specializing query reformulation
  
  Amifa Raj, Bhaskar Mitra, Nick Craswell, Michael D. Ekstrand
  
  2023 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2023
  
  Publication Preprint PDF PDF
- Overview of the TREC 2022 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
  
  Text REtrieval Conference (TREC) | March 2023
  
  Publication PDF Access
- Zero-shot Clarifying Question Generation for Conversational Search
  
  Zhenduo Wang, Yuancheng Tu, Nick Craswell, Corby Rosset, Ming Wu, Qingyao Ai
  
  2023 The Web Conference | March 2023
  
  Publication
- Neural Approaches to Conversational Information Retrieval
  
  Jianfeng Gao, Chenyan Xiong, Paul Bennett, Nick Craswell
  
  Published by Springer | March 2023, Vol 44
  
  ISBN: 978-3-031-23079-0
  
  The Information Retrieval Series.
  
  Preprint
- Are We There Yet? A Decision Framework for Replacing Term-Based Retrieval with Dense Retrieval Systems
  
  Sebastian Hofstätter, Nick Craswell, Bhaskar Mitra, Hamed Zamani, Allan Hanbury
  
  June 2022
  
  Preprint
- Overview of the TREC 2021 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Text REtrieval Conference (TREC) | May 2022
  
  Publication PDF PDF Access
- Fostering Coopetition While Plugging Leaks: The Design and Implementation of the MS MARCO Leaderboards
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF Access
- The crowd is made of people: Observations from large-scale crowd labelling
  
  Paul Thomas, Gabriella Kazai, Ryen W. White, Nick Craswell
  
  ACM SIGIR Conference on Human Information Interaction and Retrieval | March 2022
  
  Publication DOI
- Less is Less: When Are Snippets Insufficient for Human vs Machine Relevance Estimation?
  
  Gabriella Kazai, Bhaskar Mitra, Anlei Dong, Nick Craswell, Linjun Yang
  
  European Conference on Information Retrieval | November 2021
  
  Publication Preprint PDF PDF
- Analysing mixed initiatives and search strategies during conversational search
  
  Mohammad Aliannejadi, Leif Azzopardi, Hamed Zamani, Evangelos Kanoulas, Paul Thomas, Nick Craswell
  
  ACM International Conference on Information and Knowledge Management | November 2021
  
  DOI
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  ACM Transactions on Information Systems | August 2021, Vol 39(4)
  
  Article 39
  
  Publication
- Do affective cues validate behavioural metrics for search?
  
  Daniel McDuff, Paul Thomas, Kael Rowan, Nick Craswell, Mary Czerwinski
  
  SIGIR 2021 | July 2021
  
  PDF DOI
- Report on the future conversations workshop at CHIIR 2021
  
  Damiano Spina, Johanne R Trippas, Paul Thomas, Hideo Joho, Katriina Byström, Leigh Clark, Nick Craswell, Mary Czerwinski, David Elsweiler, Alexander Frummet, Souvick Ghosh, Johannes Kiesel, Irene Lopatovska, Daniel McDuff, Selina Meyer, Ahmed Mourad, Paul Owoicho, Sachin Pathiyan Cherumanal, Daniel Russell, Laurianne Sitbon
  
  SIGIR Forum | June 2021, Vol 55(1): pp. 1-22
  
  Publication PDF
- Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
  
  Sebastian Hofstatter, Bhaskar Mitra, Hamed Zamani, Nick Craswell, Allan Hanbury
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
  
  Publication Preprint PDF PDF
- TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees, Ian Soboroff
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF
- MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- Social competence in conversational retrieval
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  Future Conversations | March 2021
  
  Publication
- Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | March 2021
  
  Publication Preprint PDF Access
- Overview of the TREC 2020 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos
  
  Text REtrieval Conference (TREC) | February 2021
  
  Publication Preprint PDF PDF Access
- Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Text REtrieval Conference (TREC) | November 2020
  
  Publication Preprint PDF PDF Access
- Simulating information retrieval test collections
  
  David Hawking, Bodo von Billerbeck, Paul Thomas, Nick Craswell
  
  Published by Morgan & Claypool | September 2020
  
  Editor(s): Gary Marchionini
  
  ISBN: 9781681739571
  
  Synthesis Lectures on Information Concepts, Retrieval, and Services
  
  Publication DOI
- Expressions of style in information-seeking conversation with an embodied agent
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2020
  
  PDF DOI
- Conformer-Kernel with Query Term Independence for Document Retrieval
  
  Bhaskar Mitra, Sebastian Hofstätter, Hamed Zamani, Nick Craswell
  
  July 2020
  
  Preprint Github
- MIMICS: A Large-Scale Data Collection for Search Clarification
  
  Hamed Zamani, Gord Lueck, Everest Chen, Rodolfo Quispe, Flint Luu, Nick Craswell
  
  June 2020
- ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search
  
  Nick Craswell, Daniel Campos, Bhaskar Mitra, Emine Yilmaz, Bodo von Billerbeck
  
  Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | June 2020
  
  Publication Preprint PDF Access
- Analyzing and Learning from User Interactions for Search Clarification
  
  Hamed Zamani, Bhaskar Mitra, Everest Chen, Gord Lueck, Fernando Diaz, Paul Bennett, Nick Craswell, Susan Dumais
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF PDF
- Local Self-Attention over Long Text for Efficient Document Retrieval
  
  Sebastian Hofstätter, Hamed Zamani, Bhaskar Mitra, Nick Craswell, Allan Hanbury
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF
- Generating Clarifying Questions for Information Retrieval
  
  Hamed Zamani, Susan Dumais, Nick Craswell, Paul Bennett, Gord Lueck
  
  The Web Conference 2020 (formerly WWW conference) | April 2020
  
  Publication
- Leading Conversational Search by Suggesting Useful Questions
  
  Corby Rosset, Chenyan Xiong, Xia Song, Daniel Campos, Nick Craswell, Saurabh Tiwary, Paul Bennett
  
  The Web Conference 2020 (formerly WWW conference) | April 2020
  
  Publication Publication
- On the Reliability of Test Collections for Evaluating Systems of Different Types
  
  Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Daniel Campos
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | April 2020
  
  Publication Preprint PDF PDF Access
- Overview of the TREC 2019 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees
  
  Text REtrieval Conference (TREC) | March 2020
  
  Publication PDF PDF Access
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  International Workshop on Conversational Approaches to Information Retrieval | March 2020
  
  An extended version of this paper also appears as Thomas et al., ACM Trans. Info. Sys. 39(2), 2021: https://dl.acm.org/doi/pdf/10.1145/3439869.
  
  Publication
- Data-driven evaluation metrics for heterogeneous search engine result pages
  
  Leif Azzopardi, Ryen W. White, Paul Thomas, Nick Craswell
  
  Proceedings of the Conference on Human Information Interaction and Retrieval | March 2020
  
  DOI Publication PDF
- Duet at TREC 2019 Deep Learning Track
  
  Bhaskar Mitra, Nick Craswell
  
  Text REtrieval Conference (TREC) | December 2019
  
  Publication Preprint PDF PDF Access
- Investigating searchers’ mental models to inform search explanations
  
  Paul Thomas, Bodo von Billerbeck, Nick Craswell, Ryen W. White
  
  ACM Transactions on Information Systems | December 2019, Vol 38(1)
  
  Article 10
  
  DOI Publication
- The Emotion Profile of Web Search
  
  Gabriella Kazai, Paul Thomas, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2019
  
  Publication
- Generic Intent Representation in Web Search
  
  Hongfei Zhang, Xia Song, Chenyan Xiong, Corby Rosset, Paul Bennett, Nick Craswell, Saurabh Tiwary
  
  The 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019) | July 2019
  
  Publication Project Project
- Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
  
  Bhaskar Mitra, Corby Rosset, David Hawking, Nick Craswell, Fernando Diaz, Emine Yilmaz
  
  July 2019
  
  Preprint Project Project
- An Axiomatic Approach to Regularizing Neural Ranking Models
  
  Corby Rosset, Bhaskar Mitra, Chenyan Xiong, Nick Craswell, Xia Song, Saurabh Tiwary
  
  Proceedings of the 42nd international ACM SIGIR conference on Research & development in information retrieval | April 2019
  
  Publication Preprint PDF PDF Project Project
- An updated duet model for passage re-ranking
  
  Bhaskar Mitra, Nick Craswell
  
  March 2019
  
  Preprint
- An Introduction to Neural Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Foundations and Trends® in Information Retrieval | December 2018, Vol 13(1): pp. 1-126
  
  Publication PDF PDF
- Better effectiveness metrics for SERPs, cards, and rankings
  
  Paul Thomas, Alistair Moffat, Peter Bailey, Falk Scholer, Nick Craswell
  
  Australasian Document Computing Symposium | December 2018
  
  Best paper award
  
  DOI Publication
- Measuring the Utility of Search Engine Result Pages: An Information Foraging Based Measure
  
  Leif Azzopardi, Paul Thomas, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2018
  
  Publication Publication Publication
- Style and alignment in information-seeking conversation
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell, Gloria Mark
  
  Proceedings of the Conference on Computer-Human Information Interaction and Retrieval | March 2018
  
  Publication Publication
- Neural Ranking Models with Multiple Document Fields
  
  Hamed Zamani, Bhaskar Mitra, Xia Song, Nick Craswell, Saurabh Tiwary
  
  Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM'18) | November 2017
  
  Publication Preprint PDF Project Project
- Multimodal analysis of vocal collaborative search: A public corpus and results
  
  Daniel McDuff, Paul Thomas, Mary Czerwinski, Nick Craswell
  
  Proceedings of the ACM International Conference on Multimodal Interaction | November 2017
  
  Publication Publication
- Neural information retrieval: introduction to the special issue
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Information Retrieval Journal : Special issue Neural | November 2017
  
  Publication PDF
- Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rĳke, Jiafeng Guo, Bhaskar Mitra
  
  SIGIR Forum | October 2017, Vol 52
  
  Publication PDF PDF
- SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval | August 2017
  
  Publication PDF
- MISC: A data set of information-seeking conversations
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the 1st International Workshop on Conversational Approaches to Information Retrieval | August 2017
  
  Publication
- Luandri: a Clean Lua Interface to the Indri Search Engine
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 40th international ACM SIGIR conference on Research & development in information retrieval | August 2017
  
  Publication Preprint PDF PDF
- Neural Models for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  May 2017
  
  Preprint Video
- Learning to Match Using Local and Distributed Representations of Text for Web Search
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 26th International Conference on World Wide Web | April 2017
  
  Publication Preprint PDF PDF
- A Theoretical Framework for Conversational Search
  
  Filip Radlinski, Nick Craswell
  
  CHIIR 2017 | March 2017
  
  Publication
- Neural Text Embeddings for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM'17) | February 2017
  
  Publication PDF
- Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rĳke, Bhaskar Mitra
  
  SIGIR Forum | December 2016, Vol 50(2): pp. 96-103
  
  Publication PDF
- Neu-IR: The SIGIR 2016 Workshop on Neural Information Retrieval
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2016
  
  Publication PDF
- Ingrams – a neuropsychological explanation for why people search
  
  Peter Bailey, Nick Craswell
  
  Proceedings of the 39th Annual ACM SIGIR Conference (SIGIR 2016) | July 2016
  
  Publication Publication
- A Proposal for Evaluating Answer Distillation from Web Data
  
  Bhaskar Mitra, Grady Simon, Jianfeng Gao, Nick Craswell, Li Deng
  
  Proceedings of the Second WebQA Workshop (SIGIR 2016) | July 2016
  
  PDF PDF
- Query Expansion with Locally-Trained Word Embeddings
  
  Fernando Diaz, Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics | June 2016
  
  Publication Preprint PDF PDF
- Explicit In Situ User Feedback for Web Search Results
  
  Jin Young Kim, Jaime Teevan, Nick Craswell
  
  Proceedings of the 39th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2016), Pisa, Italy | June 2016
  
  Publication
- Improving Document Ranking with Dual Word Embeddings
  
  Eric Nalisnick, Bhaskar Mitra, Nick Craswell, Rich Caruana
  
  WWW'16 | April 2016
  
  Publication PDF PDF Project
- Did You Say U2 or Youtube? Inferring Implicit Transcripts from Voice Search Logs
  
  Nick Craswell, Umut Ozertem, Milad Shokouhi
  
  Proceedings of the 25th International World Wide Web Conference (WWW2016) | April 2016
  
  Publication
- A Dual Embedding Space Model for Document Ranking
  
  Bhaskar Mitra, Eric Nalisnick, Nick Craswell, Rich Caruana
  
  February 2016
  
  This paper is an extended evaluation and analysis of the model proposed in a poster to appear in WWW'16, April 11 - 15, 2016, Montreal, Canada.
  
  Preprint PDF Access Project
- Query Auto-Completion for Rare Prefixes
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM) | October 2015
  
  Publication PDF PDF
- Relevance and Effort: An Analysis of Document Utility
  
  Emine Yilmaz, Manisha Verma, Nick Craswell, Filip Radlinski, Peter Bailey
  
  Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
  
  Publication
- Beyond Clicks: Query Reformulation as a Predictor of Search Satisfaction
  
  Ahmed Awadallah, Xiaolin Shi, Nick Craswell, Bill Ramsey
  
  ACM International Conference on Information and Knowledge Management (CIKM) | October 2013
  
  Publication
- Optimized Interleaving for Online Retrieval Evaluation
  
  Filip Radlinski, Nick Craswell
  
  ACM International Conference on Web Search and Data Mining (WSDM) | February 2013
  
  ACM International Conference on Web Search and Data Mining (WSDM)
  
  Best Paper Award
  
  Publication
- Robust Query Rewriting using Anchor Data
  
  Nick Craswell, Bodo von Billerbeck, Dennis Fetterly, Marc Najork
  
  6th ACM International Conference on Web Search and Data Mining (WSDM) | February 2013
  
  Publication
- Microsoft Research at TREC 2011 Web Track
  
  Bodo von Billerbeck, Nick Craswell, Dennis Fetterly, Marc Najork
  
  Proc. of the 20th Text Retrieval Conference (TREC) | November 2011
  
  Publication
- The Power of Peers
  
  Nick Craswell, Dennis Fetterly, Marc Najork
  
  33rd European Conference on IR Research (ECIR) | April 2011
  
  33rd European Conference on IR Research (ECIR)
  
  Publication
- LambdaMerge: Merging the Results of Query Reformulations
  
  Daniel Sheldon, Milad Shokouhi, Martin Szummer, Nick Craswell
  
  Conf. Web search and data mining (WSDM) | February 2011
  
  Publication Publication
- Microsoft Research at TREC 2010 Web Track
  
  Nick Craswell, Dennis Fetterly, Marc Najork
  
  Proc. of the 19th Text Retrieval Conference (TREC) | November 2010
  
  Proc. of the 19th Text Retrieval Conference (TREC)
  
  Publication
- Evaluating Search Systems Using Result Page Context
  
  Peter Bailey, Nick Craswell, Ryen W. White, Liwei Chen, Ashwin Satyanarayana, S. M. M. Tahaghoghi
  
  3rd Symposium on Information Interaction in Context (IIiX 2010), New Brunswick, NJ, USA | August 2010
  
  Best Paper Finalist
  
  Publication
- Evaluating Whole-Page Relevance
  
  Peter Bailey, Nick Craswell, Ryen W. White, Liwei Chen, Ashwin Satyanarayana, S.M.M. Tahaghoghi
  
  33rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2010), Geneva, Switzerland | July 2010
  
  Publication
- Metrics for Assessing Sets of Subtopics
  
  Filip Radlinski, Martin Szummer, Nick Craswell
  
  SIGIR Conf. Research and Development in Information Retrieval | July 2010
  
  Publication
- Comparing the Sensitivity of Information Retrieval Metrics
  
  Filip Radlinski, Nick Craswell
  
  Proceedings of SIGIR | July 2010
  
  Publication
- Inferring Query Intent from Reformulations and Clicks
  
  Filip Radlinski, Martin Szummer, Nick Craswell
  
  Proc. 19th Annual International World Wide Web Conference (WWW '10). | April 2010
  
  Proc. 19th Annual International World Wide Web Conference (WWW '10).
  
  Publication
- Microsoft Research at TREC 2009: Web and Relevance Feedback Tracks
  
  Nick Craswell, Dennis Fetterly, Marc Najork, Stephen Robertson, Emine Yilmaz
  
  Proc. of the 18th Text Retrieval Conference (TREC) | November 2009
  
  Publication
- Measuring the Search Effectiveness of a Breadth-First Crawl
  
  Dennis Fetterly, Nick Craswell, Vishwa Vinay
  
  Proceedings of the 31st European Conference on Information Retrieval (ECIR) | April 2009
  
  Proceedings of the 31st European Conference on Information Retrieval (ECIR)
  
  Publication
- ACM Proceedings of the 2009 Workshop on Web Search Click Data
  
  Nick Craswell, Rosie Jones, Georges Dupret, Evelyne Viegas
  
  February 2009
  
  ISBN: 978-1-60558-434-8
  
  Publication
- Wikipedia Pages as Entry Points for Book Search
  
  Gabriella Kazai, Nick Craswell
  
  February 2009
  
  Proceedings of the Second ACM International Conference on Web Search and Data Mining (WSDM'09)
- Efficient and Effective Link Analysis with Precomputed SALSA Maps
  
  Marc Najork, Nick Craswell
  
  17th ACM Conference on Information and Knowledge Management (CIKM) | October 2008
  
  17th ACM Conference on Information and Knowledge Management (CIKM)
  
  Publication
- A Decision Theoretic Framework for Ranking using Implicit Feedback
  
  Onno Zoeter, Mike Taylor, Ed Snelson, John Guiver, Nick Craswell, Martin Szummer
  
  SIGIR 2008 Workshop on Learning to Rank for Information Retrieval | July 2008
  
  Publication
- Relevance assessment: are judges exchangeable and does it matter
  
  Peter Bailey, Nick Craswell, Ian Soboroff, Paul Thomas, Arjen P. de Vries, Emine Yilmaz
  
  SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval | January 2008
  
  ISBN: 978-1-60558-164-4
  
  Publication Publication
- Overview of the INEX 2007 Entity Ranking Track
  
  Arjen P. de Vries, Anne-Marie Vercoustre, James A. Thom, Nick Craswell, Mounia Lalmas
  
  January 2008
  
  ISBN: 978-3-540-85901-7
  
  Publication Publication
- An experimental comparison of click position-bias models
  
  Nick Craswell, Mike Taylor
  
  WSDM '08: Proceedings of the international conference on Web search and web data mining | January 2008
  
  ISBN: 978-1-59593-927-9
  
  Publication Publication
- Relevance Weighting for Query Independent Evidence
  
  Nick Craswell, Stephen Robertson, Hugo Zaragoza, Mike Taylor
  
  Proceedings of ACM SIGIR 2005 | January 2005
  
  ISBN: 1-59593-034-5
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_sigir05.pdf
  
  Publication
- Predicting Fame and Fortune: PageRank or Indegree?
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  Proceedings of the Australasian Document Computing Symposium, ADCS2003 | December 2003
  
  Proceedings of the Australasian Document Computing Symposium, ADCS2003
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_adcs03.pdf
  
  Publication
- Automated Discovery of Search Interfaces on the Web
  
  Jared Cope, Nick Craswell, David Hawking
  
  The Fourteenth Australasian Database Conference | January 2003
  
  The Fourteenth Australasian Database Conference
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/cope_adc03.pdf
  
  Publication
- Overview of the TREC-2002 Web Track
  
  Nick Craswell, David Hawking
  
  Proceedings of TREC-2002 | November 2002
  
  Proceedings of TREC-2002
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec02wt.pdf
  
  Publication
- Buying bestsellers online: A case study in Search & Searchability
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  7th Australasian Document Computing Symposium | January 2002
  
  7th Australasian Document Computing Symposium
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_adcs02.pdf
  
  Publication
- ACSys TREC-7 experiments
  
  David Hawking, Nick Craswell, Paul Thistlewaite
  
  TREC-7, NIST special publication 500-242 | January 1998
  
  TREC-7, NIST special publication 500-242
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_trec98.pdf
  
  Publication
- MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels
  
  Qi Chen, Xiubo Geng, Corby Rosset, Carolyn Buractaon, Jingwen Lu, Tao Shen, Kun Zhou, Chenyan Xiong, Yeyun Gong, Paul Bennett, Nick Craswell, Xing Xie, Fan Yang, Bryan Tower, Zheng Liu, Mingqin Li, Chuanjie Liu, Jason (Zengzhong) Li, Rangan Majumder, Jennifer Neville, Harsha Simhadri, Manik Varma, Yujing Wang, Linjun Yang, Mao Yang, Ce Zhang
  
  2024 The Web Conference | May 2024
  
  Publication PDF Access
- Focused crawling for both topical relevance and quality of medical information
  
  Thanh Tang, David Hawking, Nick Craswell, Kathleen Griffiths
  
  Proceedings of CIKM 2005 | January 2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/tang_cikm05.pdf
  
  Publication
- Patterns of gender-specializing query reformulation
  
  Amifa Raj, Bhaskar Mitra, Nick Craswell, Michael D. Ekstrand
  
  2023 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2023
  
  Publication Preprint PDF PDF
- The crowd is made of people: Observations from large-scale crowd labelling
  
  Paul Thomas, Gabriella Kazai, Ryen W. White, Nick Craswell
  
  ACM SIGIR Conference on Human Information Interaction and Retrieval | March 2022
  
  Publication DOI
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  ACM Transactions on Information Systems | August 2021, Vol 39(4)
  
  Article 39
  
  Publication
- Report on the future conversations workshop at CHIIR 2021
  
  Damiano Spina, Johanne R Trippas, Paul Thomas, Hideo Joho, Katriina Byström, Leigh Clark, Nick Craswell, Mary Czerwinski, David Elsweiler, Alexander Frummet, Souvick Ghosh, Johannes Kiesel, Irene Lopatovska, Daniel McDuff, Selina Meyer, Ahmed Mourad, Paul Owoicho, Sachin Pathiyan Cherumanal, Daniel Russell, Laurianne Sitbon
  
  SIGIR Forum | June 2021, Vol 55(1): pp. 1-22
  
  Publication PDF
- Social competence in conversational retrieval
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  Future Conversations | March 2021
  
  Publication
- Expressions of style in information-seeking conversation with an embodied agent
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2020
  
  PDF DOI
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  International Workshop on Conversational Approaches to Information Retrieval | March 2020
  
  An extended version of this paper also appears as Thomas et al., ACM Trans. Info. Sys. 39(2), 2021: https://dl.acm.org/doi/pdf/10.1145/3439869.
  
  Publication
- Style and alignment in information-seeking conversation
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell, Gloria Mark
  
  Proceedings of the Conference on Computer-Human Information Interaction and Retrieval | March 2018
  
  Publication Publication
- Multimodal analysis of vocal collaborative search: A public corpus and results
  
  Daniel McDuff, Paul Thomas, Mary Czerwinski, Nick Craswell
  
  Proceedings of the ACM International Conference on Multimodal Interaction | November 2017
  
  Publication Publication
- MISC: A data set of information-seeking conversations
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the 1st International Workshop on Conversational Approaches to Information Retrieval | August 2017
  
  Publication
- Query Auto-Completion for Rare Prefixes
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM) | October 2015
  
  Publication PDF PDF
- The Impact of Crawl Policy on Web Search Effectiveness
  
  Dennis Fetterly, Nick Craswell, Vishwa Vinay
  
  Proceedings of the 32nd annual international ACM SIGIR conference on Research and development in information retrieval | July 2009
  
  Publication
- The TREC terabyte retrieval track
  
  Charles Clarke, Nick Craswell, Ian Soboroff
  
  SIGIR Forum | January 2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/clarke_sigirforum05.pdf
  
  Publication
- Toward Better Weighting of Anchors
  
  David Hawking, Trystan Upstill, Nick Craswell
  
  Proceedings of ACM SIGIR 2004 (poster abstract) | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_sigir04poster.pdf
  
  Publication
- TREC10 Web and Interactive Tracks at CSIRO
  
  Nick Craswell, David Hawking, Ross Wilkinson, Mingfang Wu
  
  Proceedings of TREC-2001 | November 2001
  
  Proceedings of TREC-2001
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec01.pdf
  
  Publication
- Methods for Distributed Information Retrieval
  
  Nick Craswell
  
  January 2000
  
  January 2000
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_thesis00.pdf
  
  Publication
- Aglets: A good idea for spidering?
  
  Jason Haines, Brendan Humphreys, Chris Johnson Nick Craswell, Paul Thistlewaite, Nick Craswell
  
  4th IDEA Workshop Proceedings | January 1997
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_idea97.pdf
  
  Publication
- A Two-Stage Model for Expert Search
  
  Yunbo Cao, Jingjing Liu, Shenghua Bao, Hang Li, Nick Craswell
  
  MSR-TR-2008-143 | October 2008
  
  Publication
- Testbed for Information Extraction from Deep Web
  
  Yasuhiro Yamada, Nick Craswell, Tetsuya Nakatoh, Sachio Hirokawa
  
  WWW2004 Poster Proceedings | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/yamada_www2004poster.pdf
  
  Publication
- Aglets: A good idea for spidering?
  
  Jason Haines, Brendan Humphreys, Chris Johnson Nick Craswell, Paul Thistlewaite, Nick Craswell
  
  4th IDEA Workshop Proceedings | January 1997
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_idea97.pdf
  
  Publication
- Behavioral Classification on the Click Graph
  
  Martin Szummer, Nick Craswell
  
  World Wide Web Conference | April 2008
  
  World Wide Web Conference
  
  Publication
- Optimisation methods for ranking functions with multiple parameters
  
  Mike Taylor, Hugo Zaragoza, Nick Craswell, Stephen Robertson, Chris J.C. Burges
  
  Proceedings of CIKM 2006 | January 2006
  
  Proceedings of CIKM 2006
  
  ISBN: 1-59593-433-2
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/taylor_cikm06.pdf
  
  Publication
- Microsoft Cambridge at TREC-14: Enterprise Track
  
  Nick Craswell, Hugo Zaragoza, Stephen Robertson
  
  Proceedings of TREC-2005 | January 2005
  
  Proceedings of TREC-2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec05.pdf
  
  Publication
- Microsoft Cambridge at TREC 2004: Web and HARD track
  
  Hugo Zaragoza, Nick Craswell, Mike Taylor, Suchi Saria, Stephen Robertson
  
  Proceedings of TREC-2004 | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/zaragoza_trec04.pdf
  
  Publication Publication

- Which Search Engine is Best at Finding Online Services?
  
  David Hawking, Nick Craswell, Kathleen Griffiths
  
  Poster Proceedings of WWW10 | May 2001
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_www01poster.pdf
  
  Publication
- ACSys TREC-7 experiments
  
  David Hawking, Nick Craswell, Paul Thistlewaite
  
  TREC-7, NIST special publication 500-242 | January 1998
  
  TREC-7, NIST special publication 500-242
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_trec98.pdf
  
  Publication
- Aglets: A good idea for spidering?
  
  Jason Haines, Brendan Humphreys, Chris Johnson Nick Craswell, Paul Thistlewaite, Nick Craswell
  
  4th IDEA Workshop Proceedings | January 1997
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_idea97.pdf
  
  Publication
- Evaluation Validity in Information Retrieval
  
  Paul Thomas, Nick Craswell, Mark Sanderson, Seth Spielman, Robert Sim, Ryen W. White
  
  2026 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2026
  
  DOI
- LLM4Eval@WSDM 2025: Large Language Model for Evaluation in Information Retrieval
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L A Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  2025 Web Search and Data Mining | March 2025
  
  Publication
- JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment
  
  Hossein A. Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra
  
  2025 The Web Conference | January 2025
  
  Publication Preprint PDF
- SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval
  
  Hossein A. Rahmani, Xi Wang, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas
  
  2025 The Web Conference | January 2025
  
  Publication PDF Preprint
- LLM4Eval: Large Language Model for Evaluation in IR
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  Workshop description
  
  Publication PDF
- What Matters in a Measure? A Perspective from Large-Scale Search Evaluation
  
  Paul Thomas, Gabriella Kazai, Nick Craswell, Seth Spielman
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  Best paper nomination
  
  DOI PDF
- Large Language Models Can Accurately Predict Searcher Preferences
  
  Paul Thomas, Seth Spielman, Nick Craswell, Bhaskar Mitra
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
  
  An earlier version of this paper appeared as arXiv preprint arXiv:2309.10621v1 [cs.IR].
  
  Publication PDF PDF
- Synthetic Test Collections for Retrieval Evaluation
  
  Hossein A. Rahmani, Nick Craswell, Emine Yilmaz, Bhaskar Mitra, Daniel Campos
  
  2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | May 2024
  
  Publication Preprint PDF
- MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels
  
  Qi Chen, Xiubo Geng, Corby Rosset, Carolyn Buractaon, Jingwen Lu, Tao Shen, Kun Zhou, Chenyan Xiong, Yeyun Gong, Paul Bennett, Nick Craswell, Xing Xie, Fan Yang, Bryan Tower, Zheng Liu, Mingqin Li, Chuanjie Liu, Jason (Zengzhong) Li, Rangan Majumder, Jennifer Neville, Harsha Simhadri, Manik Varma, Yujing Wang, Linjun Yang, Mao Yang, Ce Zhang
  
  2024 The Web Conference | May 2024
  
  Publication PDF Access
- Towards Group-aware Search Success
  
  Haolun Wu, Bhaskar Mitra, Nick Craswell
  
  International Conference on the Theory of Information Retrieval | April 2024
  
  Publication Preprint PDF
- Overview of the TREC 2023 Deep Learning Track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Hossein A. Rahmani, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
  
  Text REtrieval Conference (TREC) | February 2024
  
  Publication PDF Access
- Patterns of gender-specializing query reformulation
  
  Amifa Raj, Bhaskar Mitra, Nick Craswell, Michael D. Ekstrand
  
  2023 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2023
  
  Publication Preprint PDF PDF
- Overview of the TREC 2022 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
  
  Text REtrieval Conference (TREC) | March 2023
  
  Publication PDF Access
- Zero-shot Clarifying Question Generation for Conversational Search
  
  Zhenduo Wang, Yuancheng Tu, Nick Craswell, Corby Rosset, Ming Wu, Qingyao Ai
  
  2023 The Web Conference | March 2023
  
  Publication
- Overview of the TREC 2021 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Text REtrieval Conference (TREC) | May 2022
  
  Publication PDF PDF Access
- Fostering Coopetition While Plugging Leaks: The Design and Implementation of the MS MARCO Leaderboards
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
  
  Publication PDF PDF Access
- The crowd is made of people: Observations from large-scale crowd labelling
  
  Paul Thomas, Gabriella Kazai, Ryen W. White, Nick Craswell
  
  ACM SIGIR Conference on Human Information Interaction and Retrieval | March 2022
  
  Publication DOI
- Less is Less: When Are Snippets Insufficient for Human vs Machine Relevance Estimation?
  
  Gabriella Kazai, Bhaskar Mitra, Anlei Dong, Nick Craswell, Linjun Yang
  
  European Conference on Information Retrieval | November 2021
  
  Publication Preprint PDF PDF
- Analysing mixed initiatives and search strategies during conversational search
  
  Mohammad Aliannejadi, Leif Azzopardi, Hamed Zamani, Evangelos Kanoulas, Paul Thomas, Nick Craswell
  
  ACM International Conference on Information and Knowledge Management | November 2021
  
  DOI
- Do affective cues validate behavioural metrics for search?
  
  Daniel McDuff, Paul Thomas, Kael Rowan, Nick Craswell, Mary Czerwinski
  
  SIGIR 2021 | July 2021
  
  PDF DOI
- Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
  
  Sebastian Hofstatter, Bhaskar Mitra, Hamed Zamani, Nick Craswell, Allan Hanbury
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
  
  Publication Preprint PDF PDF
- TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees, Ian Soboroff
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF
- MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
  
  Publication Preprint PDF PDF Access
- Social competence in conversational retrieval
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  Future Conversations | March 2021
  
  Publication
- Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard
  
  Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
  
  Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | March 2021
  
  Publication Preprint PDF Access
- Overview of the TREC 2020 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos
  
  Text REtrieval Conference (TREC) | February 2021
  
  Publication Preprint PDF PDF Access
- Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track
  
  Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
  
  Text REtrieval Conference (TREC) | November 2020
  
  Publication Preprint PDF PDF Access
- Expressions of style in information-seeking conversation with an embodied agent
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2020
  
  PDF DOI
- ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search
  
  Nick Craswell, Daniel Campos, Bhaskar Mitra, Emine Yilmaz, Bodo von Billerbeck
  
  Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | June 2020
  
  Publication Preprint PDF Access
- Analyzing and Learning from User Interactions for Search Clarification
  
  Hamed Zamani, Bhaskar Mitra, Everest Chen, Gord Lueck, Fernando Diaz, Paul Bennett, Nick Craswell, Susan Dumais
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF PDF
- Local Self-Attention over Long Text for Efficient Document Retrieval
  
  Sebastian Hofstätter, Hamed Zamani, Bhaskar Mitra, Nick Craswell, Allan Hanbury
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
  
  Publication Preprint PDF
- Generating Clarifying Questions for Information Retrieval
  
  Hamed Zamani, Susan Dumais, Nick Craswell, Paul Bennett, Gord Lueck
  
  The Web Conference 2020 (formerly WWW conference) | April 2020
  
  Publication
- Leading Conversational Search by Suggesting Useful Questions
  
  Corby Rosset, Chenyan Xiong, Xia Song, Daniel Campos, Nick Craswell, Saurabh Tiwary, Paul Bennett
  
  The Web Conference 2020 (formerly WWW conference) | April 2020
  
  Publication Publication
- On the Reliability of Test Collections for Evaluating Systems of Different Types
  
  Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Daniel Campos
  
  Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | April 2020
  
  Publication Preprint PDF PDF Access
- Overview of the TREC 2019 deep learning track
  
  Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees
  
  Text REtrieval Conference (TREC) | March 2020
  
  Publication PDF PDF Access
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  International Workshop on Conversational Approaches to Information Retrieval | March 2020
  
  An extended version of this paper also appears as Thomas et al., ACM Trans. Info. Sys. 39(2), 2021: https://dl.acm.org/doi/pdf/10.1145/3439869.
  
  Publication
- Data-driven evaluation metrics for heterogeneous search engine result pages
  
  Leif Azzopardi, Ryen W. White, Paul Thomas, Nick Craswell
  
  Proceedings of the Conference on Human Information Interaction and Retrieval | March 2020
  
  DOI Publication PDF
- Duet at TREC 2019 Deep Learning Track
  
  Bhaskar Mitra, Nick Craswell
  
  Text REtrieval Conference (TREC) | December 2019
  
  Publication Preprint PDF PDF Access
- The Emotion Profile of Web Search
  
  Gabriella Kazai, Paul Thomas, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2019
  
  Publication
- Generic Intent Representation in Web Search
  
  Hongfei Zhang, Xia Song, Chenyan Xiong, Corby Rosset, Paul Bennett, Nick Craswell, Saurabh Tiwary
  
  The 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019) | July 2019
  
  Publication Project Project
- An Axiomatic Approach to Regularizing Neural Ranking Models
  
  Corby Rosset, Bhaskar Mitra, Chenyan Xiong, Nick Craswell, Xia Song, Saurabh Tiwary
  
  Proceedings of the 42nd international ACM SIGIR conference on Research & development in information retrieval | April 2019
  
  Publication Preprint PDF PDF Project Project
- Better effectiveness metrics for SERPs, cards, and rankings
  
  Paul Thomas, Alistair Moffat, Peter Bailey, Falk Scholer, Nick Craswell
  
  Australasian Document Computing Symposium | December 2018
  
  Best paper award
  
  DOI Publication
- Measuring the Utility of Search Engine Result Pages: An Information Foraging Based Measure
  
  Leif Azzopardi, Paul Thomas, Nick Craswell
  
  Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) | July 2018
  
  Publication Publication Publication
- Style and alignment in information-seeking conversation
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell, Gloria Mark
  
  Proceedings of the Conference on Computer-Human Information Interaction and Retrieval | March 2018
  
  Publication Publication
- Neural Ranking Models with Multiple Document Fields
  
  Hamed Zamani, Bhaskar Mitra, Xia Song, Nick Craswell, Saurabh Tiwary
  
  Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM'18) | November 2017
  
  Publication Preprint PDF Project Project
- Multimodal analysis of vocal collaborative search: A public corpus and results
  
  Daniel McDuff, Paul Thomas, Mary Czerwinski, Nick Craswell
  
  Proceedings of the ACM International Conference on Multimodal Interaction | November 2017
  
  Publication Publication
- SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval | August 2017
  
  Publication PDF
- MISC: A data set of information-seeking conversations
  
  Paul Thomas, Daniel McDuff, Mary Czerwinski, Nick Craswell
  
  Proceedings of the 1st International Workshop on Conversational Approaches to Information Retrieval | August 2017
  
  Publication
- Luandri: a Clean Lua Interface to the Indri Search Engine
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 40th international ACM SIGIR conference on Research & development in information retrieval | August 2017
  
  Publication Preprint PDF PDF
- Learning to Match Using Local and Distributed Representations of Text for Web Search
  
  Bhaskar Mitra, Fernando Diaz, Nick Craswell
  
  Proceedings of the 26th International Conference on World Wide Web | April 2017
  
  Publication Preprint PDF PDF
- A Theoretical Framework for Conversational Search
  
  Filip Radlinski, Nick Craswell
  
  CHIIR 2017 | March 2017
  
  Publication
- Neural Text Embeddings for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM'17) | February 2017
  
  Publication PDF
- Neu-IR: The SIGIR 2016 Workshop on Neural Information Retrieval
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
  
  Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2016
  
  Publication PDF
- Ingrams – a neuropsychological explanation for why people search
  
  Peter Bailey, Nick Craswell
  
  Proceedings of the 39th Annual ACM SIGIR Conference (SIGIR 2016) | July 2016
  
  Publication Publication
- A Proposal for Evaluating Answer Distillation from Web Data
  
  Bhaskar Mitra, Grady Simon, Jianfeng Gao, Nick Craswell, Li Deng
  
  Proceedings of the Second WebQA Workshop (SIGIR 2016) | July 2016
  
  PDF PDF
- Query Expansion with Locally-Trained Word Embeddings
  
  Fernando Diaz, Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics | June 2016
  
  Publication Preprint PDF PDF
- Explicit In Situ User Feedback for Web Search Results
  
  Jin Young Kim, Jaime Teevan, Nick Craswell
  
  Proceedings of the 39th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2016), Pisa, Italy | June 2016
  
  Publication
- Improving Document Ranking with Dual Word Embeddings
  
  Eric Nalisnick, Bhaskar Mitra, Nick Craswell, Rich Caruana
  
  WWW'16 | April 2016
  
  Publication PDF PDF Project
- Did You Say U2 or Youtube? Inferring Implicit Transcripts from Voice Search Logs
  
  Nick Craswell, Umut Ozertem, Milad Shokouhi
  
  Proceedings of the 25th International World Wide Web Conference (WWW2016) | April 2016
  
  Publication
- Query Auto-Completion for Rare Prefixes
  
  Bhaskar Mitra, Nick Craswell
  
  Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM) | October 2015
  
  Publication PDF PDF
- Relevance and Effort: An Analysis of Document Utility
  
  Emine Yilmaz, Manisha Verma, Nick Craswell, Filip Radlinski, Peter Bailey
  
  Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
  
  Publication
- Beyond Clicks: Query Reformulation as a Predictor of Search Satisfaction
  
  Ahmed Awadallah, Xiaolin Shi, Nick Craswell, Bill Ramsey
  
  ACM International Conference on Information and Knowledge Management (CIKM) | October 2013
  
  Publication
- Optimized Interleaving for Online Retrieval Evaluation
  
  Filip Radlinski, Nick Craswell
  
  ACM International Conference on Web Search and Data Mining (WSDM) | February 2013
  
  ACM International Conference on Web Search and Data Mining (WSDM)
  
  Best Paper Award
  
  Publication
- Robust Query Rewriting using Anchor Data
  
  Nick Craswell, Bodo von Billerbeck, Dennis Fetterly, Marc Najork
  
  6th ACM International Conference on Web Search and Data Mining (WSDM) | February 2013
  
  Publication
- Microsoft Research at TREC 2011 Web Track
  
  Bodo von Billerbeck, Nick Craswell, Dennis Fetterly, Marc Najork
  
  Proc. of the 20th Text Retrieval Conference (TREC) | November 2011
  
  Publication
- The Power of Peers
  
  Nick Craswell, Dennis Fetterly, Marc Najork
  
  33rd European Conference on IR Research (ECIR) | April 2011
  
  33rd European Conference on IR Research (ECIR)
  
  Publication
- LambdaMerge: Merging the Results of Query Reformulations
  
  Daniel Sheldon, Milad Shokouhi, Martin Szummer, Nick Craswell
  
  Conf. Web search and data mining (WSDM) | February 2011
  
  Publication Publication
- Microsoft Research at TREC 2010 Web Track
  
  Nick Craswell, Dennis Fetterly, Marc Najork
  
  Proc. of the 19th Text Retrieval Conference (TREC) | November 2010
  
  Proc. of the 19th Text Retrieval Conference (TREC)
  
  Publication
- Evaluating Search Systems Using Result Page Context
  
  Peter Bailey, Nick Craswell, Ryen W. White, Liwei Chen, Ashwin Satyanarayana, S. M. M. Tahaghoghi
  
  3rd Symposium on Information Interaction in Context (IIiX 2010), New Brunswick, NJ, USA | August 2010
  
  Best Paper Finalist
  
  Publication
- Evaluating Whole-Page Relevance
  
  Peter Bailey, Nick Craswell, Ryen W. White, Liwei Chen, Ashwin Satyanarayana, S.M.M. Tahaghoghi
  
  33rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2010), Geneva, Switzerland | July 2010
  
  Publication
- Metrics for Assessing Sets of Subtopics
  
  Filip Radlinski, Martin Szummer, Nick Craswell
  
  SIGIR Conf. Research and Development in Information Retrieval | July 2010
  
  Publication
- Comparing the Sensitivity of Information Retrieval Metrics
  
  Filip Radlinski, Nick Craswell
  
  Proceedings of SIGIR | July 2010
  
  Publication
- Inferring Query Intent from Reformulations and Clicks
  
  Filip Radlinski, Martin Szummer, Nick Craswell
  
  Proc. 19th Annual International World Wide Web Conference (WWW '10). | April 2010
  
  Proc. 19th Annual International World Wide Web Conference (WWW '10).
  
  Publication
- Microsoft Research at TREC 2009: Web and Relevance Feedback Tracks
  
  Nick Craswell, Dennis Fetterly, Marc Najork, Stephen Robertson, Emine Yilmaz
  
  Proc. of the 18th Text Retrieval Conference (TREC) | November 2009
  
  Publication
- The Impact of Crawl Policy on Web Search Effectiveness
  
  Dennis Fetterly, Nick Craswell, Vishwa Vinay
  
  Proceedings of the 32nd annual international ACM SIGIR conference on Research and development in information retrieval | July 2009
  
  Publication
- Measuring the Search Effectiveness of a Breadth-First Crawl
  
  Dennis Fetterly, Nick Craswell, Vishwa Vinay
  
  Proceedings of the 31st European Conference on Information Retrieval (ECIR) | April 2009
  
  Proceedings of the 31st European Conference on Information Retrieval (ECIR)
  
  Publication
- ACM Proceedings of the 2009 Workshop on Web Search Click Data
  
  Nick Craswell, Rosie Jones, Georges Dupret, Evelyne Viegas
  
  February 2009
  
  ISBN: 978-1-60558-434-8
  
  Publication
- Wikipedia Pages as Entry Points for Book Search
  
  Gabriella Kazai, Nick Craswell
  
  February 2009
  
  Proceedings of the Second ACM International Conference on Web Search and Data Mining (WSDM'09)
- Efficient and Effective Link Analysis with Precomputed SALSA Maps
  
  Marc Najork, Nick Craswell
  
  17th ACM Conference on Information and Knowledge Management (CIKM) | October 2008
  
  17th ACM Conference on Information and Knowledge Management (CIKM)
  
  Publication
- A Decision Theoretic Framework for Ranking using Implicit Feedback
  
  Onno Zoeter, Mike Taylor, Ed Snelson, John Guiver, Nick Craswell, Martin Szummer
  
  SIGIR 2008 Workshop on Learning to Rank for Information Retrieval | July 2008
  
  Publication
- Behavioral Classification on the Click Graph
  
  Martin Szummer, Nick Craswell
  
  World Wide Web Conference | April 2008
  
  World Wide Web Conference
  
  Publication
- Relevance assessment: are judges exchangeable and does it matter
  
  Peter Bailey, Nick Craswell, Ian Soboroff, Paul Thomas, Arjen P. de Vries, Emine Yilmaz
  
  SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval | January 2008
  
  ISBN: 978-1-60558-164-4
  
  Publication Publication
- Overview of the INEX 2007 Entity Ranking Track
  
  Arjen P. de Vries, Anne-Marie Vercoustre, James A. Thom, Nick Craswell, Mounia Lalmas
  
  January 2008
  
  ISBN: 978-3-540-85901-7
  
  Publication Publication
- An experimental comparison of click position-bias models
  
  Nick Craswell, Mike Taylor
  
  WSDM '08: Proceedings of the international conference on Web search and web data mining | January 2008
  
  ISBN: 978-1-59593-927-9
  
  Publication Publication
- Random Walks on the Click Graph
  
  Nick Craswell, Martin Szummer
  
  Proceedings of SIGIR 2007 | July 2007
  
  ISBN: 978-1-59593-597-7
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_sigir07.pdf
  
  Publication Github
- Searching Documents Based on Relevance and Type
  
  Jun Xu, Yunbo Cao, Hang Li, Nick Craswell, Yalou Huang
  
  ECIR 2007 | January 2007
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/xu_ecir07.pdf
  
  Publication
- Optimisation methods for ranking functions with multiple parameters
  
  Mike Taylor, Hugo Zaragoza, Nick Craswell, Stephen Robertson, Chris J.C. Burges
  
  Proceedings of CIKM 2006 | January 2006
  
  Proceedings of CIKM 2006
  
  ISBN: 1-59593-433-2
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/taylor_cikm06.pdf
  
  Publication
- Relevance Weighting for Query Independent Evidence
  
  Nick Craswell, Stephen Robertson, Hugo Zaragoza, Mike Taylor
  
  Proceedings of ACM SIGIR 2005 | January 2005
  
  ISBN: 1-59593-034-5
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_sigir05.pdf
  
  Publication
- Focused crawling for both topical relevance and quality of medical information
  
  Thanh Tang, David Hawking, Nick Craswell, Kathleen Griffiths
  
  Proceedings of CIKM 2005 | January 2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/tang_cikm05.pdf
  
  Publication
- Microsoft Cambridge at TREC-14: Enterprise Track
  
  Nick Craswell, Hugo Zaragoza, Stephen Robertson
  
  Proceedings of TREC-2005 | January 2005
  
  Proceedings of TREC-2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec05.pdf
  
  Publication
- The TREC terabyte retrieval track
  
  Charles Clarke, Nick Craswell, Ian Soboroff
  
  SIGIR Forum | January 2005
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/clarke_sigirforum05.pdf
  
  Publication
- Microsoft Cambridge at TREC 2004: Web and HARD track
  
  Hugo Zaragoza, Nick Craswell, Mike Taylor, Suchi Saria, Stephen Robertson
  
  Proceedings of TREC-2004 | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/zaragoza_trec04.pdf
  
  Publication Publication
- Testbed for Information Extraction from Deep Web
  
  Yasuhiro Yamada, Nick Craswell, Tetsuya Nakatoh, Sachio Hirokawa
  
  WWW2004 Poster Proceedings | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/yamada_www2004poster.pdf
  
  Publication
- Toward Better Weighting of Anchors
  
  David Hawking, Trystan Upstill, Nick Craswell
  
  Proceedings of ACM SIGIR 2004 (poster abstract) | January 2004
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/hawking_sigir04poster.pdf
  
  Publication
- Predicting Fame and Fortune: PageRank or Indegree?
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  Proceedings of the Australasian Document Computing Symposium, ADCS2003 | December 2003
  
  Proceedings of the Australasian Document Computing Symposium, ADCS2003
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_adcs03.pdf
  
  Publication
- Query-independent evidence in home page finding
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  January 2003
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_tois03.pdf
  
  Publication
- Automated Discovery of Search Interfaces on the Web
  
  Jared Cope, Nick Craswell, David Hawking
  
  The Fourteenth Australasian Database Conference | January 2003
  
  The Fourteenth Australasian Database Conference
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/cope_adc03.pdf
  
  Publication
- Overview of the TREC-2002 Web Track
  
  Nick Craswell, David Hawking
  
  Proceedings of TREC-2002 | November 2002
  
  Proceedings of TREC-2002
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec02wt.pdf
  
  Publication
- Buying bestsellers online: A case study in Search & Searchability
  
  Trystan Upstill, Nick Craswell, David Hawking
  
  7th Australasian Document Computing Symposium | January 2002
  
  7th Australasian Document Computing Symposium
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/upstill_adcs02.pdf
  
  Publication
- TREC10 Web and Interactive Tracks at CSIRO
  
  Nick Craswell, David Hawking, Ross Wilkinson, Mingfang Wu
  
  Proceedings of TREC-2001 | November 2001
  
  Proceedings of TREC-2001
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_trec01.pdf
  
  Publication
- All claims are equal, but some claims are more equal than others: Importance-sensitive factuality evaluation of LLM generations
  
  Miriam Wanner, Leif Azzopardi, Paul Thomas, Soham Dan, Ben Van Durme, Nick Craswell
  
  October 2025
  
  preprint arXiv:2510.07083
  
  Publication
- Proceedings of The First Workshop on Large Language Models for Evaluation in Information Retrieval (LLM4Eval 2024)
  
  Clemencia Siro, Mohammad Aliannejadi, Hossein A. Rahmani, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  September 2024
  
  Publication
- LLMJudge: LLMs for Relevance Judgments
  
  Hossein A Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas, Charles L A Clarke, Mohammad Aliannejadi, Clemencia Siro, Guglielmo Faggioli
  
  August 2024
  
  Publication Preprint
- Neural Models for Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  May 2017
  
  Preprint Video
- A Dual Embedding Space Model for Document Ranking
  
  Bhaskar Mitra, Eric Nalisnick, Nick Craswell, Rich Caruana
  
  February 2016
  
  This paper is an extended evaluation and analysis of the model proposed in a poster to appear in WWW'16, April 11 - 15, 2016, Montreal, Canada.
  
  Preprint PDF Access Project
- Report from the 4th Strategic Workshop on Information Retrieval in Lorne (SWIRL 2025)
  
  Johanne R Trippas, J. Shane Culpepper, Mohammad Aliannejadi, James Allan, Enrique Amigó, Jaime Arguello, Leif Azzopardi, Peter Bailey, Jamie Callan, Rob Capra, Nick Craswell, Bruce Croft, Jeff Dalton, Gianluca Demartini, Laura Dietz, Zhicheng Dou, Carsten Eickhoff, Michael Ekstrand, Nicola Ferro, Norbert Fuhr, Dorota Glowacka, Faegheh Hasibi, Danula Hettiachchi, Rosie Jones, Jaap Kamps, Noriko Kando, Sarvnaz Karimi, Makoto P. Kato, Bevan Koopman, Yiqun Liu, Chenglong Ma, Joel Mackenzie, Maria Maistro, Jiaxin Mao, Dana McKay, Bhaskar Mitra, Stefano Mizzaro, Alistair Moffat, Josiane Mothe, Iadh Ounis, Lida Rashidi, Yongli Ren, Mark Sanderson, Rodrygo Santos, Falk Scholer, Chirag A Shah, Laurianne Sitbon, Ian Soboroff, Damiano Spina, Paul Thomas, Julián Urbano, Arjen P. de Vries, Ryen W. White, Abby Yuan, Hamed Zamani, Oleg Zendel, Min Zhang, Shengyao Zhuang, Justin Zobel, Guido Zuccon
  
  ACM SIGIR Forum | June 2025, Vol 59(1): pp. 1-68
  
  DOI PDF
- Summarize with Caution: Comparing Global Feature Attributions
  
  Alex Okeson, Rich Caruana, Nick Craswell, Kori Inkpen, Scott M. Lundberg, Harsha Nori, Hanna Wallach, Jennifer Wortman Vaughan
  
  Bulletin of the IEEE Computer Society Technical Committee on Data Engineering | December 2021
  
  Publication Project
- Theories of conversation for conversational IR
  
  Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell
  
  ACM Transactions on Information Systems | August 2021, Vol 39(4)
  
  Article 39
  
  Publication
- Report on the future conversations workshop at CHIIR 2021
  
  Damiano Spina, Johanne R Trippas, Paul Thomas, Hideo Joho, Katriina Byström, Leigh Clark, Nick Craswell, Mary Czerwinski, David Elsweiler, Alexander Frummet, Souvick Ghosh, Johannes Kiesel, Irene Lopatovska, Daniel McDuff, Selina Meyer, Ahmed Mourad, Paul Owoicho, Sachin Pathiyan Cherumanal, Daniel Russell, Laurianne Sitbon
  
  SIGIR Forum | June 2021, Vol 55(1): pp. 1-22
  
  Publication PDF
- Investigating searchers’ mental models to inform search explanations
  
  Paul Thomas, Bodo von Billerbeck, Nick Craswell, Ryen W. White
  
  ACM Transactions on Information Systems | December 2019, Vol 38(1)
  
  Article 10
  
  DOI Publication
- An Introduction to Neural Information Retrieval
  
  Bhaskar Mitra, Nick Craswell
  
  Foundations and Trends® in Information Retrieval | December 2018, Vol 13(1): pp. 1-126
  
  Publication PDF PDF
- Neural information retrieval: introduction to the special issue
  
  Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
  
  Information Retrieval Journal : Special issue Neural | November 2017
  
  Publication PDF
- Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR’17)
  
  Nick Craswell, W. Bruce Croft, Maarten de Rĳke, Jiafeng Guo, Bhaskar Mitra
  
  SIGIR Forum | October 2017, Vol 52
  
  Publication PDF PDF
- Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)
  
  Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rĳke, Bhaskar Mitra
  
  SIGIR Forum | December 2016, Vol 50(2): pp. 96-103
  
  Publication PDF
- The CSIRO Enterprise Search Test Collection
  
  Peter Bailey, Nick Craswell, Ian Soboroff, Arjen P. de Vries
  
  SIGIR Forum | January 2007, Vol 41(2): pp. 42-45
  
  Publication
- Quality and Relevance of Domain-specific Search: A Case Study in Mental Health
  
  Thanh Tang, Nick Craswell, David Hawking, Kathleen Griffiths, Helen Christensen
  
  Information Retrieval | January 2006, pp. 207-225
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/tang_ir06.pdf
  
  Publication
- Judging the Judges: A Collection of LLM-Generated Relevance Judgements
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  February 2025
  
  Preprint
- Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024
  
  Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
  
  August 2024
  
  Publication Preprint PDF
- Are We There Yet? A Decision Framework for Replacing Term-Based Retrieval with Dense Retrieval Systems
  
  Sebastian Hofstätter, Nick Craswell, Bhaskar Mitra, Hamed Zamani, Allan Hanbury
  
  June 2022
  
  Preprint
- Conformer-Kernel with Query Term Independence for Document Retrieval
  
  Bhaskar Mitra, Sebastian Hofstätter, Hamed Zamani, Nick Craswell
  
  July 2020
  
  Preprint Github
- MIMICS: A Large-Scale Data Collection for Search Clarification
  
  Hamed Zamani, Gord Lueck, Everest Chen, Rodolfo Quispe, Flint Luu, Nick Craswell
  
  June 2020
- Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
  
  Bhaskar Mitra, Corby Rosset, David Hawking, Nick Craswell, Fernando Diaz, Emine Yilmaz
  
  July 2019
  
  Preprint Project Project
- An updated duet model for passage re-ranking
  
  Bhaskar Mitra, Nick Craswell
  
  March 2019
  
  Preprint
- Neural Approaches to Conversational Information Retrieval
  
  Jianfeng Gao, Chenyan Xiong, Paul Bennett, Nick Craswell
  
  Published by Springer | March 2023, Vol 44
  
  ISBN: 978-3-031-23079-0
  
  The Information Retrieval Series.
  
  Preprint
- Simulating information retrieval test collections
  
  David Hawking, Bodo von Billerbeck, Paul Thomas, Nick Craswell
  
  Published by Morgan & Claypool | September 2020
  
  Editor(s): Gary Marchionini
  
  ISBN: 9781681739571
  
  Synthesis Lectures on Information Concepts, Retrieval, and Services
  
  Publication DOI
- A Two-Stage Model for Expert Search
  
  Yunbo Cao, Jingjing Liu, Shenghua Bao, Hang Li, Nick Craswell
  
  MSR-TR-2008-143 | October 2008
  
  Publication
- A Two-Stage Model for Expert Search
  
  Yunbo Cao, JJ (Jingjing) Liu, Shenghua Bao, Hang Li, Nick Craswell
  
  MSR-TR-2008-195 | October 2008
  
  Publication
- Methods for Distributed Information Retrieval
  
  Nick Craswell
  
  January 2000
  
  January 2000
  
  \urlhttp://research.microsoft.com/users/nickcr/pubs/craswell_thesis00.pdf
  
  Publication

Nick Craswell

Publications

Evaluation Validity in Information Retrieval

All claims are equal, but some claims are more equal than others: Importance-sensitive factuality evaluation of LLM generations

Report from the 4th Strategic Workshop on Information Retrieval in Lorne (SWIRL 2025)

LLM4Eval@WSDM 2025: Large Language Model for Evaluation in Information Retrieval

Judging the Judges: A Collection of LLM-Generated Relevance Judgements

JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment

SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval

Proceedings of The First Workshop on Large Language Models for Evaluation in Information Retrieval (LLM4Eval 2024)

LLMJudge: LLMs for Relevance Judgments

Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024

Large Language Models Can Accurately Predict Searcher Preferences

LLM4Eval: Large Language Model for Evaluation in IR

What Matters in a Measure? A Perspective from Large-Scale Search Evaluation

Synthetic Test Collections for Retrieval Evaluation

MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels

Towards Group-aware Search Success

Overview of the TREC 2023 Deep Learning Track

Patterns of gender-specializing query reformulation

Overview of the TREC 2022 deep learning track

Neural Approaches to Conversational Information Retrieval

Zero-shot Clarifying Question Generation for Conversational Search

Are We There Yet? A Decision Framework for Replacing Term-Based Retrieval with Dense Retrieval Systems

Overview of the TREC 2021 deep learning track

Fostering Coopetition While Plugging Leaks: The Design and Implementation of the MS MARCO Leaderboards

The crowd is made of people: Observations from large-scale crowd labelling

Summarize with Caution: Comparing Global Feature Attributions

Less is Less: When Are Snippets Insufficient for Human vs Machine Relevance Estimation?

Analysing mixed initiatives and search strategies during conversational search

Theories of conversation for conversational IR

Do affective cues validate behavioural metrics for search?

Report on the future conversations workshop at CHIIR 2021

Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking

Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence

MS MARCO: Benchmarking Ranking Models in the Large-Data Regime

TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime

Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard

Social competence in conversational retrieval

Overview of the TREC 2020 deep learning track

Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track

Simulating information retrieval test collections

Conformer-Kernel with Query Term Independence for Document Retrieval

Expressions of style in information-seeking conversation with an embodied agent

MIMICS: A Large-Scale Data Collection for Search Clarification

ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search

Analyzing and Learning from User Interactions for Search Clarification

Local Self-Attention over Long Text for Efficient Document Retrieval

Generating Clarifying Questions for Information Retrieval

Leading Conversational Search by Suggesting Useful Questions

On the Reliability of Test Collections for Evaluating Systems of Different Types

Data-driven evaluation metrics for heterogeneous search engine result pages

Overview of the TREC 2019 deep learning track

Theories of conversation for conversational IR

Duet at TREC 2019 Deep Learning Track

Investigating searchers’ mental models to inform search explanations

Generic Intent Representation in Web Search

The Emotion Profile of Web Search

Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks

An Axiomatic Approach to Regularizing Neural Ranking Models

An updated duet model for passage re-ranking

An Introduction to Neural Information Retrieval

Better effectiveness metrics for SERPs, cards, and rankings

Measuring the Utility of Search Engine Result Pages: An Information Foraging Based Measure

Style and alignment in information-seeking conversation

Neural Ranking Models with Multiple Document Fields

Multimodal analysis of vocal collaborative search: A public corpus and results

Neural information retrieval: introduction to the special issue

Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR’17)

MISC: A data set of information-seeking conversations

SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR’17)

Luandri: a Clean Lua Interface to the Indri Search Engine

Neural Models for Information Retrieval

Learning to Match Using Local and Distributed Representations of Text for Web Search

A Theoretical Framework for Conversational Search

Neural Text Embeddings for Information Retrieval

Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)

Ingrams – a neuropsychological explanation for why people search

Neu-IR: The SIGIR 2016 Workshop on Neural Information Retrieval

A Proposal for Evaluating Answer Distillation from Web Data