KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrieval
Marah I Abdin, Suriya Gunasekar, Varun Chandrasekaran, Jerry Li, Mert Yuksekgonul, Rahee Peshawaria, Ranjita Naik, Besmira Nushi
October 2023
Besmira Nushi
Principal Researcher
Publications
Publications by Year
-
Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models
Mert Yuksekgonul, Varun Chandrasekaran, Erik Jones, Suriya Gunasekar, Ranjita Naik, Hamid Palangi, Ece Kamar, Besmira Nushi
September 2023
Social Biases through the Text-to-Image Generation Lens
AIES 2023 | August 2023
Mitigating Spurious Correlations in Multi-modal Models during Fine-tuning
Yu Yang, Besmira Nushi, Hamid Palangi, Baharan Mirzasoleiman
ICML 2023 | June 2023
Benchmarking Spatial Relationships in Text-to-Image Generation
Tejas Gokhale, Hamid Palangi, Besmira Nushi, Vibhav Vineet, Eric Horvitz, Ece Kamar, Chitta Baral, Yezhou Yang
MSR-TR-2023-44 | May 2023
Published by Microsoft
-
Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging
Riccardo Fogliato, Shreya Chappidi, Matthew Lungren, Michael Fitzke, Mark Parkinson, Diane Wilson, Paul Fisher, Eric Horvitz, Kori Inkpen, Besmira Nushi
FAccT 2022 | May 2022
HINT: Integration Testing for AI-based Features with Humans in the Loop
Quanze Chen, Tobias Schnabel, Besmira Nushi, Saleema Amershi
International Conference on Intelligent User Interfaces | March 2022
Investigations of Performance and Bias in Human-AI Teamwork in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Ece Kamar
AAAI 2022 | February 2022
-
Hierarchical Analysis of Visual COVID-19 Features from Chest Radiographs
Shruthi Bannur, Ozan Oktay, Melanie Bernhardt, Anton Schwaighofer, Rajesh Jena, Besmira Nushi, Sharan Wadhwani, Aditya Nori, Kal Natarajan, Shazad Ashraf, Javier Alvarez-Valle, Daniel Coelho de Castro
ICML 2021 Workshop on Interpretable Machine Learning in Healthcare | July 2021
Understanding Failures of Deep Networks via Robust Feature Extraction
Sahil Singla, Besmira Nushi, Shital Shah, Ece Kamar, Eric Horvitz
CVPR 2021 | June 2021
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, Daniel S. Weld
CHI | May 2021
Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork
Gagan Bansal, Besmira Nushi, Ece Kamar, Eric Horvitz, Dan Weld
AAAI 2021 | February 2021
-
An Empirical Analysis of Backward Compatibility in Machine Learning Systems
Megha Srivastava, Besmira Nushi, Ece Kamar, Shital Shah, Eric Horvitz
KDD | August 2020
SQuINTing at VQA Models: Introspecting VQA Models with Sub-Questions
Ramprasaath R. Selvaraju, Purva Tendulkar, Devi Parikh, Eric Horvitz, Marco Tulio Ribeiro, Besmira Nushi, Ece Kamar
CVPR 2020 | June 2020
Characterizing Search-Engine Traffic to Internet Research Agency Web Properties
Alexander Spangher, Gireeja Ranade, Besmira Nushi, Adam Fourney, Eric Horvitz
The Web Conference (TheWebConf '20) | April 2020
Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations
Aditya Modi, Debadeepta Dey, Alekh Agarwal, Adith Swaminathan, Besmira Nushi, Sean Andrist, Eric Horvitz
The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20) | February 2020
-
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Gagan Bansal, Besmira Nushi, Ece Kamar, Walter Lasecki, Dan Weld, Eric Horvitz
HCOMP 2019 | October 2019
What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Siddharth Suri, Ece Kamar
7th Conference on Human Computation and Crowdsourcing (HCOMP 2019) | September 2019
Software Engineering for Machine Learning: A Case Study
Saleema Amershi, Andrew Begel, Christian Bird, Robert DeLIne, Harald Gall, Ece Kamar, Nachi Nagappan, Besmira Nushi, Tom Zimmermann
International Conference on Software Engineering (ICSE 2019) - Software Engineering in Practice track | May 2019
ICSE 2019 Best Paper Award
Guidelines for Human-AI Interaction
Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, Eric Horvitz
CHI 2019 | May 2019
CHI 2019 Honorable Mention Award
Overcoming Blind Spots in the Real World: Leveraging Complementary Abilities for Joint Execution
Ramya Ramakrishnan, Ece Kamar, Besmira Nushi, Debadeepta Dey, Julie Shah, Eric Horvitz
AAAI Conference on Artificial Intelligence | January 2019
Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Gagan Bansal, Besmira Nushi, Ece Kamar, Dan Weld, Walter Lasecki, Eric Horvitz
AAAI Conference on Artificial Intelligence | January 2019
-
-
-
Quality Control and Optimization for Hybrid Crowd-Machine Learning Systems
December 2016
December 2016
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems
Besmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
Thirty-First AAAI Conference on Artificial Intelligence | November 2016
Learning and Feature Selection under Budget Constraints in Crowdsourcing.
Besmira Nushi, Adish Singla, Andreas Krause, Donald Kossmann
HCOMP 2016 | January 2016
-
Crowd-STAR: A Social Task Routing Framework for Online Communities.
Besmira Nushi, Omar Alonso, Martin Hentschel, Vasileios Kandylas
ICWE 2015 | November 2015
Crowd Access Path Optimization: Diversity matters.
Besmira Nushi, Adish Singla, Anja Gruenheid, Erfan Zamanian, Andreas Krause, Donald Kossmann
HCOMP 2015 | January 2015
-
When is A = B?
Donald Kossmann, Anja Gruenheid, Besmira Nushi
Bulletin of the EATCS | November 2013
-
Uncertain time-series similarity: Return to the basics.
Michele Dallachiesa, Besmira Nushi, Katsiaryna Mirylenka, Themis Palpanas
Very Large Databases | November 2012
-
Similarity matching for uncertain time series: analytical and experimental comparison.
Michele Dallachiesa, Besmira Nushi, Katsiaryna Mirylenka, Themis Palpanas
Proceedings of the 2nd ACM SIGSPATIAL International Workshop on Querying and Mining Uncertain Spatio-Temporal Data | November 2011
Publications by Research Area
-
KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrieval
Marah I Abdin, Suriya Gunasekar, Varun Chandrasekaran, Jerry Li, Mert Yuksekgonul, Rahee Peshawaria, Ranjita Naik, Besmira Nushi
October 2023
Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models
Mert Yuksekgonul, Varun Chandrasekaran, Erik Jones, Suriya Gunasekar, Ranjita Naik, Hamid Palangi, Ece Kamar, Besmira Nushi
September 2023
Social Biases through the Text-to-Image Generation Lens
AIES 2023 | August 2023
Mitigating Spurious Correlations in Multi-modal Models during Fine-tuning
Yu Yang, Besmira Nushi, Hamid Palangi, Baharan Mirzasoleiman
ICML 2023 | June 2023
Benchmarking Spatial Relationships in Text-to-Image Generation
Tejas Gokhale, Hamid Palangi, Besmira Nushi, Vibhav Vineet, Eric Horvitz, Ece Kamar, Chitta Baral, Yezhou Yang
MSR-TR-2023-44 | May 2023
Published by Microsoft
Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging
Riccardo Fogliato, Shreya Chappidi, Matthew Lungren, Michael Fitzke, Mark Parkinson, Diane Wilson, Paul Fisher, Eric Horvitz, Kori Inkpen, Besmira Nushi
FAccT 2022 | May 2022
HINT: Integration Testing for AI-based Features with Humans in the Loop
Quanze Chen, Tobias Schnabel, Besmira Nushi, Saleema Amershi
International Conference on Intelligent User Interfaces | March 2022
Investigations of Performance and Bias in Human-AI Teamwork in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Ece Kamar
AAAI 2022 | February 2022
Hierarchical Analysis of Visual COVID-19 Features from Chest Radiographs
Shruthi Bannur, Ozan Oktay, Melanie Bernhardt, Anton Schwaighofer, Rajesh Jena, Besmira Nushi, Sharan Wadhwani, Aditya Nori, Kal Natarajan, Shazad Ashraf, Javier Alvarez-Valle, Daniel Coelho de Castro
ICML 2021 Workshop on Interpretable Machine Learning in Healthcare | July 2021
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, Daniel S. Weld
CHI | May 2021
Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork
Gagan Bansal, Besmira Nushi, Ece Kamar, Eric Horvitz, Dan Weld
AAAI 2021 | February 2021
An Empirical Analysis of Backward Compatibility in Machine Learning Systems
Megha Srivastava, Besmira Nushi, Ece Kamar, Shital Shah, Eric Horvitz
KDD | August 2020
SQuINTing at VQA Models: Introspecting VQA Models with Sub-Questions
Ramprasaath R. Selvaraju, Purva Tendulkar, Devi Parikh, Eric Horvitz, Marco Tulio Ribeiro, Besmira Nushi, Ece Kamar
CVPR 2020 | June 2020
Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations
Aditya Modi, Debadeepta Dey, Alekh Agarwal, Adith Swaminathan, Besmira Nushi, Sean Andrist, Eric Horvitz
The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20) | February 2020
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Gagan Bansal, Besmira Nushi, Ece Kamar, Walter Lasecki, Dan Weld, Eric Horvitz
HCOMP 2019 | October 2019
What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Siddharth Suri, Ece Kamar
7th Conference on Human Computation and Crowdsourcing (HCOMP 2019) | September 2019
Software Engineering for Machine Learning: A Case Study
Saleema Amershi, Andrew Begel, Christian Bird, Robert DeLIne, Harald Gall, Ece Kamar, Nachi Nagappan, Besmira Nushi, Tom Zimmermann
International Conference on Software Engineering (ICSE 2019) - Software Engineering in Practice track | May 2019
ICSE 2019 Best Paper Award
Guidelines for Human-AI Interaction
Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, Eric Horvitz
CHI 2019 | May 2019
CHI 2019 Honorable Mention Award
Overcoming Blind Spots in the Real World: Leveraging Complementary Abilities for Joint Execution
Ramya Ramakrishnan, Ece Kamar, Besmira Nushi, Debadeepta Dey, Julie Shah, Eric Horvitz
AAAI Conference on Artificial Intelligence | January 2019
Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Gagan Bansal, Besmira Nushi, Ece Kamar, Dan Weld, Walter Lasecki, Eric Horvitz
AAAI Conference on Artificial Intelligence | January 2019
Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure
Besmira Nushi, Ece Kamar, Eric Horvitz
HCOMP 2018 | July 2018
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems.
Besmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
AAAI 2017 | January 2017
Quality Control and Optimization for Hybrid Crowd-Machine Learning Systems
December 2016
December 2016
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems
Besmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
Thirty-First AAAI Conference on Artificial Intelligence | November 2016
Learning and Feature Selection under Budget Constraints in Crowdsourcing.
Besmira Nushi, Adish Singla, Andreas Krause, Donald Kossmann
HCOMP 2016 | January 2016
Crowd Access Path Optimization: Diversity matters.
Besmira Nushi, Adish Singla, Anja Gruenheid, Erfan Zamanian, Andreas Krause, Donald Kossmann
HCOMP 2015 | January 2015
-
KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrieval
Marah I Abdin, Suriya Gunasekar, Varun Chandrasekaran, Jerry Li, Mert Yuksekgonul, Rahee Peshawaria, Ranjita Naik, Besmira Nushi
October 2023
Characterizing Search-Engine Traffic to Internet Research Agency Web Properties
Alexander Spangher, Gireeja Ranade, Besmira Nushi, Adam Fourney, Eric Horvitz
The Web Conference (TheWebConf '20) | April 2020
Crowd-STAR: A Social Task Routing Framework for Online Communities.
Besmira Nushi, Omar Alonso, Martin Hentschel, Vasileios Kandylas
ICWE 2015 | November 2015
-
Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging
Riccardo Fogliato, Shreya Chappidi, Matthew Lungren, Michael Fitzke, Mark Parkinson, Diane Wilson, Paul Fisher, Eric Horvitz, Kori Inkpen, Besmira Nushi
FAccT 2022 | May 2022
HINT: Integration Testing for AI-based Features with Humans in the Loop
Quanze Chen, Tobias Schnabel, Besmira Nushi, Saleema Amershi
International Conference on Intelligent User Interfaces | March 2022
Investigations of Performance and Bias in Human-AI Teamwork in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Ece Kamar
AAAI 2022 | February 2022
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, Daniel S. Weld
CHI | May 2021
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Gagan Bansal, Besmira Nushi, Ece Kamar, Walter Lasecki, Dan Weld, Eric Horvitz
HCOMP 2019 | October 2019
What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Siddharth Suri, Ece Kamar
7th Conference on Human Computation and Crowdsourcing (HCOMP 2019) | September 2019
Guidelines for Human-AI Interaction
Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, Eric Horvitz
CHI 2019 | May 2019
CHI 2019 Honorable Mention Award
Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Gagan Bansal, Besmira Nushi, Ece Kamar, Dan Weld, Walter Lasecki, Eric Horvitz
AAAI Conference on Artificial Intelligence | January 2019
Quality Control and Optimization for Hybrid Crowd-Machine Learning Systems
December 2016
December 2016
-
Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging
Riccardo Fogliato, Shreya Chappidi, Matthew Lungren, Michael Fitzke, Mark Parkinson, Diane Wilson, Paul Fisher, Eric Horvitz, Kori Inkpen, Besmira Nushi
FAccT 2022 | May 2022
What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Siddharth Suri, Ece Kamar
7th Conference on Human Computation and Crowdsourcing (HCOMP 2019) | September 2019
-
Hierarchical Analysis of Visual COVID-19 Features from Chest Radiographs
Shruthi Bannur, Ozan Oktay, Melanie Bernhardt, Anton Schwaighofer, Rajesh Jena, Besmira Nushi, Sharan Wadhwani, Aditya Nori, Kal Natarajan, Shazad Ashraf, Javier Alvarez-Valle, Daniel Coelho de Castro
ICML 2021 Workshop on Interpretable Machine Learning in Healthcare | July 2021
Understanding Failures of Deep Networks via Robust Feature Extraction
Sahil Singla, Besmira Nushi, Shital Shah, Ece Kamar, Eric Horvitz
CVPR 2021 | June 2021
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems.
Besmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
AAAI 2017 | January 2017
-
Hierarchical Analysis of Visual COVID-19 Features from Chest Radiographs
Shruthi Bannur, Ozan Oktay, Melanie Bernhardt, Anton Schwaighofer, Rajesh Jena, Besmira Nushi, Sharan Wadhwani, Aditya Nori, Kal Natarajan, Shazad Ashraf, Javier Alvarez-Valle, Daniel Coelho de Castro
ICML 2021 Workshop on Interpretable Machine Learning in Healthcare | July 2021
-
Software Engineering for Machine Learning: A Case Study
Saleema Amershi, Andrew Begel, Christian Bird, Robert DeLIne, Harald Gall, Ece Kamar, Nachi Nagappan, Besmira Nushi, Tom Zimmermann
International Conference on Software Engineering (ICSE 2019) - Software Engineering in Practice track | May 2019
ICSE 2019 Best Paper Award
-
When is A = B?
Donald Kossmann, Anja Gruenheid, Besmira Nushi
Bulletin of the EATCS | November 2013
Uncertain time-series similarity: Return to the basics.
Michele Dallachiesa, Besmira Nushi, Katsiaryna Mirylenka, Themis Palpanas
Very Large Databases | November 2012
-
When is A = B?
Donald Kossmann, Anja Gruenheid, Besmira Nushi
Bulletin of the EATCS | November 2013
Uncertain time-series similarity: Return to the basics.
Michele Dallachiesa, Besmira Nushi, Katsiaryna Mirylenka, Themis Palpanas
Very Large Databases | November 2012
Similarity matching for uncertain time series: analytical and experimental comparison.
Michele Dallachiesa, Besmira Nushi, Katsiaryna Mirylenka, Themis Palpanas
Proceedings of the 2nd ACM SIGSPATIAL International Workshop on Querying and Mining Uncertain Spatio-Temporal Data | November 2011
Publications by Type
-
KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrieval
Marah I Abdin, Suriya Gunasekar, Varun Chandrasekaran, Jerry Li, Mert Yuksekgonul, Rahee Peshawaria, Ranjita Naik, Besmira Nushi
October 2023
Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models
Mert Yuksekgonul, Varun Chandrasekaran, Erik Jones, Suriya Gunasekar, Ranjita Naik, Hamid Palangi, Ece Kamar, Besmira Nushi
September 2023
-
Benchmarking Spatial Relationships in Text-to-Image Generation
Tejas Gokhale, Hamid Palangi, Besmira Nushi, Vibhav Vineet, Eric Horvitz, Ece Kamar, Chitta Baral, Yezhou Yang
MSR-TR-2023-44 | May 2023
Published by Microsoft
-
Social Biases through the Text-to-Image Generation Lens
AIES 2023 | August 2023
Mitigating Spurious Correlations in Multi-modal Models during Fine-tuning
Yu Yang, Besmira Nushi, Hamid Palangi, Baharan Mirzasoleiman
ICML 2023 | June 2023
Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging
Riccardo Fogliato, Shreya Chappidi, Matthew Lungren, Michael Fitzke, Mark Parkinson, Diane Wilson, Paul Fisher, Eric Horvitz, Kori Inkpen, Besmira Nushi
FAccT 2022 | May 2022
HINT: Integration Testing for AI-based Features with Humans in the Loop
Quanze Chen, Tobias Schnabel, Besmira Nushi, Saleema Amershi
International Conference on Intelligent User Interfaces | March 2022
Investigations of Performance and Bias in Human-AI Teamwork in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Ece Kamar
AAAI 2022 | February 2022
Hierarchical Analysis of Visual COVID-19 Features from Chest Radiographs
Shruthi Bannur, Ozan Oktay, Melanie Bernhardt, Anton Schwaighofer, Rajesh Jena, Besmira Nushi, Sharan Wadhwani, Aditya Nori, Kal Natarajan, Shazad Ashraf, Javier Alvarez-Valle, Daniel Coelho de Castro
ICML 2021 Workshop on Interpretable Machine Learning in Healthcare | July 2021
Understanding Failures of Deep Networks via Robust Feature Extraction
Sahil Singla, Besmira Nushi, Shital Shah, Ece Kamar, Eric Horvitz
CVPR 2021 | June 2021
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, Daniel S. Weld
CHI | May 2021
Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork
Gagan Bansal, Besmira Nushi, Ece Kamar, Eric Horvitz, Dan Weld
AAAI 2021 | February 2021
An Empirical Analysis of Backward Compatibility in Machine Learning Systems
Megha Srivastava, Besmira Nushi, Ece Kamar, Shital Shah, Eric Horvitz
KDD | August 2020
SQuINTing at VQA Models: Introspecting VQA Models with Sub-Questions
Ramprasaath R. Selvaraju, Purva Tendulkar, Devi Parikh, Eric Horvitz, Marco Tulio Ribeiro, Besmira Nushi, Ece Kamar
CVPR 2020 | June 2020
Characterizing Search-Engine Traffic to Internet Research Agency Web Properties
Alexander Spangher, Gireeja Ranade, Besmira Nushi, Adam Fourney, Eric Horvitz
The Web Conference (TheWebConf '20) | April 2020
Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations
Aditya Modi, Debadeepta Dey, Alekh Agarwal, Adith Swaminathan, Besmira Nushi, Sean Andrist, Eric Horvitz
The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20) | February 2020
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Gagan Bansal, Besmira Nushi, Ece Kamar, Walter Lasecki, Dan Weld, Eric Horvitz
HCOMP 2019 | October 2019
What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Siddharth Suri, Ece Kamar
7th Conference on Human Computation and Crowdsourcing (HCOMP 2019) | September 2019
Software Engineering for Machine Learning: A Case Study
Saleema Amershi, Andrew Begel, Christian Bird, Robert DeLIne, Harald Gall, Ece Kamar, Nachi Nagappan, Besmira Nushi, Tom Zimmermann
International Conference on Software Engineering (ICSE 2019) - Software Engineering in Practice track | May 2019
ICSE 2019 Best Paper Award
Guidelines for Human-AI Interaction
Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, Eric Horvitz
CHI 2019 | May 2019
CHI 2019 Honorable Mention Award
Overcoming Blind Spots in the Real World: Leveraging Complementary Abilities for Joint Execution
Ramya Ramakrishnan, Ece Kamar, Besmira Nushi, Debadeepta Dey, Julie Shah, Eric Horvitz
AAAI Conference on Artificial Intelligence | January 2019
Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Gagan Bansal, Besmira Nushi, Ece Kamar, Dan Weld, Walter Lasecki, Eric Horvitz
AAAI Conference on Artificial Intelligence | January 2019
Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure
Besmira Nushi, Ece Kamar, Eric Horvitz
HCOMP 2018 | July 2018
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems.
Besmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
AAAI 2017 | January 2017
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems
Besmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
Thirty-First AAAI Conference on Artificial Intelligence | November 2016
Learning and Feature Selection under Budget Constraints in Crowdsourcing.
Besmira Nushi, Adish Singla, Andreas Krause, Donald Kossmann
HCOMP 2016 | January 2016
Crowd-STAR: A Social Task Routing Framework for Online Communities.
Besmira Nushi, Omar Alonso, Martin Hentschel, Vasileios Kandylas
ICWE 2015 | November 2015
Crowd Access Path Optimization: Diversity matters.
Besmira Nushi, Adish Singla, Anja Gruenheid, Erfan Zamanian, Andreas Krause, Donald Kossmann
HCOMP 2015 | January 2015
Similarity matching for uncertain time series: analytical and experimental comparison.
Michele Dallachiesa, Besmira Nushi, Katsiaryna Mirylenka, Themis Palpanas
Proceedings of the 2nd ACM SIGSPATIAL International Workshop on Querying and Mining Uncertain Spatio-Temporal Data | November 2011
-
Quality Control and Optimization for Hybrid Crowd-Machine Learning Systems
December 2016
December 2016
-
When is A = B?
Donald Kossmann, Anja Gruenheid, Besmira Nushi
Bulletin of the EATCS | November 2013
Uncertain time-series similarity: Return to the basics.
Michele Dallachiesa, Besmira Nushi, Katsiaryna Mirylenka, Themis Palpanas
Very Large Databases | November 2012