The Collaboration Gap
Tim R. Davidson, Adam Fourney, Saleema Amershi, Robert West, Eric Horvitz, Ece Kamar
November 2025
Ece Kamar
CVP and Lab Director of AI Frontiers
论文与出版物
-
-
-
Magentic-UI: Towards Human-in-the-loop Agentic Systems
Hussein Mozannar, Gagan Bansal, Cheng Tan, Adam Fourney, Victor Dibia, Jingya Chen, Jack Gerrits, Tyler Payne, Matheus Kunzler Maldaner, Madeleine Grunde-McLaughlin, Eric Zhu, Griffin Bassman, Jacob Alber, Peter Chang, Ricky Loynd, Friederike Niedtner, Ece Kamar, Maya Murad, Rafah Hosn, Saleema Amershi
MSR-TR-2025-40 | July 2025
作者：Microsoft
-
-
-
Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks
Adam Fourney, Gagan Bansal, Hussein Mozannar, Cheng Tan, Eduardo Salinas, Erkang (Eric) Zhu, Friederike Niedtner, Grace Proebsting, Griffin Bassman, Jack Gerrits, Jacob Alber, Peter Chang, Ricky Loynd, Robert West, Victor Dibia, Ahmed Awadallah, Ece Kamar, Rafah Hosn, Saleema Amershi
MSR-TR-2024-47 | November 2024
作者：Microsoft
-
-
-
Teaching Language Models to Hallucinate Less with Synthetic Tasks
Erik Jones, Hamid Palangi, Clarisse Simoes Ribeiro, Varun Chandrasekaran, Subhabrata (Subho) Mukherjee, Arindam Mitra, Ahmed Awadallah, Ece Kamar
ICLR 2024 | October 2023
-
Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models
Mert Yuksekgonul, Varun Chandrasekaran, Erik Jones, Suriya Gunasekar, Ranjita Naik, Hamid Palangi, Ece Kamar, Besmira Nushi
September 2023
-
Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions
John Chung, Ece Kamar, Saleema Amershi
ACL 2023 | July 2023
-
Benchmarking Spatial Relationships in Text-to-Image Generation
Tejas Gokhale, Hamid Palangi, Besmira Nushi, Vibhav Vineet, Eric Horvitz, Ece Kamar, Chitta Baral, Yezhou Yang
MSR-TR-2023-44 | May 2023
作者：Microsoft
-
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, Yi Zhang
March 2023
-
-
-
Avoiding Negative Side Effects of Autonomous Systems in the Open World
Sandhya Saisubramanian, Ece Kamar, Shlomo Zilberstein
May 2022
-
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, Ece Kamar
ACL 2022 | May 2022
-
Investigations of Performance and Bias in Human-AI Teamwork in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Ece Kamar
AAAI 2022 | February 2022
-
-
-
Avoiding Negative Side Effects due to Incomplete Knowledge of AI Systems
Sandhya Saisubramanian, Shlomo Zilberstein, Ece Kamar
October 2021
-
Algorithmic Recourse in the Wild: Understanding the Impact of Data and Model Shifts
Kaivalya Rawal, Ece Kamar, Himabindu Lakkaraju
June 2021
-
Understanding Failures of Deep Networks via Robust Feature Extraction
Sahil Singla, Besmira Nushi, Shital Shah, Ece Kamar, Eric Horvitz
CVPR 2021 | June 2021
-
Understanding the Representation and Representativeness of Age in AI Data Sets
Joon Sung Park, Michael S. Bernstein, Robin N. Brewer, Ece Kamar, Meredith Ringel Morris
2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21) | May 2021
-
Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs
Solon Barocas, Anhong Guo, Ece Kamar, Jacquelyn Krones, Meredith Ringel Morris, Jennifer Wortman Vaughan, Duncan Wadsworth, Hanna Wallach
AIES 2021 | May 2021
-
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, Daniel S. Weld
CHI | May 2021
-
A Bayesian Approach to Identifying Representational Errors
Ramya Ramakrishnan, Vaibhav Unhelkar, Ece Kamar, Julie Shah
March 2021
-
Designing an Online Infrastructure for Collecting AI Data From People With Disabilities
Joon Sung Park, Danielle Bragg, Ece Kamar, Meredith Ringel Morris
FAccT | March 2021
-
Improving the Performance-Compatibility Tradeoff with Personalized Objective Functions
Jonathan Martinez, Kobi Gal, Ece Kamar, Levi H. S. Lelis
The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21) | February 2021
-
Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork
Gagan Bansal, Besmira Nushi, Ece Kamar, Eric Horvitz, Dan Weld
AAAI 2021 | February 2021
-
Learning to Complement Humans
Bryan Wilder, Eric Horvitz, Ece Kamar
IJCAI 2020 | January 2021
-
-
-
An Empirical Analysis of Backward Compatibility in Machine Learning Systems
Megha Srivastava, Besmira Nushi, Ece Kamar, Shital Shah, Eric Horvitz
KDD | August 2020
-
Security and Machine Learning in the Real World
Ivan Evtimov, Weidong Cui, Ece Kamar, Emre Kiciman, Tadayoshi Kohno, Jerry Li
July 2020
-
SQuINTing at VQA Models: Introspecting VQA Models with Sub-Questions
Ramprasaath R. Selvaraju, Purva Tendulkar, Devi Parikh, Eric Horvitz, Marco Tulio Ribeiro, Besmira Nushi, Ece Kamar
CVPR 2020 | June 2020
-
-
-
Toward Fairness in AI for People with Disabilities: A Research Roadmap
Anhong Guo, Ece Kamar, Jennifer Wortman Vaughan, Hanna Wallach, Meredith Ringel Morris
ASSETS 2019 Workshop on AI Fairness for People with Disabilities | October 2019
-
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Gagan Bansal, Besmira Nushi, Ece Kamar, Walter Lasecki, Dan Weld, Eric Horvitz
HCOMP 2019 | October 2019
-
What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Siddharth Suri, Ece Kamar
7th Conference on Human Computation and Crowdsourcing (HCOMP 2019) | September 2019
-
Software Engineering for Machine Learning: A Case Study
Saleema Amershi, Andrew Begel, Christian Bird, Robert DeLIne, Harald Gall, Ece Kamar, Nachi Nagappan, Besmira Nushi, Tom Zimmermann
International Conference on Software Engineering (ICSE 2019) - Software Engineering in Practice track | May 2019
ICSE 2019 Best Paper Award
-
Studying Preferences and Concerns about Information Disclosure in Email Notifications
Yongsung Kim, Adam Fourney, Ece Kamar
The Web Conference (WWW '19) | May 2019
-
People and AI See Things Different Implications of Mismatched Perception on HCI for AI Systems
Saleema Amershi, Ece Kamar, Emre Kiciman
Workshop on Human-Centered Machine Learning Perspectives at CHI 2019 | May 2019
-
Overcoming Blind Spots in the Real World: Leveraging Complementary Abilities for Joint Execution
Ramya Ramakrishnan, Ece Kamar, Besmira Nushi, Debadeepta Dey, Julie Shah, Eric Horvitz
AAAI Conference on Artificial Intelligence | January 2019
-
Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Gagan Bansal, Besmira Nushi, Ece Kamar, Dan Weld, Walter Lasecki, Eric Horvitz
AAAI Conference on Artificial Intelligence | January 2019
-
-
-
Evaluating and Complementing Vision-to-Language Technology for People who are Blind with Conversational Crowdsourcing
Elliot Salisbury, Ece Kamar, Meredith Ringel Morris
Proceedings of IJCAI 2018 | July 2018
-
Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure
Besmira Nushi, Ece Kamar, Eric Horvitz
HCOMP 2018 | July 2018
-
Discovering Blind Spots in Reinforcement Learning
Ramya Ramakrishnan, Ece Kamar, Debadeepta Dey, Julie Shah, Eric Horvitz
17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2018) | May 2018
-
Optimizing Interventions via Offline Policy Evaluation: Studies in Citizen Science
Avi Segal, Kobi Gal, Ece Kamar, Eric Horvitz
AAAI-18: Thirty-Second AAAI Conference on Artificial Intelligence | January 2018
-
-
-
Toward Scalable Social Alt Text: Conversational Crowdsourcing as a Tool for Refining Vision-to-Language Technology for the Blind
Elliot Salisbury, Ece Kamar, Meredith Ringel Morris
Proceedings of HCOMP 2017 | October 2017
Best Paper Award
-
What Went Wrong and Why? Diagnosing Situated Interaction Failures in the Wild
Sean Andrist, Dan Bohus, Ece Kamar, Eric Horvitz
ICSR 2017: Social Robotics | July 2017
-
Revolt: Collaborative Crowdsourcing for Labeling Machine Learning Datasets
Joseph Chee Chang, Saleema Amershi, Ece Kamar
Proceedings of the Conference on Human Factors in Computing Systems (CHI 2017) | May 2017
-
Communicating Context to the Crowd for Complex Writing Tasks
Niloufar Salehi, Jaime Teevan, Shamsi Iqbal, Ece Kamar
Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work (CSCW 2017), Portland, OR | February 2017
Honorable Mention
-
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems.
Besmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
AAAI 2017 | January 2017
-
-
-
Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration
Himabindu Lakkaraju, Ece Kamar, Rich Caruana, Eric Horvitz
Thirty-First AAAI Conference on Artificial Intelligence | December 2016
-
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems
Besmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
Thirty-First AAAI Conference on Artificial Intelligence | November 2016
-
Hybrid Intelligence and the Future of Work
In Proceedings of the Productivity Decomposed: Getting Big Things Done with Little Microtasks Workshop at CHI 2016 | May 2016
-
Toward a Learning Science for Complex Crowdsourcing Tasks
Shayan Doroudi, Ece Kamar, Emma Brunskill, Eric Horvitz
Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems | May 2016
-
Interactive Teaching Strategies for Agent Training
Ofra Amir, Ece Kamar, Andrey Kolobov, Barbara Grosz
IJCAI 2016 | May 2016
-
Intervention Strategies for Increasing Engagement in Crowdsourcing: Platform, Predictions, and Experiments
Avi Segal, Ya’akov (Kobi) Gal, Ece Kamar, Eric Horvitz, Alex Bowyer, Grant Miller
Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16) | April 2016
-
Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence
Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16) | March 2016
-
-
-
Identifying and Accounting for Task-Dependent Bias in Crowdsourcing
Ece Kamar, Ashish Kapoor, Eric Horvitz
In Proceedings of HCOMP 2015 | August 2015
-
Metareasoning for Planning Under Uncertainty
Christopher H. Lin, Andrey Kolobov, Ece Kamar, Eric Horvitz
IJCAI 2015 | July 2015
-
Clustering for Set Partitioning: A Case Study in Carpooling
Cathy Wu, Ece Kamar, Eric Horvitz
In Proceedings of the Workshop on Optimization for Machine Learning (OPT) at NIPS 2015. | May 2015
-
Planning for Crowdsourcing Hierarchical Tasks
Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2015), Bordini, Elkind, Weiss, Yolum (eds.), May 4-8, 2015, Istanbul, Turkey. | May 2015
-
The Cost of Asking Crowd Workers to Behave Maliciously
Walter. S. Lasecki, Jaime Teevan, Ece Kamar
In Proceedings of the Workshop on Human-Agent Interaction Design and Models (HAIDM 2015) at AAMAS 2015 | February 2015
-
Preserving Privacy in Crowd-Powered Systems
Walter S. Lasecki, Mitchell Gordon, Jaime Teevan, Ece Kamar, Jeff P. Bigham
In Proceedings of the Workshop on Human-Agent Interaction Design and Models (HAIDM 2015) at AAMAS 2015 | January 2015
-
-
-
Signals in the Silence: Models of Implicit Feedback in a Recommendation System for Crowdsourcing
Christopher Lin, Ece Kamar, Eric Horvitz
In Proceedings of AAAI 2014 | June 2014
-
Crowdsourcing Language Generation Templates for Dialogue Systems
Margaret Mitchell, Dan Bohus, Ece Kamar
Proceedings of the INLG and SIGDIAL 2014 Joint Session | June 2014
ISBN: 978-1-941643-23-5
-
Gauss Meets Canadian Traveler: Shortest-Path Problems with Correlated Natural Dynamics
Debadeepta Dey, Andrey Kolobov, Rich Caruana, Ece Kamar, Eric Horvitz, Ashish Kapoor
AAMAS 2014 | May 2014
-
Stochastic Privacy
Adish Singla, Eric Horvitz, Ece Kamar, Ryen W. White
April 2014
-
Information Extraction and Manipulation Threats in Crowd-Powered Systems
Walter S. Lasecki, Jaime Teevan, Ece Kamar
CSCW 2014 | February 2014
-
-
-
Lifelong Learning for Acquiring the Wisdom of the Crowd
Ece Kamar, Ashish Kapoor, Eric Horvitz
IJCAI '13 Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence | August 2013
-
Light at the End of the Tunnel: A Monte Carlo Approach to Computing Value of Information
Twelth International Conference on Autonomous Agents and Mutli-agent Systems | May 2013
-
Look versus Leap: Computing Value of Information with High-Dimensional Streaming Evidence
Stephanie Rosenthal, Dan Bohus, Ece Kamar, Eric Horvitz
International Joint Conference on Artificial Intelligence | April 2013
-
Volunteering Versus Work for Pay: Incentives and Tradeoffs In Crowdsourcing
Andrew Mao, Ece Kamar, Yiling Chen, Eric Horvitz, Megan E Schwamb, Chris J Lintott, Arfon M Smith
First AAAI Conference on Human Computation and Crowdsourcing | March 2013
-
Why Stop Now? Predicting Worker Engagement in Online Crowdsourcing
Andrew Mao, Ece Kamar, Eric Horvitz
First AAAI Conference on Human Computation and Crowdsourcing | March 2013
-
Modeling Information Exchange Opportunities For Effective Human-Computer Teamwork
Ece Kamar, Ya’akov (Kobi) Gal, Barbara J. Grosz
Artificial Intelligence Journal | February 2013
-
Conversations in the Crowd: Collecting Data for Task-Oriented Dialog Learning
Walter Lasecki, Ece Kamar, Dan Bohus
In Proceedings of the Human Computation Workshop on Scaling Speech and Language Understanding and Dialog through Crowdsourcing at HCOMP 2013. | January 2013
-
-
-
Crowdsourcing the Acquisition of Natural Language Corpora: Methods and Observations
William Yang Wang, Dan Bohus, Ece Kamar, Eric Horvitz
October 2012
-
Towards Situated Collaboration
Dan Bohus, Ece Kamar, Eric Horvitz
In NAACL Workshop on Future Directions and Challenges in Spoken Dialog Systems: Tools and Data | April 2012
-
Incentives and Truthful Reporting in Consensus-centric Crowdsourcing
MSR-TR-2012-16 | February 2012
-
Combining Human and Machine Intelligence in Large-scale Crowdsourcing
Ece Kamar, Severin Hacker, Eric Horvitz
AAMAS | January 2012
-
Incentives for Truthful Reporting in Crowdsourcing
AAMAS | January 2012
-
-
-
Jogger: Models for Context-Sensitive Reminding
The Tenth International Conference on Autonomous Agents and Multiagent Systems | January 2011
Short Paper
-
MSRC at TREC 2011 Crowdsourcing Track
Paul Bennett, Ece Kamar, Gabriella Kazai
In Proceedings of TREC 2011 | January 2011
-
-
-
Investigation of Principles of Context-Sensitive Reminding
MSR-TR-2010-174 | October 2010
-
Reasoning Effectively Under Uncertainty for Human-Computer Teamwork
May 2010
May 2010
-
-
-
Collaboration and Shared Plans in the Open World: Studies of Ridesharing
IJCAI'09 Proceedings of the 21st International Joint Conference on Artificial Intelligence | July 2009
-
Computation and Crowds: Models for Dynamic Ridesharing
Collaboration and Shared Plans in the Open World: Studies of Ridesharing, IJCAI 2009, Pasadena, CA | July 2009
-
Managing Helpful Behavior in Collaborative Activities of Heterogeneous Agent Groups
Twenty-First IAAI Conference | June 2009
-
Incorporating Helpful Behavior into Collaborative Planning
Ece Kamar, Ya'akov Gal, Barbara J. Grosz
In Proceedings of AAMAS 2009 | May 2009
-
Generating Shared Transportation Plans Under Varying Preferences: Ridesharing Models and Mechanisms
MSR-TR-2009-2011 | March 2009
-
Modeling User Perception of Interaction Opportunities for Effective Teamwork
Ece Kamar, Ya’akov Gal, Barbara J. Grosz
In Proceedings of IEEE SocialCom 2009 | January 2009
-
-
-
Mobile Opportunistic Commerce: Mechanisms, Architecture, and Application
Ece Kamar, Eric Horvitz, Chris Meek
In Proceedings of AAMAS 2008 | May 2008
ISBN: 978-0-9817381-1-6
-
-
-
Modeling User Perception of Interaction Opportunities in Collaborative Human-Computer Settings
Ece Kamar, Barbara J Grosz, David Sarne
In Proceedings of AAAI 2007, Student Abstract and Poster Program | July 2007
-
Applying MDP Approaches For Estimating Outcome of Interaction in Collaborative Human-Computer Settings
Ece Kamar, Barbara J. Grosz
In Proceedings of Workshop on Multi-agent Sequential Decision Making in Uncertain Domains (MSDM) 2007 | January 2007
-
-
-
The Collaboration Gap
Tim R. Davidson, Adam Fourney, Saleema Amershi, Robert West, Eric Horvitz, Ece Kamar
November 2025
-
Magentic-UI: Towards Human-in-the-loop Agentic Systems
Hussein Mozannar, Gagan Bansal, Cheng Tan, Adam Fourney, Victor Dibia, Jingya Chen, Jack Gerrits, Tyler Payne, Matheus Kunzler Maldaner, Madeleine Grunde-McLaughlin, Eric Zhu, Griffin Bassman, Jacob Alber, Peter Chang, Ricky Loynd, Friederike Niedtner, Ece Kamar, Maya Murad, Rafah Hosn, Saleema Amershi
MSR-TR-2025-40 | July 2025
作者：Microsoft
-
Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks
Adam Fourney, Gagan Bansal, Hussein Mozannar, Cheng Tan, Eduardo Salinas, Erkang (Eric) Zhu, Friederike Niedtner, Grace Proebsting, Griffin Bassman, Jack Gerrits, Jacob Alber, Peter Chang, Ricky Loynd, Robert West, Victor Dibia, Ahmed Awadallah, Ece Kamar, Rafah Hosn, Saleema Amershi
MSR-TR-2024-47 | November 2024
作者：Microsoft
-
Teaching Language Models to Hallucinate Less with Synthetic Tasks
Erik Jones, Hamid Palangi, Clarisse Simoes Ribeiro, Varun Chandrasekaran, Subhabrata (Subho) Mukherjee, Arindam Mitra, Ahmed Awadallah, Ece Kamar
ICLR 2024 | October 2023
-
Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models
Mert Yuksekgonul, Varun Chandrasekaran, Erik Jones, Suriya Gunasekar, Ranjita Naik, Hamid Palangi, Ece Kamar, Besmira Nushi
September 2023
-
Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions
John Chung, Ece Kamar, Saleema Amershi
ACL 2023 | July 2023
-
Benchmarking Spatial Relationships in Text-to-Image Generation
Tejas Gokhale, Hamid Palangi, Besmira Nushi, Vibhav Vineet, Eric Horvitz, Ece Kamar, Chitta Baral, Yezhou Yang
MSR-TR-2023-44 | May 2023
作者：Microsoft
-
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, Yi Zhang
March 2023
-
Avoiding Negative Side Effects of Autonomous Systems in the Open World
Sandhya Saisubramanian, Ece Kamar, Shlomo Zilberstein
May 2022
-
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, Ece Kamar
ACL 2022 | May 2022
-
Investigations of Performance and Bias in Human-AI Teamwork in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Ece Kamar
AAAI 2022 | February 2022
-
Avoiding Negative Side Effects due to Incomplete Knowledge of AI Systems
Sandhya Saisubramanian, Shlomo Zilberstein, Ece Kamar
October 2021
-
Algorithmic Recourse in the Wild: Understanding the Impact of Data and Model Shifts
Kaivalya Rawal, Ece Kamar, Himabindu Lakkaraju
June 2021
-
Understanding the Representation and Representativeness of Age in AI Data Sets
Joon Sung Park, Michael S. Bernstein, Robin N. Brewer, Ece Kamar, Meredith Ringel Morris
2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21) | May 2021
-
Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs
Solon Barocas, Anhong Guo, Ece Kamar, Jacquelyn Krones, Meredith Ringel Morris, Jennifer Wortman Vaughan, Duncan Wadsworth, Hanna Wallach
AIES 2021 | May 2021
-
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, Daniel S. Weld
CHI | May 2021
-
A Bayesian Approach to Identifying Representational Errors
Ramya Ramakrishnan, Vaibhav Unhelkar, Ece Kamar, Julie Shah
March 2021
-
Designing an Online Infrastructure for Collecting AI Data From People With Disabilities
Joon Sung Park, Danielle Bragg, Ece Kamar, Meredith Ringel Morris
FAccT | March 2021
-
Improving the Performance-Compatibility Tradeoff with Personalized Objective Functions
Jonathan Martinez, Kobi Gal, Ece Kamar, Levi H. S. Lelis
The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21) | February 2021
-
Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork
Gagan Bansal, Besmira Nushi, Ece Kamar, Eric Horvitz, Dan Weld
AAAI 2021 | February 2021
-
Learning to Complement Humans
Bryan Wilder, Eric Horvitz, Ece Kamar
IJCAI 2020 | January 2021
-
An Empirical Analysis of Backward Compatibility in Machine Learning Systems
Megha Srivastava, Besmira Nushi, Ece Kamar, Shital Shah, Eric Horvitz
KDD | August 2020
-
Security and Machine Learning in the Real World
Ivan Evtimov, Weidong Cui, Ece Kamar, Emre Kiciman, Tadayoshi Kohno, Jerry Li
July 2020
-
SQuINTing at VQA Models: Introspecting VQA Models with Sub-Questions
Ramprasaath R. Selvaraju, Purva Tendulkar, Devi Parikh, Eric Horvitz, Marco Tulio Ribeiro, Besmira Nushi, Ece Kamar
CVPR 2020 | June 2020
-
Toward Fairness in AI for People with Disabilities: A Research Roadmap
Anhong Guo, Ece Kamar, Jennifer Wortman Vaughan, Hanna Wallach, Meredith Ringel Morris
ASSETS 2019 Workshop on AI Fairness for People with Disabilities | October 2019
-
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Gagan Bansal, Besmira Nushi, Ece Kamar, Walter Lasecki, Dan Weld, Eric Horvitz
HCOMP 2019 | October 2019
-
What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Siddharth Suri, Ece Kamar
7th Conference on Human Computation and Crowdsourcing (HCOMP 2019) | September 2019
-
Software Engineering for Machine Learning: A Case Study
Saleema Amershi, Andrew Begel, Christian Bird, Robert DeLIne, Harald Gall, Ece Kamar, Nachi Nagappan, Besmira Nushi, Tom Zimmermann
International Conference on Software Engineering (ICSE 2019) - Software Engineering in Practice track | May 2019
ICSE 2019 Best Paper Award
-
People and AI See Things Different Implications of Mismatched Perception on HCI for AI Systems
Saleema Amershi, Ece Kamar, Emre Kiciman
Workshop on Human-Centered Machine Learning Perspectives at CHI 2019 | May 2019
-
Overcoming Blind Spots in the Real World: Leveraging Complementary Abilities for Joint Execution
Ramya Ramakrishnan, Ece Kamar, Besmira Nushi, Debadeepta Dey, Julie Shah, Eric Horvitz
AAAI Conference on Artificial Intelligence | January 2019
-
Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Gagan Bansal, Besmira Nushi, Ece Kamar, Dan Weld, Walter Lasecki, Eric Horvitz
AAAI Conference on Artificial Intelligence | January 2019
-
Evaluating and Complementing Vision-to-Language Technology for People who are Blind with Conversational Crowdsourcing
Elliot Salisbury, Ece Kamar, Meredith Ringel Morris
Proceedings of IJCAI 2018 | July 2018
-
Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure
Besmira Nushi, Ece Kamar, Eric Horvitz
HCOMP 2018 | July 2018
-
Discovering Blind Spots in Reinforcement Learning
Ramya Ramakrishnan, Ece Kamar, Debadeepta Dey, Julie Shah, Eric Horvitz
17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2018) | May 2018
-
Optimizing Interventions via Offline Policy Evaluation: Studies in Citizen Science
Avi Segal, Kobi Gal, Ece Kamar, Eric Horvitz
AAAI-18: Thirty-Second AAAI Conference on Artificial Intelligence | January 2018
-
What Went Wrong and Why? Diagnosing Situated Interaction Failures in the Wild
Sean Andrist, Dan Bohus, Ece Kamar, Eric Horvitz
ICSR 2017: Social Robotics | July 2017
-
Revolt: Collaborative Crowdsourcing for Labeling Machine Learning Datasets
Joseph Chee Chang, Saleema Amershi, Ece Kamar
Proceedings of the Conference on Human Factors in Computing Systems (CHI 2017) | May 2017
-
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems.
Besmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
AAAI 2017 | January 2017
-
Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration
Himabindu Lakkaraju, Ece Kamar, Rich Caruana, Eric Horvitz
Thirty-First AAAI Conference on Artificial Intelligence | December 2016
-
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems
Besmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
Thirty-First AAAI Conference on Artificial Intelligence | November 2016
-
Toward a Learning Science for Complex Crowdsourcing Tasks
Shayan Doroudi, Ece Kamar, Emma Brunskill, Eric Horvitz
Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems | May 2016
-
Interactive Teaching Strategies for Agent Training
Ofra Amir, Ece Kamar, Andrey Kolobov, Barbara Grosz
IJCAI 2016 | May 2016
-
Intervention Strategies for Increasing Engagement in Crowdsourcing: Platform, Predictions, and Experiments
Avi Segal, Ya’akov (Kobi) Gal, Ece Kamar, Eric Horvitz, Alex Bowyer, Grant Miller
Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16) | April 2016
-
Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence
Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16) | March 2016
-
Identifying and Accounting for Task-Dependent Bias in Crowdsourcing
Ece Kamar, Ashish Kapoor, Eric Horvitz
In Proceedings of HCOMP 2015 | August 2015
-
Metareasoning for Planning Under Uncertainty
Christopher H. Lin, Andrey Kolobov, Ece Kamar, Eric Horvitz
IJCAI 2015 | July 2015
-
Clustering for Set Partitioning: A Case Study in Carpooling
Cathy Wu, Ece Kamar, Eric Horvitz
In Proceedings of the Workshop on Optimization for Machine Learning (OPT) at NIPS 2015. | May 2015
-
Planning for Crowdsourcing Hierarchical Tasks
Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2015), Bordini, Elkind, Weiss, Yolum (eds.), May 4-8, 2015, Istanbul, Turkey. | May 2015
-
The Cost of Asking Crowd Workers to Behave Maliciously
Walter. S. Lasecki, Jaime Teevan, Ece Kamar
In Proceedings of the Workshop on Human-Agent Interaction Design and Models (HAIDM 2015) at AAMAS 2015 | February 2015
-
Signals in the Silence: Models of Implicit Feedback in a Recommendation System for Crowdsourcing
Christopher Lin, Ece Kamar, Eric Horvitz
In Proceedings of AAAI 2014 | June 2014
-
Crowdsourcing Language Generation Templates for Dialogue Systems
Margaret Mitchell, Dan Bohus, Ece Kamar
Proceedings of the INLG and SIGDIAL 2014 Joint Session | June 2014
ISBN: 978-1-941643-23-5
-
Gauss Meets Canadian Traveler: Shortest-Path Problems with Correlated Natural Dynamics
Debadeepta Dey, Andrey Kolobov, Rich Caruana, Ece Kamar, Eric Horvitz, Ashish Kapoor
AAMAS 2014 | May 2014
-
Stochastic Privacy
Adish Singla, Eric Horvitz, Ece Kamar, Ryen W. White
April 2014
-
Lifelong Learning for Acquiring the Wisdom of the Crowd
Ece Kamar, Ashish Kapoor, Eric Horvitz
IJCAI '13 Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence | August 2013
-
Look versus Leap: Computing Value of Information with High-Dimensional Streaming Evidence
Stephanie Rosenthal, Dan Bohus, Ece Kamar, Eric Horvitz
International Joint Conference on Artificial Intelligence | April 2013
-
Modeling Information Exchange Opportunities For Effective Human-Computer Teamwork
Ece Kamar, Ya’akov (Kobi) Gal, Barbara J. Grosz
Artificial Intelligence Journal | February 2013
-
Conversations in the Crowd: Collecting Data for Task-Oriented Dialog Learning
Walter Lasecki, Ece Kamar, Dan Bohus
In Proceedings of the Human Computation Workshop on Scaling Speech and Language Understanding and Dialog through Crowdsourcing at HCOMP 2013. | January 2013
-
Crowdsourcing the Acquisition of Natural Language Corpora: Methods and Observations
William Yang Wang, Dan Bohus, Ece Kamar, Eric Horvitz
October 2012
-
Towards Situated Collaboration
Dan Bohus, Ece Kamar, Eric Horvitz
In NAACL Workshop on Future Directions and Challenges in Spoken Dialog Systems: Tools and Data | April 2012
-
Incentives and Truthful Reporting in Consensus-centric Crowdsourcing
MSR-TR-2012-16 | February 2012
-
Incentives for Truthful Reporting in Crowdsourcing
AAMAS | January 2012
-
Combining Human and Machine Intelligence in Large-scale Crowdsourcing
Ece Kamar, Severin Hacker, Eric Horvitz
AAMAS | January 2012
-
MSRC at TREC 2011 Crowdsourcing Track
Paul Bennett, Ece Kamar, Gabriella Kazai
In Proceedings of TREC 2011 | January 2011
-
Reasoning Effectively Under Uncertainty for Human-Computer Teamwork
May 2010
May 2010
-
Collaboration and Shared Plans in the Open World: Studies of Ridesharing
IJCAI'09 Proceedings of the 21st International Joint Conference on Artificial Intelligence | July 2009
-
Computation and Crowds: Models for Dynamic Ridesharing
Collaboration and Shared Plans in the Open World: Studies of Ridesharing, IJCAI 2009, Pasadena, CA | July 2009
-
Managing Helpful Behavior in Collaborative Activities of Heterogeneous Agent Groups
Twenty-First IAAI Conference | June 2009
-
Incorporating Helpful Behavior into Collaborative Planning
Ece Kamar, Ya'akov Gal, Barbara J. Grosz
In Proceedings of AAMAS 2009 | May 2009
-
Modeling User Perception of Interaction Opportunities for Effective Teamwork
Ece Kamar, Ya’akov Gal, Barbara J. Grosz
In Proceedings of IEEE SocialCom 2009 | January 2009
-
Mobile Opportunistic Commerce: Mechanisms, Architecture, and Application
Ece Kamar, Eric Horvitz, Chris Meek
In Proceedings of AAMAS 2008 | May 2008
ISBN: 978-0-9817381-1-6
-
Modeling User Perception of Interaction Opportunities in Collaborative Human-Computer Settings
Ece Kamar, Barbara J Grosz, David Sarne
In Proceedings of AAAI 2007, Student Abstract and Poster Program | July 2007
-
Applying MDP Approaches For Estimating Outcome of Interaction in Collaborative Human-Computer Settings
Ece Kamar, Barbara J. Grosz
In Proceedings of Workshop on Multi-agent Sequential Decision Making in Uncertain Domains (MSDM) 2007 | January 2007
-
-
-
Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions
John Chung, Ece Kamar, Saleema Amershi
ACL 2023 | July 2023
-
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, Ece Kamar
ACL 2022 | May 2022
-
-
-
Investigations of Performance and Bias in Human-AI Teamwork in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Ece Kamar
AAAI 2022 | February 2022
-
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, Daniel S. Weld
CHI | May 2021
-
Designing an Online Infrastructure for Collecting AI Data From People With Disabilities
Joon Sung Park, Danielle Bragg, Ece Kamar, Meredith Ringel Morris
FAccT | March 2021
-
Toward Fairness in AI for People with Disabilities: A Research Roadmap
Anhong Guo, Ece Kamar, Jennifer Wortman Vaughan, Hanna Wallach, Meredith Ringel Morris
ASSETS 2019 Workshop on AI Fairness for People with Disabilities | October 2019
-
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Gagan Bansal, Besmira Nushi, Ece Kamar, Walter Lasecki, Dan Weld, Eric Horvitz
HCOMP 2019 | October 2019
-
What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Siddharth Suri, Ece Kamar
7th Conference on Human Computation and Crowdsourcing (HCOMP 2019) | September 2019
-
Studying Preferences and Concerns about Information Disclosure in Email Notifications
Yongsung Kim, Adam Fourney, Ece Kamar
The Web Conference (WWW '19) | May 2019
-
Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Gagan Bansal, Besmira Nushi, Ece Kamar, Dan Weld, Walter Lasecki, Eric Horvitz
AAAI Conference on Artificial Intelligence | January 2019
-
Evaluating and Complementing Vision-to-Language Technology for People who are Blind with Conversational Crowdsourcing
Elliot Salisbury, Ece Kamar, Meredith Ringel Morris
Proceedings of IJCAI 2018 | July 2018
-
Toward Scalable Social Alt Text: Conversational Crowdsourcing as a Tool for Refining Vision-to-Language Technology for the Blind
Elliot Salisbury, Ece Kamar, Meredith Ringel Morris
Proceedings of HCOMP 2017 | October 2017
Best Paper Award
-
Revolt: Collaborative Crowdsourcing for Labeling Machine Learning Datasets
Joseph Chee Chang, Saleema Amershi, Ece Kamar
Proceedings of the Conference on Human Factors in Computing Systems (CHI 2017) | May 2017
-
Information Extraction and Manipulation Threats in Crowd-Powered Systems
Walter S. Lasecki, Jaime Teevan, Ece Kamar
CSCW 2014 | February 2014
-
Why Stop Now? Predicting Worker Engagement in Online Crowdsourcing
Andrew Mao, Ece Kamar, Eric Horvitz
First AAAI Conference on Human Computation and Crowdsourcing | March 2013
-
Combining Human and Machine Intelligence in Large-scale Crowdsourcing
Ece Kamar, Severin Hacker, Eric Horvitz
AAMAS | January 2012
-
-
-
Understanding Failures of Deep Networks via Robust Feature Extraction
Sahil Singla, Besmira Nushi, Shital Shah, Ece Kamar, Eric Horvitz
CVPR 2021 | June 2021
-
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems.
Besmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
AAAI 2017 | January 2017
-
Gauss Meets Canadian Traveler: Shortest-Path Problems with Correlated Natural Dynamics
Debadeepta Dey, Andrey Kolobov, Rich Caruana, Ece Kamar, Eric Horvitz, Ashish Kapoor
AAMAS 2014 | May 2014
-
-
-
Understanding the Representation and Representativeness of Age in AI Data Sets
Joon Sung Park, Michael S. Bernstein, Robin N. Brewer, Ece Kamar, Meredith Ringel Morris
2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21) | May 2021
-
What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Siddharth Suri, Ece Kamar
7th Conference on Human Computation and Crowdsourcing (HCOMP 2019) | September 2019
-
Why Stop Now? Predicting Worker Engagement in Online Crowdsourcing
Andrew Mao, Ece Kamar, Eric Horvitz
First AAAI Conference on Human Computation and Crowdsourcing | March 2013
-
Volunteering Versus Work for Pay: Incentives and Tradeoffs In Crowdsourcing
Andrew Mao, Ece Kamar, Yiling Chen, Eric Horvitz, Megan E Schwamb, Chris J Lintott, Arfon M Smith
First AAAI Conference on Human Computation and Crowdsourcing | March 2013
-
Jogger: Models for Context-Sensitive Reminding
The Tenth International Conference on Autonomous Agents and Multiagent Systems | January 2011
Short Paper
-
-
-
Designing an Online Infrastructure for Collecting AI Data From People With Disabilities
Joon Sung Park, Danielle Bragg, Ece Kamar, Meredith Ringel Morris
FAccT | March 2021
-
Toward Fairness in AI for People with Disabilities: A Research Roadmap
Anhong Guo, Ece Kamar, Jennifer Wortman Vaughan, Hanna Wallach, Meredith Ringel Morris
ASSETS 2019 Workshop on AI Fairness for People with Disabilities | October 2019
-
-
-
Security and Machine Learning in the Real World
Ivan Evtimov, Weidong Cui, Ece Kamar, Emre Kiciman, Tadayoshi Kohno, Jerry Li
July 2020
-
What Went Wrong and Why? Diagnosing Situated Interaction Failures in the Wild
Sean Andrist, Dan Bohus, Ece Kamar, Eric Horvitz
ICSR 2017: Social Robotics | July 2017
-
Generating Shared Transportation Plans Under Varying Preferences: Ridesharing Models and Mechanisms
MSR-TR-2009-2011 | March 2009
-
-
-
Software Engineering for Machine Learning: A Case Study
Saleema Amershi, Andrew Begel, Christian Bird, Robert DeLIne, Harald Gall, Ece Kamar, Nachi Nagappan, Besmira Nushi, Tom Zimmermann
International Conference on Software Engineering (ICSE 2019) - Software Engineering in Practice track | May 2019
ICSE 2019 Best Paper Award
-
-
-
Communicating Context to the Crowd for Complex Writing Tasks
Niloufar Salehi, Jaime Teevan, Shamsi Iqbal, Ece Kamar
Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work (CSCW 2017), Portland, OR | February 2017
Honorable Mention
-
Preserving Privacy in Crowd-Powered Systems
Walter S. Lasecki, Mitchell Gordon, Jaime Teevan, Ece Kamar, Jeff P. Bigham
In Proceedings of the Workshop on Human-Agent Interaction Design and Models (HAIDM 2015) at AAMAS 2015 | January 2015
-
-
-
The Cost of Asking Crowd Workers to Behave Maliciously
Walter. S. Lasecki, Jaime Teevan, Ece Kamar
In Proceedings of the Workshop on Human-Agent Interaction Design and Models (HAIDM 2015) at AAMAS 2015 | February 2015
-
Preserving Privacy in Crowd-Powered Systems
Walter S. Lasecki, Mitchell Gordon, Jaime Teevan, Ece Kamar, Jeff P. Bigham
In Proceedings of the Workshop on Human-Agent Interaction Design and Models (HAIDM 2015) at AAMAS 2015 | January 2015
-
-
-
Hybrid Intelligence and the Future of Work
In Proceedings of the Productivity Decomposed: Getting Big Things Done with Little Microtasks Workshop at CHI 2016 | May 2016
-
Clustering for Set Partitioning: A Case Study in Carpooling
Cathy Wu, Ece Kamar, Eric Horvitz
In Proceedings of the Workshop on Optimization for Machine Learning (OPT) at NIPS 2015. | May 2015
-
Light at the End of the Tunnel: A Monte Carlo Approach to Computing Value of Information
Twelth International Conference on Autonomous Agents and Mutli-agent Systems | May 2013
-
-
-
Why Stop Now? Predicting Worker Engagement in Online Crowdsourcing
Andrew Mao, Ece Kamar, Eric Horvitz
First AAAI Conference on Human Computation and Crowdsourcing | March 2013
-
Volunteering Versus Work for Pay: Incentives and Tradeoffs In Crowdsourcing
Andrew Mao, Ece Kamar, Yiling Chen, Eric Horvitz, Megan E Schwamb, Chris J Lintott, Arfon M Smith
First AAAI Conference on Human Computation and Crowdsourcing | March 2013
-
-
-
Investigation of Principles of Context-Sensitive Reminding
MSR-TR-2010-174 | October 2010
-
-
-
The Collaboration Gap
Tim R. Davidson, Adam Fourney, Saleema Amershi, Robert West, Eric Horvitz, Ece Kamar
November 2025
-
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, Yi Zhang
March 2023
-
Avoiding Negative Side Effects of Autonomous Systems in the Open World
Sandhya Saisubramanian, Ece Kamar, Shlomo Zilberstein
May 2022
-
Algorithmic Recourse in the Wild: Understanding the Impact of Data and Model Shifts
Kaivalya Rawal, Ece Kamar, Himabindu Lakkaraju
June 2021
-
A Bayesian Approach to Identifying Representational Errors
Ramya Ramakrishnan, Vaibhav Unhelkar, Ece Kamar, Julie Shah
March 2021
-
-
-
Magentic-UI: Towards Human-in-the-loop Agentic Systems
Hussein Mozannar, Gagan Bansal, Cheng Tan, Adam Fourney, Victor Dibia, Jingya Chen, Jack Gerrits, Tyler Payne, Matheus Kunzler Maldaner, Madeleine Grunde-McLaughlin, Eric Zhu, Griffin Bassman, Jacob Alber, Peter Chang, Ricky Loynd, Friederike Niedtner, Ece Kamar, Maya Murad, Rafah Hosn, Saleema Amershi
MSR-TR-2025-40 | July 2025
作者：Microsoft
-
Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks
Adam Fourney, Gagan Bansal, Hussein Mozannar, Cheng Tan, Eduardo Salinas, Erkang (Eric) Zhu, Friederike Niedtner, Grace Proebsting, Griffin Bassman, Jack Gerrits, Jacob Alber, Peter Chang, Ricky Loynd, Robert West, Victor Dibia, Ahmed Awadallah, Ece Kamar, Rafah Hosn, Saleema Amershi
MSR-TR-2024-47 | November 2024
作者：Microsoft
-
Benchmarking Spatial Relationships in Text-to-Image Generation
Tejas Gokhale, Hamid Palangi, Besmira Nushi, Vibhav Vineet, Eric Horvitz, Ece Kamar, Chitta Baral, Yezhou Yang
MSR-TR-2023-44 | May 2023
作者：Microsoft
-
Incentives and Truthful Reporting in Consensus-centric Crowdsourcing
MSR-TR-2012-16 | February 2012
-
Investigation of Principles of Context-Sensitive Reminding
MSR-TR-2010-174 | October 2010
-
Generating Shared Transportation Plans Under Varying Preferences: Ridesharing Models and Mechanisms
MSR-TR-2009-2011 | March 2009
-
-
-
Teaching Language Models to Hallucinate Less with Synthetic Tasks
Erik Jones, Hamid Palangi, Clarisse Simoes Ribeiro, Varun Chandrasekaran, Subhabrata (Subho) Mukherjee, Arindam Mitra, Ahmed Awadallah, Ece Kamar
ICLR 2024 | October 2023
-
Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions
John Chung, Ece Kamar, Saleema Amershi
ACL 2023 | July 2023
-
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, Ece Kamar
ACL 2022 | May 2022
-
Investigations of Performance and Bias in Human-AI Teamwork in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Ece Kamar
AAAI 2022 | February 2022
-
Understanding Failures of Deep Networks via Robust Feature Extraction
Sahil Singla, Besmira Nushi, Shital Shah, Ece Kamar, Eric Horvitz
CVPR 2021 | June 2021
-
Understanding the Representation and Representativeness of Age in AI Data Sets
Joon Sung Park, Michael S. Bernstein, Robin N. Brewer, Ece Kamar, Meredith Ringel Morris
2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21) | May 2021
-
Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs
Solon Barocas, Anhong Guo, Ece Kamar, Jacquelyn Krones, Meredith Ringel Morris, Jennifer Wortman Vaughan, Duncan Wadsworth, Hanna Wallach
AIES 2021 | May 2021
-
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, Daniel S. Weld
CHI | May 2021
-
Designing an Online Infrastructure for Collecting AI Data From People With Disabilities
Joon Sung Park, Danielle Bragg, Ece Kamar, Meredith Ringel Morris
FAccT | March 2021
-
Improving the Performance-Compatibility Tradeoff with Personalized Objective Functions
Jonathan Martinez, Kobi Gal, Ece Kamar, Levi H. S. Lelis
The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21) | February 2021
-
Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork
Gagan Bansal, Besmira Nushi, Ece Kamar, Eric Horvitz, Dan Weld
AAAI 2021 | February 2021
-
Learning to Complement Humans
Bryan Wilder, Eric Horvitz, Ece Kamar
IJCAI 2020 | January 2021
-
An Empirical Analysis of Backward Compatibility in Machine Learning Systems
Megha Srivastava, Besmira Nushi, Ece Kamar, Shital Shah, Eric Horvitz
KDD | August 2020
-
SQuINTing at VQA Models: Introspecting VQA Models with Sub-Questions
Ramprasaath R. Selvaraju, Purva Tendulkar, Devi Parikh, Eric Horvitz, Marco Tulio Ribeiro, Besmira Nushi, Ece Kamar
CVPR 2020 | June 2020
-
Toward Fairness in AI for People with Disabilities: A Research Roadmap
Anhong Guo, Ece Kamar, Jennifer Wortman Vaughan, Hanna Wallach, Meredith Ringel Morris
ASSETS 2019 Workshop on AI Fairness for People with Disabilities | October 2019
-
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Gagan Bansal, Besmira Nushi, Ece Kamar, Walter Lasecki, Dan Weld, Eric Horvitz
HCOMP 2019 | October 2019
-
What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring
Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Siddharth Suri, Ece Kamar
7th Conference on Human Computation and Crowdsourcing (HCOMP 2019) | September 2019
-
Software Engineering for Machine Learning: A Case Study
Saleema Amershi, Andrew Begel, Christian Bird, Robert DeLIne, Harald Gall, Ece Kamar, Nachi Nagappan, Besmira Nushi, Tom Zimmermann
International Conference on Software Engineering (ICSE 2019) - Software Engineering in Practice track | May 2019
ICSE 2019 Best Paper Award
-
Studying Preferences and Concerns about Information Disclosure in Email Notifications
Yongsung Kim, Adam Fourney, Ece Kamar
The Web Conference (WWW '19) | May 2019
-
People and AI See Things Different Implications of Mismatched Perception on HCI for AI Systems
Saleema Amershi, Ece Kamar, Emre Kiciman
Workshop on Human-Centered Machine Learning Perspectives at CHI 2019 | May 2019
-
Overcoming Blind Spots in the Real World: Leveraging Complementary Abilities for Joint Execution
Ramya Ramakrishnan, Ece Kamar, Besmira Nushi, Debadeepta Dey, Julie Shah, Eric Horvitz
AAAI Conference on Artificial Intelligence | January 2019
-
Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Gagan Bansal, Besmira Nushi, Ece Kamar, Dan Weld, Walter Lasecki, Eric Horvitz
AAAI Conference on Artificial Intelligence | January 2019
-
Evaluating and Complementing Vision-to-Language Technology for People who are Blind with Conversational Crowdsourcing
Elliot Salisbury, Ece Kamar, Meredith Ringel Morris
Proceedings of IJCAI 2018 | July 2018
-
Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure
Besmira Nushi, Ece Kamar, Eric Horvitz
HCOMP 2018 | July 2018
-
Discovering Blind Spots in Reinforcement Learning
Ramya Ramakrishnan, Ece Kamar, Debadeepta Dey, Julie Shah, Eric Horvitz
17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2018) | May 2018
-
Optimizing Interventions via Offline Policy Evaluation: Studies in Citizen Science
Avi Segal, Kobi Gal, Ece Kamar, Eric Horvitz
AAAI-18: Thirty-Second AAAI Conference on Artificial Intelligence | January 2018
-
Toward Scalable Social Alt Text: Conversational Crowdsourcing as a Tool for Refining Vision-to-Language Technology for the Blind
Elliot Salisbury, Ece Kamar, Meredith Ringel Morris
Proceedings of HCOMP 2017 | October 2017
Best Paper Award
-
What Went Wrong and Why? Diagnosing Situated Interaction Failures in the Wild
Sean Andrist, Dan Bohus, Ece Kamar, Eric Horvitz
ICSR 2017: Social Robotics | July 2017
-
Revolt: Collaborative Crowdsourcing for Labeling Machine Learning Datasets
Joseph Chee Chang, Saleema Amershi, Ece Kamar
Proceedings of the Conference on Human Factors in Computing Systems (CHI 2017) | May 2017
-
Communicating Context to the Crowd for Complex Writing Tasks
Niloufar Salehi, Jaime Teevan, Shamsi Iqbal, Ece Kamar
Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work (CSCW 2017), Portland, OR | February 2017
Honorable Mention
-
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems.
Besmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
AAAI 2017 | January 2017
-
Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration
Himabindu Lakkaraju, Ece Kamar, Rich Caruana, Eric Horvitz
Thirty-First AAAI Conference on Artificial Intelligence | December 2016
-
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems
Besmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
Thirty-First AAAI Conference on Artificial Intelligence | November 2016
-
Hybrid Intelligence and the Future of Work
In Proceedings of the Productivity Decomposed: Getting Big Things Done with Little Microtasks Workshop at CHI 2016 | May 2016
-
Toward a Learning Science for Complex Crowdsourcing Tasks
Shayan Doroudi, Ece Kamar, Emma Brunskill, Eric Horvitz
Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems | May 2016
-
Interactive Teaching Strategies for Agent Training
Ofra Amir, Ece Kamar, Andrey Kolobov, Barbara Grosz
IJCAI 2016 | May 2016
-
Intervention Strategies for Increasing Engagement in Crowdsourcing: Platform, Predictions, and Experiments
Avi Segal, Ya’akov (Kobi) Gal, Ece Kamar, Eric Horvitz, Alex Bowyer, Grant Miller
Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16) | April 2016
-
Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence
Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16) | March 2016
-
Identifying and Accounting for Task-Dependent Bias in Crowdsourcing
Ece Kamar, Ashish Kapoor, Eric Horvitz
In Proceedings of HCOMP 2015 | August 2015
-
Metareasoning for Planning Under Uncertainty
Christopher H. Lin, Andrey Kolobov, Ece Kamar, Eric Horvitz
IJCAI 2015 | July 2015
-
Clustering for Set Partitioning: A Case Study in Carpooling
Cathy Wu, Ece Kamar, Eric Horvitz
In Proceedings of the Workshop on Optimization for Machine Learning (OPT) at NIPS 2015. | May 2015
-
Planning for Crowdsourcing Hierarchical Tasks
Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2015), Bordini, Elkind, Weiss, Yolum (eds.), May 4-8, 2015, Istanbul, Turkey. | May 2015
-
The Cost of Asking Crowd Workers to Behave Maliciously
Walter. S. Lasecki, Jaime Teevan, Ece Kamar
In Proceedings of the Workshop on Human-Agent Interaction Design and Models (HAIDM 2015) at AAMAS 2015 | February 2015
-
Preserving Privacy in Crowd-Powered Systems
Walter S. Lasecki, Mitchell Gordon, Jaime Teevan, Ece Kamar, Jeff P. Bigham
In Proceedings of the Workshop on Human-Agent Interaction Design and Models (HAIDM 2015) at AAMAS 2015 | January 2015
-
Signals in the Silence: Models of Implicit Feedback in a Recommendation System for Crowdsourcing
Christopher Lin, Ece Kamar, Eric Horvitz
In Proceedings of AAAI 2014 | June 2014
-
Crowdsourcing Language Generation Templates for Dialogue Systems
Margaret Mitchell, Dan Bohus, Ece Kamar
Proceedings of the INLG and SIGDIAL 2014 Joint Session | June 2014
ISBN: 978-1-941643-23-5
-
Gauss Meets Canadian Traveler: Shortest-Path Problems with Correlated Natural Dynamics
Debadeepta Dey, Andrey Kolobov, Rich Caruana, Ece Kamar, Eric Horvitz, Ashish Kapoor
AAMAS 2014 | May 2014
-
Stochastic Privacy
Adish Singla, Eric Horvitz, Ece Kamar, Ryen W. White
April 2014
-
Information Extraction and Manipulation Threats in Crowd-Powered Systems
Walter S. Lasecki, Jaime Teevan, Ece Kamar
CSCW 2014 | February 2014
-
Lifelong Learning for Acquiring the Wisdom of the Crowd
Ece Kamar, Ashish Kapoor, Eric Horvitz
IJCAI '13 Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence | August 2013
-
Light at the End of the Tunnel: A Monte Carlo Approach to Computing Value of Information
Twelth International Conference on Autonomous Agents and Mutli-agent Systems | May 2013
-
Look versus Leap: Computing Value of Information with High-Dimensional Streaming Evidence
Stephanie Rosenthal, Dan Bohus, Ece Kamar, Eric Horvitz
International Joint Conference on Artificial Intelligence | April 2013
-
Why Stop Now? Predicting Worker Engagement in Online Crowdsourcing
Andrew Mao, Ece Kamar, Eric Horvitz
First AAAI Conference on Human Computation and Crowdsourcing | March 2013
-
Volunteering Versus Work for Pay: Incentives and Tradeoffs In Crowdsourcing
Andrew Mao, Ece Kamar, Yiling Chen, Eric Horvitz, Megan E Schwamb, Chris J Lintott, Arfon M Smith
First AAAI Conference on Human Computation and Crowdsourcing | March 2013
-
Conversations in the Crowd: Collecting Data for Task-Oriented Dialog Learning
Walter Lasecki, Ece Kamar, Dan Bohus
In Proceedings of the Human Computation Workshop on Scaling Speech and Language Understanding and Dialog through Crowdsourcing at HCOMP 2013. | January 2013
-
Crowdsourcing the Acquisition of Natural Language Corpora: Methods and Observations
William Yang Wang, Dan Bohus, Ece Kamar, Eric Horvitz
October 2012
-
Towards Situated Collaboration
Dan Bohus, Ece Kamar, Eric Horvitz
In NAACL Workshop on Future Directions and Challenges in Spoken Dialog Systems: Tools and Data | April 2012
-
Incentives for Truthful Reporting in Crowdsourcing
AAMAS | January 2012
-
Combining Human and Machine Intelligence in Large-scale Crowdsourcing
Ece Kamar, Severin Hacker, Eric Horvitz
AAMAS | January 2012
-
MSRC at TREC 2011 Crowdsourcing Track
Paul Bennett, Ece Kamar, Gabriella Kazai
In Proceedings of TREC 2011 | January 2011
-
Jogger: Models for Context-Sensitive Reminding
The Tenth International Conference on Autonomous Agents and Multiagent Systems | January 2011
Short Paper
-
Collaboration and Shared Plans in the Open World: Studies of Ridesharing
IJCAI'09 Proceedings of the 21st International Joint Conference on Artificial Intelligence | July 2009
-
Managing Helpful Behavior in Collaborative Activities of Heterogeneous Agent Groups
Twenty-First IAAI Conference | June 2009
-
Incorporating Helpful Behavior into Collaborative Planning
Ece Kamar, Ya'akov Gal, Barbara J. Grosz
In Proceedings of AAMAS 2009 | May 2009
-
Modeling User Perception of Interaction Opportunities for Effective Teamwork
Ece Kamar, Ya’akov Gal, Barbara J. Grosz
In Proceedings of IEEE SocialCom 2009 | January 2009
-
Mobile Opportunistic Commerce: Mechanisms, Architecture, and Application
Ece Kamar, Eric Horvitz, Chris Meek
In Proceedings of AAMAS 2008 | May 2008
ISBN: 978-0-9817381-1-6
-
Modeling User Perception of Interaction Opportunities in Collaborative Human-Computer Settings
Ece Kamar, Barbara J Grosz, David Sarne
In Proceedings of AAAI 2007, Student Abstract and Poster Program | July 2007
-
Applying MDP Approaches For Estimating Outcome of Interaction in Collaborative Human-Computer Settings
Ece Kamar, Barbara J. Grosz
In Proceedings of Workshop on Multi-agent Sequential Decision Making in Uncertain Domains (MSDM) 2007 | January 2007
-
-
-
Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models
Mert Yuksekgonul, Varun Chandrasekaran, Erik Jones, Suriya Gunasekar, Ranjita Naik, Hamid Palangi, Ece Kamar, Besmira Nushi
September 2023
-
Avoiding Negative Side Effects due to Incomplete Knowledge of AI Systems
Sandhya Saisubramanian, Shlomo Zilberstein, Ece Kamar
October 2021
-
Security and Machine Learning in the Real World
Ivan Evtimov, Weidong Cui, Ece Kamar, Emre Kiciman, Tadayoshi Kohno, Jerry Li
July 2020
-
-
-
Modeling Information Exchange Opportunities For Effective Human-Computer Teamwork
Ece Kamar, Ya’akov (Kobi) Gal, Barbara J. Grosz
Artificial Intelligence Journal | February 2013
-
Computation and Crowds: Models for Dynamic Ridesharing
Collaboration and Shared Plans in the Open World: Studies of Ridesharing, IJCAI 2009, Pasadena, CA | July 2009
-
-