Dehumanizing Machines: Mitigating Anthropomorphic Behaviors in Text Generation Systems
Myra Cheng, Su Lin Blodgett, Alicia DeVrio, Lisa Egede, Alexandra Olteanu
2025 Meeting of the Association for Computational Linguistics | July 2025
SAC Highlight
Su Lin Blodgett
Principal Researcher
Publications
-
-
-
Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
-
Measuring Machine Learning Harms from Stereotypes: Requires Understanding Who is Being Harmed by Which Errors in What Ways
Angelina Wang, Xuechunzi Bai, Solon Barocas, Su Lin Blodgett
2025 ACM Conference on Fairness, Accountability, and Transparency | June 2025
-
Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor
Alexandra Olteanu, Su Lin Blodgett, Agathe Balayn, Angelina Wang, Fernando Diaz, Flavio du Pin Calmon, Margaret Mitchell, Michael Ekstrand, Reuben Binns, Solon Barocas
NeurIPS 2025 | June 2025
-
“It was 80% me, 20% AI”: Seeking Authenticity in Co-Writing with Large Language Models
Angel Hsing-Chi Hwang, Q. Vera Liao, Su Lin Blodgett, Alexandra Olteanu, Adam Trischler
2025 Conference on Computer Supported Cooperative Work | May 2025
-
A Taxonomy of Linguistic Expressions That Contribute To Anthropomorphism of Language Technologies
Alicia DeVrio, Myra Cheng, Lisa Egede, Alexandra Olteanu, Su Lin Blodgett
CHI 2025 | April 2025
-
“I Am the One and Only, Your Cyber BFF”: Understanding the Impact of GenAI Requires Understanding the Impact of Anthropomorphic AI
Myra Cheng, Alicia DeVrio, Lisa Egede, Su Lin Blodgett, Alexandra Olteanu
April 2025
-
AI Automatons: AI Systems Intended to Imitate Humans
Alexandra Olteanu, Solon Barocas, Su Lin Blodgett, Lisa Egede, Alicia DeVrio, Myra Cheng
March 2025
-
Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
ICML 2025 | January 2025
-
-
-
Microsoft New Future of Work Report 2024
Jenna Butler, Mihaela Vorvoreanu, Rebecca Janssen, Abigail Sellen, Nicole Immorlica, Adam Troy, Advait Sarkar, Alex Farach, Alex Chouldechova, Alexandra Olteanu, Alexia Cambon, Arjun Radhakrishna, Asta Roseway, Ben Zorn, Brent Hecht, Daniel G. Goldstein, Dhruv Joshi, Ed Cutrell, Emre Kiciman, Gonzalo Ramos, Gustavo Soares, Hanna Wallach, Ian Drosos, Jack Williams (johnwilliams), Jacki O'Neill, Jake Hofman, Jaime Teevan, Javier Hernandez, Jennifer Wortman Vaughan, Jina Suh, John Tang, Justin Edwards, Kalika Bali, Kori Inkpen, Krishna Madhavan, Laylah Bulman, Leon Reicherts, Lev Tankelevitch, Longqi Yang, Martez Mott, Millicent Ochieng, Mercy Muchai, Nancy Baym, Najeeb Abdulhamid, Nicolai Marquardt, Ken Hinckley, Michael Bentley, Dave Brown, Hugo Romat, Nathalie Henry Riche, Samuel Maina, Shamsi Iqbal, Siân Lindley, Stephanie Nyairo, Su Lin Blodgett, Sumit Gulwani, Sunayana Sitaram, Vu Le
MSR-TR-2024-56 | December 2024
Published by Microsoft
-
Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
November 2024
-
Evaluating Generative AI Systems is a Social Science Measurement Challenge
Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
November 2024
-
Metrics for What, Metrics for Whom: Assessing Actionability of Bias Evaluation Metrics in NLP
Pieter Delobelle, Giuseppe Attanasio, Debora Nozza, Su Lin Blodgett, Zeerak Talat
2024 Empirical Methods in Natural Language Processing | November 2024
-
ECBD: Evidence-Centered Benchmark Design for NLP
Yu Lu Liu, Su Lin Blodgett, Jackie Cheung, Q. Vera Liao, Alexandra Olteanu, Ziang Xiao
2024 Meeting of the Association for Computational Linguistics | August 2024
-
Understanding the Impacts of Language Technologies’ Performance Disparities on African American Language Speakers
Jay Cunningham, Su Lin Blodgett, Hal Daumé III, Christina Harrington, Hanna Wallach, Michael Madaio
Findings of the Association for Computational Linguistics: ACL 2024 | August 2024
-
The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels
Eve Fleisig, Su Lin Blodgett, Dan Klein, Zeerak Talat
NAACL 2024 | June 2024
-
“One-Size-Fits-All”? Examining Expectations around What Constitute “Fair” or “Good” NLG System Behaviors
Li Lucy, Su Lin Blodgett, Milad Shokouhi, Hanna Wallach, Alexandra Olteanu
NAACL 2024 | June 2024
-
-
-
Responsible AI Considerations in Text Summarization Research: A Review of Current Practices
Yu Lu Liu, Meng Cao, Su Lin Blodgett, Jackie Chi Kit Cheung, Alexandra Olteanu, Adam Trischler
Findings of EMNLP 2023 | December 2023
-
FairPrism: Evaluating Fairness-Related Harms in Text Generation
Eve Fleisig, Aubrie Amstutz, Chad Atalla, Su Lin Blodgett, Hal Daumé III, Alexandra Olteanu, Emily Sheng, Dan Vann, Hanna Wallach
ACL 2023 | July 2023
-
It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance
Arjun Subramonian, Xingdi Yuan, Hal Daumé III, Su Lin Blodgett
Findings of ACL 2023 | July 2023
-
This Prompt is Measuring : Evaluating Bias Evaluation in Language Models
Seraphina Goldfarb-Tarrant, Eddie Ungless, Esma Balkir, Su Lin Blodgett
Findings of the Association for Computational Linguistics: ACL 2023 | July 2023
-
Taxonomizing and Measuring Representational Harms: A Look at Image Tagging
Jared Katzman, Angelina Wang, Morgan Scheuerman, Su Lin Blodgett, Kristen Laird, Hanna Wallach, Solon Barocas
AAAI 2022 | February 2023
-
-
-
Examining Responsibility and Deliberation in AI Impact Statements and Ethics Reviews
David Liu, Priyanka Nanayakkara, Sarah Ariyan Sakha, Grace Abuhamad, Su Lin Blodgett, Nicholas Diakopoulos, Jessica R. Hullman, Tina Eliassi-Rad
AIES 2022 | July 2022
-
Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications
Kaitlyn Zhou, Su Lin Blodgett, Adam Trischler, Hal Daumé III, Kaheer Suleman, Alexandra Olteanu
NAACL 2022 | May 2022
-
Responsible Language Technologies: Foreseeing and Mitigating Harms
Su Lin Blodgett, Q. Vera Liao, Alexandra Olteanu, Rada Mihalcea, Michael Muller, Morgan Klaus Scheuerman, Chenhao Tan, Qian Yang
CHI 2022 | April 2022
-
-
-
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets
Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, Hanna Wallach
ACL-IJCNLP 2021 | August 2021
-
A Survey of Race, Racism, and Anti-Racism in NLP
Anjalie Field, Su Lin Blodgett, Zeerak Waseem, Yulia Tsvetkov
ACL-IJCNLP 2021 | July 2021
-
Bridging Human–Computer Interaction and Natural Language Processing
Su Lin Blodgett, Michael Madaio, Brendan O’Connor, Hanna Wallach, Qian Yang
EACL 2021 Workshop | April 2021
-
-
-
Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor
Alexandra Olteanu, Su Lin Blodgett, Agathe Balayn, Angelina Wang, Fernando Diaz, Flavio du Pin Calmon, Margaret Mitchell, Michael Ekstrand, Reuben Binns, Solon Barocas
NeurIPS 2025 | June 2025
-
A Taxonomy of Linguistic Expressions That Contribute To Anthropomorphism of Language Technologies
Alicia DeVrio, Myra Cheng, Lisa Egede, Alexandra Olteanu, Su Lin Blodgett
CHI 2025 | April 2025
-
“I Am the One and Only, Your Cyber BFF”: Understanding the Impact of GenAI Requires Understanding the Impact of Anthropomorphic AI
Myra Cheng, Alicia DeVrio, Lisa Egede, Su Lin Blodgett, Alexandra Olteanu
April 2025
-
Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
ICML 2025 | January 2025
-
Microsoft New Future of Work Report 2024
Jenna Butler, Mihaela Vorvoreanu, Rebecca Janssen, Abigail Sellen, Nicole Immorlica, Adam Troy, Advait Sarkar, Alex Farach, Alex Chouldechova, Alexandra Olteanu, Alexia Cambon, Arjun Radhakrishna, Asta Roseway, Ben Zorn, Brent Hecht, Daniel G. Goldstein, Dhruv Joshi, Ed Cutrell, Emre Kiciman, Gonzalo Ramos, Gustavo Soares, Hanna Wallach, Ian Drosos, Jack Williams (johnwilliams), Jacki O'Neill, Jake Hofman, Jaime Teevan, Javier Hernandez, Jennifer Wortman Vaughan, Jina Suh, John Tang, Justin Edwards, Kalika Bali, Kori Inkpen, Krishna Madhavan, Laylah Bulman, Leon Reicherts, Lev Tankelevitch, Longqi Yang, Martez Mott, Millicent Ochieng, Mercy Muchai, Nancy Baym, Najeeb Abdulhamid, Nicolai Marquardt, Ken Hinckley, Michael Bentley, Dave Brown, Hugo Romat, Nathalie Henry Riche, Samuel Maina, Shamsi Iqbal, Siân Lindley, Stephanie Nyairo, Su Lin Blodgett, Sumit Gulwani, Sunayana Sitaram, Vu Le
MSR-TR-2024-56 | December 2024
Published by Microsoft
-
Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
November 2024
-
Evaluating Generative AI Systems is a Social Science Measurement Challenge
Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
November 2024
-
FairPrism: Evaluating Fairness-Related Harms in Text Generation
Eve Fleisig, Aubrie Amstutz, Chad Atalla, Su Lin Blodgett, Hal Daumé III, Alexandra Olteanu, Emily Sheng, Dan Vann, Hanna Wallach
ACL 2023 | July 2023
-
Taxonomizing and Measuring Representational Harms: A Look at Image Tagging
Jared Katzman, Angelina Wang, Morgan Scheuerman, Su Lin Blodgett, Kristen Laird, Hanna Wallach, Solon Barocas
AAAI 2022 | February 2023
-
Examining Responsibility and Deliberation in AI Impact Statements and Ethics Reviews
David Liu, Priyanka Nanayakkara, Sarah Ariyan Sakha, Grace Abuhamad, Su Lin Blodgett, Nicholas Diakopoulos, Jessica R. Hullman, Tina Eliassi-Rad
AIES 2022 | July 2022
-
A Survey of Race, Racism, and Anti-Racism in NLP
Anjalie Field, Su Lin Blodgett, Zeerak Waseem, Yulia Tsvetkov
ACL-IJCNLP 2021 | July 2021
-
-
-
Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
-
Dehumanizing Machines: Mitigating Anthropomorphic Behaviors in Text Generation Systems
Myra Cheng, Su Lin Blodgett, Alicia DeVrio, Lisa Egede, Alexandra Olteanu
2025 Meeting of the Association for Computational Linguistics | July 2025
SAC Highlight
-
Measuring Machine Learning Harms from Stereotypes: Requires Understanding Who is Being Harmed by Which Errors in What Ways
Angelina Wang, Xuechunzi Bai, Solon Barocas, Su Lin Blodgett
2025 ACM Conference on Fairness, Accountability, and Transparency | June 2025
-
“It was 80% me, 20% AI”: Seeking Authenticity in Co-Writing with Large Language Models
Angel Hsing-Chi Hwang, Q. Vera Liao, Su Lin Blodgett, Alexandra Olteanu, Adam Trischler
2025 Conference on Computer Supported Cooperative Work | May 2025
-
“I Am the One and Only, Your Cyber BFF”: Understanding the Impact of GenAI Requires Understanding the Impact of Anthropomorphic AI
Myra Cheng, Alicia DeVrio, Lisa Egede, Su Lin Blodgett, Alexandra Olteanu
April 2025
-
AI Automatons: AI Systems Intended to Imitate Humans
Alexandra Olteanu, Solon Barocas, Su Lin Blodgett, Lisa Egede, Alicia DeVrio, Myra Cheng
March 2025
-
Metrics for What, Metrics for Whom: Assessing Actionability of Bias Evaluation Metrics in NLP
Pieter Delobelle, Giuseppe Attanasio, Debora Nozza, Su Lin Blodgett, Zeerak Talat
2024 Empirical Methods in Natural Language Processing | November 2024
-
Understanding the Impacts of Language Technologies’ Performance Disparities on African American Language Speakers
Jay Cunningham, Su Lin Blodgett, Hal Daumé III, Christina Harrington, Hanna Wallach, Michael Madaio
Findings of the Association for Computational Linguistics: ACL 2024 | August 2024
-
ECBD: Evidence-Centered Benchmark Design for NLP
Yu Lu Liu, Su Lin Blodgett, Jackie Cheung, Q. Vera Liao, Alexandra Olteanu, Ziang Xiao
2024 Meeting of the Association for Computational Linguistics | August 2024
-
The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels
Eve Fleisig, Su Lin Blodgett, Dan Klein, Zeerak Talat
NAACL 2024 | June 2024
-
“One-Size-Fits-All”? Examining Expectations around What Constitute “Fair” or “Good” NLG System Behaviors
Li Lucy, Su Lin Blodgett, Milad Shokouhi, Hanna Wallach, Alexandra Olteanu
NAACL 2024 | June 2024
-
Responsible AI Considerations in Text Summarization Research: A Review of Current Practices
Yu Lu Liu, Meng Cao, Su Lin Blodgett, Jackie Chi Kit Cheung, Alexandra Olteanu, Adam Trischler
Findings of EMNLP 2023 | December 2023
-
It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance
Arjun Subramonian, Xingdi Yuan, Hal Daumé III, Su Lin Blodgett
Findings of ACL 2023 | July 2023
-
This Prompt is Measuring : Evaluating Bias Evaluation in Language Models
Seraphina Goldfarb-Tarrant, Eddie Ungless, Esma Balkir, Su Lin Blodgett
Findings of the Association for Computational Linguistics: ACL 2023 | July 2023
-
Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications
Kaitlyn Zhou, Su Lin Blodgett, Adam Trischler, Hal Daumé III, Kaheer Suleman, Alexandra Olteanu
NAACL 2022 | May 2022
-
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets
Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, Hanna Wallach
ACL-IJCNLP 2021 | August 2021
-
A Survey of Race, Racism, and Anti-Racism in NLP
Anjalie Field, Su Lin Blodgett, Zeerak Waseem, Yulia Tsvetkov
ACL-IJCNLP 2021 | July 2021
-
Bridging Human–Computer Interaction and Natural Language Processing
Su Lin Blodgett, Michael Madaio, Brendan O’Connor, Hanna Wallach, Qian Yang
EACL 2021 Workshop | April 2021
-
-
-
Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
-
Dehumanizing Machines: Mitigating Anthropomorphic Behaviors in Text Generation Systems
Myra Cheng, Su Lin Blodgett, Alicia DeVrio, Lisa Egede, Alexandra Olteanu
2025 Meeting of the Association for Computational Linguistics | July 2025
SAC Highlight
-
Measuring Machine Learning Harms from Stereotypes: Requires Understanding Who is Being Harmed by Which Errors in What Ways
Angelina Wang, Xuechunzi Bai, Solon Barocas, Su Lin Blodgett
2025 ACM Conference on Fairness, Accountability, and Transparency | June 2025
-
Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor
Alexandra Olteanu, Su Lin Blodgett, Agathe Balayn, Angelina Wang, Fernando Diaz, Flavio du Pin Calmon, Margaret Mitchell, Michael Ekstrand, Reuben Binns, Solon Barocas
NeurIPS 2025 | June 2025
-
“It was 80% me, 20% AI”: Seeking Authenticity in Co-Writing with Large Language Models
Angel Hsing-Chi Hwang, Q. Vera Liao, Su Lin Blodgett, Alexandra Olteanu, Adam Trischler
2025 Conference on Computer Supported Cooperative Work | May 2025
-
“I Am the One and Only, Your Cyber BFF”: Understanding the Impact of GenAI Requires Understanding the Impact of Anthropomorphic AI
Myra Cheng, Alicia DeVrio, Lisa Egede, Su Lin Blodgett, Alexandra Olteanu
April 2025
-
AI Automatons: AI Systems Intended to Imitate Humans
Alexandra Olteanu, Solon Barocas, Su Lin Blodgett, Lisa Egede, Alicia DeVrio, Myra Cheng
March 2025
-
Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
ICML 2025 | January 2025
-
Microsoft New Future of Work Report 2024
Jenna Butler, Mihaela Vorvoreanu, Rebecca Janssen, Abigail Sellen, Nicole Immorlica, Adam Troy, Advait Sarkar, Alex Farach, Alex Chouldechova, Alexandra Olteanu, Alexia Cambon, Arjun Radhakrishna, Asta Roseway, Ben Zorn, Brent Hecht, Daniel G. Goldstein, Dhruv Joshi, Ed Cutrell, Emre Kiciman, Gonzalo Ramos, Gustavo Soares, Hanna Wallach, Ian Drosos, Jack Williams (johnwilliams), Jacki O'Neill, Jake Hofman, Jaime Teevan, Javier Hernandez, Jennifer Wortman Vaughan, Jina Suh, John Tang, Justin Edwards, Kalika Bali, Kori Inkpen, Krishna Madhavan, Laylah Bulman, Leon Reicherts, Lev Tankelevitch, Longqi Yang, Martez Mott, Millicent Ochieng, Mercy Muchai, Nancy Baym, Najeeb Abdulhamid, Nicolai Marquardt, Ken Hinckley, Michael Bentley, Dave Brown, Hugo Romat, Nathalie Henry Riche, Samuel Maina, Shamsi Iqbal, Siân Lindley, Stephanie Nyairo, Su Lin Blodgett, Sumit Gulwani, Sunayana Sitaram, Vu Le
MSR-TR-2024-56 | December 2024
Published by Microsoft
-
Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
November 2024
-
Evaluating Generative AI Systems is a Social Science Measurement Challenge
Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
November 2024
-
Understanding the Impacts of Language Technologies’ Performance Disparities on African American Language Speakers
Jay Cunningham, Su Lin Blodgett, Hal Daumé III, Christina Harrington, Hanna Wallach, Michael Madaio
Findings of the Association for Computational Linguistics: ACL 2024 | August 2024
-
ECBD: Evidence-Centered Benchmark Design for NLP
Yu Lu Liu, Su Lin Blodgett, Jackie Cheung, Q. Vera Liao, Alexandra Olteanu, Ziang Xiao
2024 Meeting of the Association for Computational Linguistics | August 2024
-
-
-
“It was 80% me, 20% AI”: Seeking Authenticity in Co-Writing with Large Language Models
Angel Hsing-Chi Hwang, Q. Vera Liao, Su Lin Blodgett, Alexandra Olteanu, Adam Trischler
2025 Conference on Computer Supported Cooperative Work | May 2025
-
A Taxonomy of Linguistic Expressions That Contribute To Anthropomorphism of Language Technologies
Alicia DeVrio, Myra Cheng, Lisa Egede, Alexandra Olteanu, Su Lin Blodgett
CHI 2025 | April 2025
-
Microsoft New Future of Work Report 2024
Jenna Butler, Mihaela Vorvoreanu, Rebecca Janssen, Abigail Sellen, Nicole Immorlica, Adam Troy, Advait Sarkar, Alex Farach, Alex Chouldechova, Alexandra Olteanu, Alexia Cambon, Arjun Radhakrishna, Asta Roseway, Ben Zorn, Brent Hecht, Daniel G. Goldstein, Dhruv Joshi, Ed Cutrell, Emre Kiciman, Gonzalo Ramos, Gustavo Soares, Hanna Wallach, Ian Drosos, Jack Williams (johnwilliams), Jacki O'Neill, Jake Hofman, Jaime Teevan, Javier Hernandez, Jennifer Wortman Vaughan, Jina Suh, John Tang, Justin Edwards, Kalika Bali, Kori Inkpen, Krishna Madhavan, Laylah Bulman, Leon Reicherts, Lev Tankelevitch, Longqi Yang, Martez Mott, Millicent Ochieng, Mercy Muchai, Nancy Baym, Najeeb Abdulhamid, Nicolai Marquardt, Ken Hinckley, Michael Bentley, Dave Brown, Hugo Romat, Nathalie Henry Riche, Samuel Maina, Shamsi Iqbal, Siân Lindley, Stephanie Nyairo, Su Lin Blodgett, Sumit Gulwani, Sunayana Sitaram, Vu Le
MSR-TR-2024-56 | December 2024
Published by Microsoft
-
Understanding the Impacts of Language Technologies’ Performance Disparities on African American Language Speakers
Jay Cunningham, Su Lin Blodgett, Hal Daumé III, Christina Harrington, Hanna Wallach, Michael Madaio
Findings of the Association for Computational Linguistics: ACL 2024 | August 2024
-
Responsible Language Technologies: Foreseeing and Mitigating Harms
Su Lin Blodgett, Q. Vera Liao, Alexandra Olteanu, Rada Mihalcea, Michael Muller, Morgan Klaus Scheuerman, Chenhao Tan, Qian Yang
CHI 2022 | April 2022
-
Bridging Human–Computer Interaction and Natural Language Processing
Su Lin Blodgett, Michael Madaio, Brendan O’Connor, Hanna Wallach, Qian Yang
EACL 2021 Workshop | April 2021
-
-
-
Microsoft New Future of Work Report 2024
Jenna Butler, Mihaela Vorvoreanu, Rebecca Janssen, Abigail Sellen, Nicole Immorlica, Adam Troy, Advait Sarkar, Alex Farach, Alex Chouldechova, Alexandra Olteanu, Alexia Cambon, Arjun Radhakrishna, Asta Roseway, Ben Zorn, Brent Hecht, Daniel G. Goldstein, Dhruv Joshi, Ed Cutrell, Emre Kiciman, Gonzalo Ramos, Gustavo Soares, Hanna Wallach, Ian Drosos, Jack Williams (johnwilliams), Jacki O'Neill, Jake Hofman, Jaime Teevan, Javier Hernandez, Jennifer Wortman Vaughan, Jina Suh, John Tang, Justin Edwards, Kalika Bali, Kori Inkpen, Krishna Madhavan, Laylah Bulman, Leon Reicherts, Lev Tankelevitch, Longqi Yang, Martez Mott, Millicent Ochieng, Mercy Muchai, Nancy Baym, Najeeb Abdulhamid, Nicolai Marquardt, Ken Hinckley, Michael Bentley, Dave Brown, Hugo Romat, Nathalie Henry Riche, Samuel Maina, Shamsi Iqbal, Siân Lindley, Stephanie Nyairo, Su Lin Blodgett, Sumit Gulwani, Sunayana Sitaram, Vu Le
MSR-TR-2024-56 | December 2024
Published by Microsoft
-
-
-
Microsoft New Future of Work Report 2024
Jenna Butler, Mihaela Vorvoreanu, Rebecca Janssen, Abigail Sellen, Nicole Immorlica, Adam Troy, Advait Sarkar, Alex Farach, Alex Chouldechova, Alexandra Olteanu, Alexia Cambon, Arjun Radhakrishna, Asta Roseway, Ben Zorn, Brent Hecht, Daniel G. Goldstein, Dhruv Joshi, Ed Cutrell, Emre Kiciman, Gonzalo Ramos, Gustavo Soares, Hanna Wallach, Ian Drosos, Jack Williams (johnwilliams), Jacki O'Neill, Jake Hofman, Jaime Teevan, Javier Hernandez, Jennifer Wortman Vaughan, Jina Suh, John Tang, Justin Edwards, Kalika Bali, Kori Inkpen, Krishna Madhavan, Laylah Bulman, Leon Reicherts, Lev Tankelevitch, Longqi Yang, Martez Mott, Millicent Ochieng, Mercy Muchai, Nancy Baym, Najeeb Abdulhamid, Nicolai Marquardt, Ken Hinckley, Michael Bentley, Dave Brown, Hugo Romat, Nathalie Henry Riche, Samuel Maina, Shamsi Iqbal, Siân Lindley, Stephanie Nyairo, Su Lin Blodgett, Sumit Gulwani, Sunayana Sitaram, Vu Le
MSR-TR-2024-56 | December 2024
Published by Microsoft
-
-
-
Microsoft New Future of Work Report 2024
Jenna Butler, Mihaela Vorvoreanu, Rebecca Janssen, Abigail Sellen, Nicole Immorlica, Adam Troy, Advait Sarkar, Alex Farach, Alex Chouldechova, Alexandra Olteanu, Alexia Cambon, Arjun Radhakrishna, Asta Roseway, Ben Zorn, Brent Hecht, Daniel G. Goldstein, Dhruv Joshi, Ed Cutrell, Emre Kiciman, Gonzalo Ramos, Gustavo Soares, Hanna Wallach, Ian Drosos, Jack Williams (johnwilliams), Jacki O'Neill, Jake Hofman, Jaime Teevan, Javier Hernandez, Jennifer Wortman Vaughan, Jina Suh, John Tang, Justin Edwards, Kalika Bali, Kori Inkpen, Krishna Madhavan, Laylah Bulman, Leon Reicherts, Lev Tankelevitch, Longqi Yang, Martez Mott, Millicent Ochieng, Mercy Muchai, Nancy Baym, Najeeb Abdulhamid, Nicolai Marquardt, Ken Hinckley, Michael Bentley, Dave Brown, Hugo Romat, Nathalie Henry Riche, Samuel Maina, Shamsi Iqbal, Siân Lindley, Stephanie Nyairo, Su Lin Blodgett, Sumit Gulwani, Sunayana Sitaram, Vu Le
MSR-TR-2024-56 | December 2024
Published by Microsoft
-
-
-
“I Am the One and Only, Your Cyber BFF”: Understanding the Impact of GenAI Requires Understanding the Impact of Anthropomorphic AI
Myra Cheng, Alicia DeVrio, Lisa Egede, Su Lin Blodgett, Alexandra Olteanu
April 2025
-
Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
November 2024
-
Evaluating Generative AI Systems is a Social Science Measurement Challenge
Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
November 2024
-
-
-
Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
-
Dehumanizing Machines: Mitigating Anthropomorphic Behaviors in Text Generation Systems
Myra Cheng, Su Lin Blodgett, Alicia DeVrio, Lisa Egede, Alexandra Olteanu
2025 Meeting of the Association for Computational Linguistics | July 2025
SAC Highlight
-
Measuring Machine Learning Harms from Stereotypes: Requires Understanding Who is Being Harmed by Which Errors in What Ways
Angelina Wang, Xuechunzi Bai, Solon Barocas, Su Lin Blodgett
2025 ACM Conference on Fairness, Accountability, and Transparency | June 2025
-
Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor
Alexandra Olteanu, Su Lin Blodgett, Agathe Balayn, Angelina Wang, Fernando Diaz, Flavio du Pin Calmon, Margaret Mitchell, Michael Ekstrand, Reuben Binns, Solon Barocas
NeurIPS 2025 | June 2025
-
“It was 80% me, 20% AI”: Seeking Authenticity in Co-Writing with Large Language Models
Angel Hsing-Chi Hwang, Q. Vera Liao, Su Lin Blodgett, Alexandra Olteanu, Adam Trischler
2025 Conference on Computer Supported Cooperative Work | May 2025
-
A Taxonomy of Linguistic Expressions That Contribute To Anthropomorphism of Language Technologies
Alicia DeVrio, Myra Cheng, Lisa Egede, Alexandra Olteanu, Su Lin Blodgett
CHI 2025 | April 2025
-
Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
ICML 2025 | January 2025
-
Metrics for What, Metrics for Whom: Assessing Actionability of Bias Evaluation Metrics in NLP
Pieter Delobelle, Giuseppe Attanasio, Debora Nozza, Su Lin Blodgett, Zeerak Talat
2024 Empirical Methods in Natural Language Processing | November 2024
-
Understanding the Impacts of Language Technologies’ Performance Disparities on African American Language Speakers
Jay Cunningham, Su Lin Blodgett, Hal Daumé III, Christina Harrington, Hanna Wallach, Michael Madaio
Findings of the Association for Computational Linguistics: ACL 2024 | August 2024
-
ECBD: Evidence-Centered Benchmark Design for NLP
Yu Lu Liu, Su Lin Blodgett, Jackie Cheung, Q. Vera Liao, Alexandra Olteanu, Ziang Xiao
2024 Meeting of the Association for Computational Linguistics | August 2024
-
The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels
Eve Fleisig, Su Lin Blodgett, Dan Klein, Zeerak Talat
NAACL 2024 | June 2024
-
“One-Size-Fits-All”? Examining Expectations around What Constitute “Fair” or “Good” NLG System Behaviors
Li Lucy, Su Lin Blodgett, Milad Shokouhi, Hanna Wallach, Alexandra Olteanu
NAACL 2024 | June 2024
-
Responsible AI Considerations in Text Summarization Research: A Review of Current Practices
Yu Lu Liu, Meng Cao, Su Lin Blodgett, Jackie Chi Kit Cheung, Alexandra Olteanu, Adam Trischler
Findings of EMNLP 2023 | December 2023
-
FairPrism: Evaluating Fairness-Related Harms in Text Generation
Eve Fleisig, Aubrie Amstutz, Chad Atalla, Su Lin Blodgett, Hal Daumé III, Alexandra Olteanu, Emily Sheng, Dan Vann, Hanna Wallach
ACL 2023 | July 2023
-
It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance
Arjun Subramonian, Xingdi Yuan, Hal Daumé III, Su Lin Blodgett
Findings of ACL 2023 | July 2023
-
This Prompt is Measuring : Evaluating Bias Evaluation in Language Models
Seraphina Goldfarb-Tarrant, Eddie Ungless, Esma Balkir, Su Lin Blodgett
Findings of the Association for Computational Linguistics: ACL 2023 | July 2023
-
Taxonomizing and Measuring Representational Harms: A Look at Image Tagging
Jared Katzman, Angelina Wang, Morgan Scheuerman, Su Lin Blodgett, Kristen Laird, Hanna Wallach, Solon Barocas
AAAI 2022 | February 2023
-
Examining Responsibility and Deliberation in AI Impact Statements and Ethics Reviews
David Liu, Priyanka Nanayakkara, Sarah Ariyan Sakha, Grace Abuhamad, Su Lin Blodgett, Nicholas Diakopoulos, Jessica R. Hullman, Tina Eliassi-Rad
AIES 2022 | July 2022
-
Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications
Kaitlyn Zhou, Su Lin Blodgett, Adam Trischler, Hal Daumé III, Kaheer Suleman, Alexandra Olteanu
NAACL 2022 | May 2022
-
Responsible Language Technologies: Foreseeing and Mitigating Harms
Su Lin Blodgett, Q. Vera Liao, Alexandra Olteanu, Rada Mihalcea, Michael Muller, Morgan Klaus Scheuerman, Chenhao Tan, Qian Yang
CHI 2022 | April 2022
-
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets
Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, Hanna Wallach
ACL-IJCNLP 2021 | August 2021
-
A Survey of Race, Racism, and Anti-Racism in NLP
Anjalie Field, Su Lin Blodgett, Zeerak Waseem, Yulia Tsvetkov
ACL-IJCNLP 2021 | July 2021
-
Bridging Human–Computer Interaction and Natural Language Processing
Su Lin Blodgett, Michael Madaio, Brendan O’Connor, Hanna Wallach, Qian Yang
EACL 2021 Workshop | April 2021
-
-
-
AI Automatons: AI Systems Intended to Imitate Humans
Alexandra Olteanu, Solon Barocas, Su Lin Blodgett, Lisa Egede, Alicia DeVrio, Myra Cheng
March 2025
-
-
-
Microsoft New Future of Work Report 2024
Jenna Butler, Mihaela Vorvoreanu, Rebecca Janssen, Abigail Sellen, Nicole Immorlica, Adam Troy, Advait Sarkar, Alex Farach, Alex Chouldechova, Alexandra Olteanu, Alexia Cambon, Arjun Radhakrishna, Asta Roseway, Ben Zorn, Brent Hecht, Daniel G. Goldstein, Dhruv Joshi, Ed Cutrell, Emre Kiciman, Gonzalo Ramos, Gustavo Soares, Hanna Wallach, Ian Drosos, Jack Williams (johnwilliams), Jacki O'Neill, Jake Hofman, Jaime Teevan, Javier Hernandez, Jennifer Wortman Vaughan, Jina Suh, John Tang, Justin Edwards, Kalika Bali, Kori Inkpen, Krishna Madhavan, Laylah Bulman, Leon Reicherts, Lev Tankelevitch, Longqi Yang, Martez Mott, Millicent Ochieng, Mercy Muchai, Nancy Baym, Najeeb Abdulhamid, Nicolai Marquardt, Ken Hinckley, Michael Bentley, Dave Brown, Hugo Romat, Nathalie Henry Riche, Samuel Maina, Shamsi Iqbal, Siân Lindley, Stephanie Nyairo, Su Lin Blodgett, Sumit Gulwani, Sunayana Sitaram, Vu Le
MSR-TR-2024-56 | December 2024
Published by Microsoft
-