{"id":1013847,"date":"2024-03-11T12:00:53","date_gmt":"2024-03-11T19:00:53","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-video&p=1013847"},"modified":"2026-02-18T14:24:43","modified_gmt":"2026-02-18T22:24:43","slug":"the-metacognitive-demands-and-opportunities-of-generative-ai","status":"publish","type":"msr-video","link":"https:\/\/www.microsoft.com\/en-us\/research\/video\/the-metacognitive-demands-and-opportunities-of-generative-ai\/","title":{"rendered":"The Metacognitive Demands and Opportunities of Generative AI"},"content":{"rendered":"\n

Presented by\u00a0Lev Tankelevitch<\/a>\u00a0at\u00a0Microsoft Research Forum, Season 1, Episode 2<\/strong><\/em><\/p>\n\n\n\n

Lev Tankelevitch explored how metacognition\u2014the psychological capacity to monitor and regulate one’s cognitive processes\u2014provides a valuable perspective for comprehending and addressing the usability challenges of generative AI systems around prompting, assessing and relying on outputs, and workflow optimization.<\/p>\n\n\n\n

\n
All Research Forum sessions<\/a><\/div>\n\n\n\n
Register for the series<\/a><\/div>\n<\/div>\n\n\n
\n\t
\n\t\t
\n\t\t\t
\n\t\t\t\t\n\t\t\t\t\t\n\n

Transcript<\/h3>\n\n\n\n

The metacognitive demands and opportunities of generative AI<\/strong><\/p>\n\n\n\n

LEV TANKELEVITCH: <\/strong>My name is Lev. I\u2019m a researcher in the Collaborative Intelligence team in Microsoft Research Cambridge, UK, and today I\u2019ll be talking about what we\u2019re calling the metacognitive demands and opportunities of generative AI. So we know that AI has tremendous potential to transform personal and professional work. But as we show in our recent paper, a lot of usability challenges remain\u2014from crafting the right prompts to evaluating and relying on outputs to integrating AI into our daily workflows. And [what] we propose in a recent paper is that metacognition offers a powerful framework to understand and design for these usability challenges.  <\/p>\n\n\n\n\t\t\t\t<\/span>\n\t\t\t\t\n\t\t\t\t\t\n\n\n\n

So metacognition is thinking about<\/em> thinking and includes things like self-awareness, so our ability to be aware of our own goals, knowledge, abilities, and strategies; our confidence and its adjustment, so this is our ability to maintain an appropriate level of confidence in our knowledge and abilities and adjust that as new information comes in; task decomposition, our ability to take a cognitive task or goal and break it down into subtasks and address them in turn; and metacognitive flexibility, so our ability to recognize when a cognitive strategy isn’t<\/em> working and adapt it accordingly. Let me walk you through a simple example workflow.<\/p>\n\n\n\n

So let\u2019s say you decided to ask an AI system to help you in crafting an email. So in the beginning, you might have to craft a prompt. And so you might ask yourself, what am I trying to convey with this email? Perhaps I need to summarize x<\/em>, clarify y<\/em>, or conclude z<\/em>\u2014all in the correct tone. You might then get an output and then need to evaluate that. And then you might ask yourself, well, how can I make sense of this output? In the case of an email example, it\u2019s pretty straightforward. But what if you\u2019re working with a programming language that you\u2019re less familiar with? You might then need to iterate on your prompt. And so then you might ask yourself, well, how does it relate to my ability to craft the right prompt versus the system\u2019s performance in a given task or domain?  <\/p>\n\n\n\n

And now if you zoom out a little bit, there are these questions around what we\u2019re calling automation strategy<\/em>. So this is whether, when, and how you can apply AI to your workflows. So here you might ask yourself, is trying generative AI worth my time versus doing a task manually? And how confident am I that I can actually complete a task manually or learn AI effectively to help me do it? And then if I do decide to rely on AI on my workflows, how do I actually integrate it into my workflows most effectively? And so what we\u2019re proposing is that all these questions really reflect the metacognitive demands that generative AI systems impose on users as they interact with these systems. So, for example, at the prompt formulation stage, this involves self-awareness of task goals. So knowing exactly what you want to achieve and break that down into subgoals and subtasks and then verbalize that explicitly for an effective prompt. At the output evaluation stage, it involves well-adjusted confidence in your ability to actually evaluate that output. And so that means disentangling your confidence in the domain you\u2019re working with from the system\u2019s performance in that task or domain.  <\/p>\n\n\n\n

In the prompt iteration stage, it involves well-adjusted confidence in your prompting ability, so this is about disentangling your ability to craft an effective prompt from the system’s performance in that task or domain, and metacognitive flexibility, which is about recognizing when a prompting strategy isn\u2019t working and then adjusting it accordingly. In the automation strategy level, this is about self-awareness of the applicability and impact of AI on your workflows and well-adjusted confidence in your ability to complete a task manually or learn generative AI systems effectively to actually help you do that. And then finally, it requires metacognitive flexibility in actually recognizing when your workflow with AI isn\u2019t working effectively and adapting that accordingly.<\/p>\n\n\n\n

So beyond reframing these usability challenges through the perspective of metacognition, we know from psychology research that metacognition is both measurable and teachable. And so we can now think about how we can design systems that actually support people\u2019s metacognition as they interact with them. So, for example, you can imagine systems that support people in planning complex tasks. So let\u2019s say you\u2019ve decided to ask an AI system to help you craft an email. It might actually break that task down for you and remind you that certain types of content are more common in such emails and actually proactively prompt you to fill that content in. It might also make you aware of the fact that there\u2019s a certain tone or length that you might want to have for this email. And so in this way, it, sort of, breaks the task down for you and actually improves your self-awareness about different aspects of your task.  <\/p>\n\n\n\n

Similarly, we can imagine systems that support people in reflecting on their own cognition. So let\u2019s say you\u2019ve asked the system to help you craft a proposal based on a previous document. Now a smart system that knows in the past you\u2019ve had to edit this output quite extensively might let you know that you should specify an outline or other details and provide you with examples so that you can save time later on. Similarly, at the output evaluation stage, you can imagine how such an approach can augment AI explanations. So this is work done by the Calc Intelligence<\/a> team here at Microsoft Research, and it shows a system that can help users complete tasks in spreadsheets. And it shows a step-by-step breakdown of the approach that it took to complete that task. So you can imagine a system that proactively probes users about different steps and their uncertainty around those steps and then tailors explanations effectively to that user\u2019s uncertainty.  <\/p>\n\n\n\n

So in sum, we believe that a metacognitive perspective can really help us analyze, measure, and evaluate the usability challenges of generative AI. And<\/em> it can help us design generative AI systems that can augment human agency and workflows. For more details, I encourage you to check out the full paper, and I thank you for your time.<\/p>\n\n\t\t\t\t<\/span>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t\tShow more\t\t\t<\/button>\n\t\t<\/div>\n\t<\/div>\n<\/div>\n\n\n\n

<\/div>\n\n\n\n
\n\t
\n\t\t
\n\t\t\t\t\t\t\t\n\t\t\t\t\t\"Ask\t\t\t\t<\/a>\n\t\t\t\t\t\t\tMicrosoft research copilot experience<\/span>\n\t\t\tExplain how metacognition can inform Generative AI usability.<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

Presented by\u00a0Lev Tankelevitch\u00a0at\u00a0Microsoft Research Forum, Season 1, Episode 2 Lev Tankelevitch explored how metacognition\u2014the psychological capacity to monitor and regulate one’s cognitive processes\u2014provides a valuable perspective for comprehending and addressing the usability challenges of generative AI systems around prompting, assessing and relying on outputs, and workflow optimization.<\/p>\n","protected":false},"featured_media":1013850,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr_hide_image_in_river":null,"footnotes":""},"research-area":[13556],"msr-video-type":[268311],"msr-locale":[268875],"msr-post-option":[],"msr-session-type":[256174],"msr-impact-theme":[261670],"msr-pillar":[],"msr-episode":[269925],"msr-research-theme":[270112],"class_list":["post-1013847","msr-video","type-msr-video","status-publish","has-post-thumbnail","hentry","msr-research-area-artificial-intelligence","msr-video-type-microsoft-research-forum","msr-locale-en_us"],"msr_download_urls":"","msr_external_url":"https:\/\/youtu.be\/cyfS6APUbHU","msr_secondary_video_url":"","msr_video_file":"http:\/\/0","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/1013847","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-video"}],"version-history":[{"count":5,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/1013847\/revisions"}],"predecessor-version":[{"id":1162572,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/1013847\/revisions\/1162572"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/1013850"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1013847"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1013847"},{"taxonomy":"msr-video-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video-type?post=1013847"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1013847"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=1013847"},{"taxonomy":"msr-session-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-session-type?post=1013847"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1013847"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=1013847"},{"taxonomy":"msr-episode","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-episode?post=1013847"},{"taxonomy":"msr-research-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-theme?post=1013847"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}