{"id":1056792,"date":"2024-07-12T00:35:05","date_gmt":"2024-07-12T07:35:05","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-research-item&p=1056792"},"modified":"2024-09-05T07:18:23","modified_gmt":"2024-09-05T14:18:23","slug":"lets-fix-this-together-conversational-debugging-with-github-copilot","status":"publish","type":"msr-research-item","link":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/lets-fix-this-together-conversational-debugging-with-github-copilot\/","title":{"rendered":"Let\u2019s Fix this Together: Conversational Debugging with GitHub Copilot"},"content":{"rendered":"

Despite advancements in IDE tooling, code understanding<\/span>, generation, and automated repair, debugging continues <\/span>to present significant challenges. Existing debugging strategies <\/span>available to developers in literature are often too mechanical and <\/span>rigid for day-to-day issues. Recent advances in Large Language <\/span>Models (LLMs) promise practical solutions that allow for more <\/span>free-form debugging strategies. While LLMs offer satisfactory <\/span>assistance<\/span> in<\/span> some<\/span> cases,<\/span> they<\/span> often<\/span> leap<\/span> to<\/span> action<\/span> without <\/span>sufficient context, making implicit assumptions and providing <\/span>inaccurate responses. Moreover, the dialogue between developers <\/span>and LLMs predominantly takes the form of question-answer <\/span>pairs, placing the burden of formulating the correct questions <\/span>and sustaining multi-turn conversations on the developer.
\n<\/span>
We introduce R<\/span>OBIN<\/span>, a novel multi-agent conversational AI-<\/span>assistant within GitHub Copilot Chat, specifically designed for <\/span>debugging. R<\/span>OBIN<\/span> moves beyond the question-answer pairs by <\/span>introducing the<\/span> investigate & respond<\/span> pattern, that focuses on <\/span>using information gathered automatically from the IDE or gath<\/span>ered interactively from the developer before responding. R<\/span>OBIN <\/span>incorporates<\/span> a<\/span> general<\/span> debugging<\/span> strategy<\/span> to<\/span> systematically <\/span>analyze bugs to sustain collaborative interactions while ensuring <\/span>that the conversation does not deviate from the debugging task <\/span>at hand. Through a within-subjects user study with 16 industry <\/span>professionals, we find that equipping R<\/span>OBIN<\/span> to\u2014(1) leverage the <\/span>insert expansion<\/span> interaction pattern, (2) facilitate<\/span> turn-taking<\/span>, and <\/span>(3) utilize<\/span> debugging strategies<\/span>\u2014leads to lowered conversation <\/span>barriers, a 2.5x improvement in bug localization and a substantial <\/span>3.5x improvement in bug resolution compared to AI-assisted <\/span>debugging in Visual Studio prior to R<\/span>OBIN<\/span>.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Despite advancements in IDE tooling, code understanding, generation, and automated repair, debugging continues to present significant challenges. Existing debugging strategies available to developers in literature are often too mechanical and rigid for day-to-day issues. Recent advances in Large Language Models (LLMs) promise practical solutions that allow for more free-form debugging strategies. While LLMs offer satisfactory […]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":null,"msr_publishername":"","msr_publisher_other":"","msr_booktitle":"","msr_chapter":"","msr_edition":"","msr_editors":"","msr_how_published":"","msr_isbn":"","msr_issue":"","msr_journal":"","msr_number":"","msr_organization":"","msr_pages_string":"","msr_page_range_start":"","msr_page_range_end":"","msr_series":"","msr_volume":"","msr_copyright":"","msr_conference_name":"IEEE Symposium on Visual Languages and Human-Centric Computing (VL\/HCC)","msr_doi":"","msr_arxiv_id":"","msr_s2_paper_id":"","msr_mag_id":"","msr_pubmed_id":"","msr_other_authors":"","msr_other_contributors":"","msr_speaker":"","msr_award":"","msr_affiliation":"","msr_institution":"","msr_host":"","msr_version":"","msr_duration":"","msr_original_fields_of_study":"","msr_release_tracker_id":"","msr_s2_match_type":"","msr_citation_count_updated":"","msr_published_date":"2024-8-1","msr_highlight_text":"Best Research Paper Award","msr_notes":"","msr_longbiography":"","msr_publicationurl":"","msr_external_url":"","msr_secondary_video_url":"","msr_conference_url":"https:\/\/conf.researchr.org\/home\/vlhcc-2024","msr_journal_url":"","msr_s2_pdf_url":"","msr_year":0,"msr_citation_count":0,"msr_influential_citations":0,"msr_reference_count":0,"msr_s2_match_confidence":0,"msr_microsoftintellectualproperty":true,"msr_s2_open_access":false,"msr_s2_author_ids":[],"msr_pub_ids":[],"msr_hide_image_in_river":0,"footnotes":""},"msr-research-highlight":[246574],"research-area":[13556,13554,13560],"msr-publication-type":[193716],"msr-publisher":[],"msr-focus-area":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[246694,255217],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-1056792","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-highlight-award","msr-research-area-artificial-intelligence","msr-research-area-human-computer-interaction","msr-research-area-programming-languages-software-engineering","msr-locale-en_us","msr-field-of-study-artificial-intelligence","msr-field-of-study-human-computer-interaction-4"],"msr_publishername":"","msr_edition":"","msr_affiliation":"","msr_published_date":"2024-8-1","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"","msr_how_published":"","msr_notes":"","msr_highlight_text":"Best Research Paper Award","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"","msr_publicationurl":"","msr_doi":"","msr_publication_uploader":[{"type":"file","viewUrl":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2024\/07\/VLHCC__Robin__AI_assisted_Code_Debugging-14.pdf","id":"1056804","title":"_vlhcc__robin__ai_assisted_code_debugging-14","label_id":"243109","label":0}],"msr_related_uploader":"","msr_citation_count":0,"msr_citation_count_updated":"","msr_s2_paper_id":"","msr_influential_citations":0,"msr_reference_count":0,"msr_arxiv_id":"","msr_s2_author_ids":[],"msr_s2_open_access":false,"msr_s2_pdf_url":null,"msr_attachments":[{"id":1056804,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2024\/07\/VLHCC__Robin__AI_assisted_Code_Debugging-14.pdf"}],"msr-author-ordering":[{"type":"user_nicename","value":"Yasharth Bajpai","user_id":42228,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Yasharth Bajpai"},{"type":"user_nicename","value":"Bhavya Chopra","user_id":43434,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Bhavya Chopra"},{"type":"guest","value":"param-biyani","user_id":1048509,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=param-biyani"},{"type":"text","value":"Cagri Aslan","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Sumit Gulwani","user_id":33755,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Sumit Gulwani"},{"type":"text","value":"Dustin Coleman","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Chris Parnin","user_id":41985,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Chris Parnin"},{"type":"user_nicename","value":"Arjun Radhakrishna","user_id":39405,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Arjun Radhakrishna"},{"type":"user_nicename","value":"Gustavo Soares","user_id":39183,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Gustavo Soares"}],"msr_impact_theme":[],"msr_research_lab":[199565],"msr_event":[],"msr_group":[663303],"msr_project":[],"publication":[],"video":[],"msr-tool":[],"msr_publication_type":"inproceedings","related_content":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1056792","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":4,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1056792\/revisions"}],"predecessor-version":[{"id":1058673,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1056792\/revisions\/1058673"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1056792"}],"wp:term":[{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=1056792"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1056792"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=1056792"},{"taxonomy":"msr-publisher","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publisher?post=1056792"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=1056792"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1056792"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=1056792"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=1056792"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=1056792"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=1056792"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1056792"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=1056792"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}