{"id":1136154,"date":"2025-04-10T09:00:00","date_gmt":"2025-04-10T16:00:00","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=1136154"},"modified":"2025-08-01T08:58:26","modified_gmt":"2025-08-01T15:58:26","slug":"debug-gym-an-environment-for-ai-coding-tools-to-learn-how-to-debug-code-like-programmers","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/debug-gym-an-environment-for-ai-coding-tools-to-learn-how-to-debug-code-like-programmers\/","title":{"rendered":"Debug-gym: an environment for AI coding tools to learn how to debug code like programmers"},"content":{"rendered":"\n
\"A<\/figure>\n\n\n\n

The ongoing proliferation of AI coding tools is not only boosting developers\u2019 efficiency, it also signals a future where AI will generate a growing share of all new code. GitHub CEO Thomas Dohmke (opens in new tab)<\/span><\/a> predicted as much in 2023, when he said that “sooner than later, 80% of the code is going to be written by Copilot.”  <\/p>\n\n\n\n

Both large and small software companies are already heavily using AI to generate code. Y Combinator\u2019s Garry Tan (opens in new tab)<\/span><\/a> noted that 95% of code for a quarter of Y Combinator\u2019s latest batch of startups was written by large language models.<\/p>\n\n\n\n

In fact, most developers spend the majority of their time debugging code,<\/strong> not writing it. As maintainers of popular open-source repositories, this resonates with us. But what if an AI tool could propose fixes for hundreds of open issues, and all we had to do was approve them before merging? This was what motivated us to maximize the potential time savings from AI coding tools by teaching them to debug code. <\/p>\n\n\n\n

By debugging we mean the interactive, iterative process to fix code. Developers typically hypothesize why their code crashed, then gather evidence by stepping through the program and examining variable values. They often use debugging tools like pdb (Python debugger) to assist in gathering information. This process is repeated until the code is fixed.<\/p>\n\n\n\n

Today’s AI coding tools boost productivity and excel at suggesting solutions for bugs based on available code and error messages. However, unlike human developers, these tools don’t seek additional information when solutions fail, leaving some bugs unaddressed, as you can see in this simple demo of how a mislabeled column stumps today\u2019s coding tools (opens in new tab)<\/span><\/a>. This may leave users feeling like AI coding tools don\u2019t understand the full context of the issues they are trying to solve. <\/p>\n\n\n\n

Introducing debug-gym<\/h2>\n\n\n\n

A natural research question emerges: to what degree can LLMs use interactive debugging tools such as pdb?<\/strong> To explore this question, we released debug-gym<\/strong> (opens in new tab)<\/span><\/a> \u2013 an environment that allows code-repairing agents to access tools for active information-seeking behavior. Debug-gym expands an agent\u2019s action and observation space with feedback from tool usage, enabling setting breakpoints, navigating code, printing variable values, and creating test functions. Agents can interact with tools to investigate code or rewrite it, if confident. We believe interactive debugging with proper tools can empower coding agents to tackle real-world software engineering tasks and is central to LLM-based agent research. The fixes proposed by a coding agent with debugging capabilities, and then approved by a human programmer, will be grounded in the context of the relevant codebase, program execution and documentation, rather than relying solely on guesses based on previously seen training data.<\/p>\n\n\n\n

\"Figure
Figure 1: Diagram demonstrating the code-repairing process in outline. In most existing approaches (shown in black<\/strong>), an agent rewrites its code conditioned on error messages obtained from executing the code. debug-gym equips the agent with additional tools such as pdb (shown in red<\/strong>), so it can interactively seek necessary information from the semantic space hidden behind the code and therefore have better code-repairing performance.<\/figcaption><\/figure>\n\n\n\n

Debug-gym is designed and developed to:<\/p>\n\n\n\n