{"id":918900,"date":"2023-02-20T20:01:28","date_gmt":"2023-02-21T04:01:28","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-blog-post&p=918900"},"modified":"2023-03-03T10:55:26","modified_gmt":"2023-03-03T18:55:26","slug":"chatgpt-for-robotics","status":"publish","type":"msr-blog-post","link":"https:\/\/www.microsoft.com\/en-us\/research\/articles\/chatgpt-for-robotics\/","title":{"rendered":"ChatGPT for Robotics: Design Principles and Model Abilities"},"content":{"rendered":"\n

We extended the capabilities of ChatGPT to robotics, and controlled multiple platforms such as robot arms, drones, and home assistant robots intuitively with language.<\/p>\n\n\n\n

\"main<\/figure>\n\n\n\n
\n
\n
\n
\n
\n
\n
Paper<\/a><\/div>\n\n\n\n
Video<\/a><\/div>\n\n\n\n
PromptCraft code<\/a><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n

Have you ever wanted to tell a robot what to do using your own words, like you would to a human? Wouldn’t it be amazing to just tell your home assistant robot: “Please warm up my lunch<\/em>“, and have it find the microwave by itself? Even though language is the most intuitive way for us to express our intentions, we still rely heavily on hand-written code to control robots. Our team has been exploring how we can change this reality and make natural human-robot interactions possible using OpenAI (opens in new tab)<\/span><\/a>‘s new AI language model, ChatGPT (opens in new tab)<\/span><\/a>.<\/p>\n\n\n\n

ChatGPT is a language model trained on a massive corpus of text and human interactions, allowing it to generate coherent and grammatically correct responses to a wide range of prompts and questions. Our goal with this research is to see if ChatGPT can think beyond text, and reason about the physical world to help with robotics tasks. We want to help people interact with robots more easily, without needing to learn complex programming languages or details about robotic systems. The key challenge here is teaching ChatGPT how to solve problems considering the laws of physics, the context of the operating environment, and how the robot’s physical actions can change the state of the world. <\/p>\n\n\n\n

It turns out that ChatGPT can do a lot by itself, but it still needs some help. Our technical paper<\/a> describes a series of design principles that can be used to guide language models towards solving robotics tasks. These include, and are not limited to, special prompting structures, high-level APIs, and human feedback via text. We believe that our work is just the start of a shift in how we develop robotics systems, and we hope to inspire other researchers to jump into this exciting field. Continue reading for more technical details on our methods and ideas.<\/p>\n\n\n\n

\n