{"id":934128,"date":"2023-04-10T18:06:32","date_gmt":"2023-04-11T01:06:32","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/"},"modified":"2023-09-14T12:04:40","modified_gmt":"2023-09-14T19:04:40","slug":"chatgpt-empowered-long-step-robot-control-in-various-environments-a-case-application","status":"publish","type":"msr-research-item","link":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/chatgpt-empowered-long-step-robot-control-in-various-environments-a-case-application\/","title":{"rendered":"ChatGPT Empowered Long-Step Robot Control in Various Environments: A Case Application"},"content":{"rendered":"

This paper demonstrates how OpenAI’s ChatGPT can be used in a few-shot setting to convert natural language instructions into a sequence of executable robot actions. The paper proposes easy-to-customize input prompts for ChatGPT that meet common requirements in practical applications, such as easy integration with robot execution systems and applicability to various environments while minimizing the impact of ChatGPT’s token limit. The prompts encourage ChatGPT to output a sequence of predefined robot actions, represent the operating environment in a formalized style, and infer the updated state of the operating environment. Experiments confirmed that the proposed prompts enable ChatGPT to act according to requirements in various environments, and users can adjust ChatGPT’s output with natural language feedback for safe and robust operation. The proposed prompts and source code are open-source and publicly available at this https URL<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

This paper demonstrates how OpenAI’s ChatGPT can be used in a few-shot setting to convert natural language instructions into a sequence of executable robot actions. The paper proposes easy-to-customize input prompts for ChatGPT that meet common requirements in practical applications, such as easy integration with robot execution systems and applicability to various environments while minimizing […]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":null,"msr_publishername":"","msr_publisher_other":"","msr_booktitle":"","msr_chapter":"","msr_edition":"","msr_editors":"","msr_how_published":"","msr_isbn":"","msr_issue":"","msr_journal":"IEEE Access","msr_number":"","msr_organization":"","msr_pages_string":"","msr_page_range_start":"95060","msr_page_range_end":"95078","msr_series":"","msr_volume":"1","msr_copyright":"Creative Commons License","msr_conference_name":"","msr_doi":"","msr_arxiv_id":"","msr_s2_paper_id":"","msr_mag_id":"","msr_pubmed_id":"","msr_other_authors":"","msr_other_contributors":"","msr_speaker":"","msr_award":"","msr_affiliation":"","msr_institution":"","msr_host":"","msr_version":"","msr_duration":"","msr_original_fields_of_study":"","msr_release_tracker_id":"","msr_s2_match_type":"","msr_citation_count_updated":"","msr_published_date":"2023-8-30","msr_highlight_text":"","msr_notes":"Open Access https:\/\/doi.org\/10.1109\/ACCESS.2023.3310935","msr_longbiography":"","msr_publicationurl":"","msr_external_url":"","msr_secondary_video_url":"","msr_conference_url":"","msr_journal_url":"","msr_s2_pdf_url":"","msr_year":0,"msr_citation_count":0,"msr_influential_citations":0,"msr_reference_count":0,"msr_s2_match_confidence":0,"msr_microsoftintellectualproperty":true,"msr_s2_open_access":false,"msr_s2_author_ids":[],"msr_pub_ids":[],"msr_hide_image_in_river":0,"footnotes":""},"msr-research-highlight":[],"research-area":[13556],"msr-publication-type":[193715],"msr-publisher":[],"msr-focus-area":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[246694,249835],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-934128","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-artificial-intelligence","msr-locale-en_us","msr-field-of-study-artificial-intelligence","msr-field-of-study-robotics"],"msr_publishername":"","msr_edition":"","msr_affiliation":"","msr_published_date":"2023-8-30","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"IEEE Access","msr_volume":"1","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"","msr_how_published":"","msr_notes":"Open Access https:\/\/doi.org\/10.1109\/ACCESS.2023.3310935","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"","msr_publicationurl":"","msr_doi":"","msr_publication_uploader":[{"type":"file","viewUrl":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2023\/04\/chatgpt_robot_manipulation_prompts.pdf","id":"964833","title":"chatgpt_robot_manipulation_prompts-3","label_id":"243109","label":0},{"type":"url","viewUrl":"false","id":"false","title":"https:\/\/doi.org\/10.1109\/ACCESS.2023.3310935","label_id":"243106","label":0}],"msr_related_uploader":[{"type":"url","viewUrl":"false","id":"false","title":"https:\/\/github.com\/microsoft\/ChatGPT-Robot-Manipulation-Prompts","label_id":"264520","label":0}],"msr_citation_count":0,"msr_citation_count_updated":"","msr_s2_paper_id":"","msr_influential_citations":0,"msr_reference_count":0,"msr_arxiv_id":"","msr_s2_author_ids":[],"msr_s2_open_access":false,"msr_s2_pdf_url":null,"msr_attachments":[{"id":964833,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2023\/08\/chatgpt_robot_manipulation_prompts.pdf"},{"id":955362,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2023\/07\/chatgpt_robot_manipulation_prompts.pdf"},{"id":955359,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2023\/07\/2304.03893.pdf"},{"id":953421,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2023\/07\/chatgpt_robot_manipulation_prompts__.pdf"},{"id":935289,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2023\/04\/chatgpt_robot_manipulation_prompts_.pdf"},{"id":934227,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2023\/04\/ChatGPT_Robot_Manipulation_Prompts.pdf"},{"id":934224,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2023\/04\/ChatGPT-Robot-Manipulation-Prompts-6435721255df2.pdf"},{"id":934221,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2023\/04\/2304.03893_ARR-643571b5b6bf5.pdf"},{"id":934149,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2023\/04\/htaccess.txt"},{"id":934146,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2023\/04\/ChatGPT-Robot-Manipulation-Prompts.pdf"}],"msr-author-ordering":[{"type":"user_nicename","value":"Naoki Wake","user_id":39916,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Naoki Wake"},{"type":"user_nicename","value":"Atsushi Kanehira","user_id":42672,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Atsushi Kanehira"},{"type":"guest","value":"kazuhiro-sasabuchi","user_id":821605,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=kazuhiro-sasabuchi"},{"type":"user_nicename","value":"Jun Takamatsu","user_id":41500,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Jun Takamatsu"},{"type":"user_nicename","value":"Katsushi Ikeuchi","user_id":32500,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Katsushi Ikeuchi"}],"msr_impact_theme":[],"msr_research_lab":[],"msr_event":[],"msr_group":[668253,1057371],"msr_project":[821527],"publication":[],"video":[],"msr-tool":[],"msr_publication_type":"article","related_content":{"projects":[{"ID":821527,"post_title":"Interactive Learning-from-Observation","post_name":"interactive-learning-from-observation","post_type":"msr-project","post_date":"2022-02-24 20:57:14","post_modified":"2025-01-29 14:49:10","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/interactive-learning-from-observation\/","post_excerpt":"Service-robot solutions to empower senior citizens \u200b to achieve more and to enhance their lives The goal of this project is to develop an interactive learning-from-observation (LfO) system in the service-robot domain so as to empower senior citizens to achieve more and enhance their lives. Currently, many seniors in assisted living facilities would have preferred to remain at their homes. If we can use service robots to assist them, they can stay at home, conduct…","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/821527"}]}}]},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/934128","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":4,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/934128\/revisions"}],"predecessor-version":[{"id":964836,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/934128\/revisions\/964836"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=934128"}],"wp:term":[{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=934128"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=934128"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=934128"},{"taxonomy":"msr-publisher","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publisher?post=934128"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=934128"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=934128"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=934128"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=934128"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=934128"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=934128"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=934128"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=934128"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}