{"id":1113300,"date":"2024-12-17T18:50:08","date_gmt":"2024-12-18T02:50:08","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-blog-post&p=1113300"},"modified":"2025-10-16T23:47:05","modified_gmt":"2025-10-17T06:47:05","slug":"redefining-robot-intelligence-2024-microsoft-research-asia-startrack-scholars-program-accelerates-embodied-ai-and-large-robotics-models","status":"publish","type":"msr-blog-post","link":"https:\/\/www.microsoft.com\/en-us\/research\/articles\/redefining-robot-intelligence-2024-microsoft-research-asia-startrack-scholars-program-accelerates-embodied-ai-and-large-robotics-models\/","title":{"rendered":"Redefining robot intelligence: 2025 Microsoft Research Asia StarTrack Scholars Program accelerates embodied\u202fAI\u202fand large robotics models"},"content":{"rendered":"\n

With the rapid advancements in AI and robotics, the development of highly intelligent robots capable of seamlessly interacting with the physical environment is becoming increasingly achievable. As the next AI wave, embodied AI innovations promise to revolutionize various industries and significantly impact human life.<\/p>\n\n\n\n

Although promising progress has been made, generalist robots and embodied AI are still in their infancy. The team is thrilled to envision this future and are committed to developing cutting-edge foundational robotics models to accelerate its arrival.<\/p>\n\n\n\n

Microsoft Research Asia would like to invest in a collaborative effort to explore Embodied AI and Large Action Models. We believe that the research on embodied AI will build a solid foundation and promising development prospect for robot intelligence, benefit human society. If you are an aspiring researcher with a zeal for exploring embodied AI and large action models, we invite you to apply to the Microsoft Research Asia StarTrack Scholars Program. Applications are now open for the 2025 program. For more details and to submit your registration, visit our official website: Microsoft Research Asia StarTrack Scholars Program \u2013 Microsoft Research<\/a> <\/p>\n\n\n\n

Build foundation action model for general robots<\/h2>\n\n\n\n

Embodied AI is more than a simple fusion of robots with LLMs or VLMs. Beyond language intelligence and cognitive abilities, action intelligence is essential for executing plans and engaging with the physical world. This form of intelligence diverges significantly from language intelligence. For instance, it necessitates dense, dexterous actions and demands high levels of spatial and physical awareness. Our research is focused on creating a new generation of foundational action models that enhance spatial and physical proficiencies in perception, reasoning, and action.<\/p>\n\n\n\n

\"Cognition
Cognition and action intelligence for Embodied AI<\/figcaption><\/figure>\n\n\n\n

Spatial intelligence as the key to action capabilities<\/h2>\n\n\n\n

The fact that robots operate in a 3D physical world poses unique requirements and challenges. The team believes that spatial intelligence is crucial for developing robust action capabilities. By leveraging advanced 3D computer vision techniques, we aim to provide robots with a deep understanding of their environments. Our work includes pioneering methods for 3D human-object interaction reconstruction, enabling robots to learn from humans and navigating and manipulating the objects with unprecedented generalization and precision.<\/p>\n\n\n\n

Crafting optimal model architectures<\/h2>\n\n\n\n

Model architecture design is crucial for advancing the capabilities of embodied AI. The modality of action differs fundamentally from language and vision. Actions are continuous, dense, and require precision in both time and space. Moreover, robotic actions frequently require timely execution to interact effectively with dynamic environments. Effective model architectures must accommodate these demands by providing robust frameworks that integrate real-time sensory feedback with decision-making processes. This integration ensures that robots can adapt their actions swiftly to changes in their surroundings, maintaining elevated levels of performance in complex and unpredictable settings.<\/p>\n\n\n\n

\"Foundation
Foundation models seamlessly integrating vision, language, and action capabilities<\/figcaption><\/figure>\n\n\n\n

From spatial reasoning to physical mastery<\/h2>\n\n\n\n

As robots transition from static observers to dynamic participants in the physical world, achieving physical mastery becomes a pivotal goal. This transformation requires moving beyond mere spatial reasoning to encompass the nuanced skills necessary for interacting with complex environments. Our journey from spatial reasoning to physical mastery involves integrating human-like multimodal sensor fusion into our models. For example, tactile and force sensors are crucial for tasks requiring delicate manipulation and feedback. By embedding these capabilities, we aim to enable robots to perform complex tasks with human-like dexterity and adaptability.<\/p>\n\n\n\n

Potential research topics for StarTrack Scholars Program<\/h2>\n\n\n\n

The team invites scholars to explore a range of exciting research topics within the 2025 StarTrack Scholars program, including but not limited to:<\/p>\n\n\n\n