The need

Robotics can help automate tasks that are repetitive, dangerous or vulnerable to human error. However, automation without intelligence creates a system that cannot respond to variables, new environments or dynamic requirements.

The idea

Combining AI with robotics creates smarter autonomous systems. With machine learning, image recognition, cognitive services, and more – robots can learn and respond to requirements, beyond simple commands.

The solution

AI provides a platform to develop intelligent bots. By adding cognitive services to the bot, we can make our bot smart – with capabilities like language understanding, image recognition, text recognition, translation, and more.

Technical details of Intelligent Robotics

Building an Intelligent Robot

In this lab, you will learn about the heart of robotics programming using the Robot Operating system (ROS) with Python and how to use Gazebo, the robot simulator. You will also learn how to deploy your code to a real industrial robot. This lab will give you the confidence to start your journey with intelligent collaborative robotics.

Meet Paul-E

This lab uses a collaborative robot based on Sawyer from Rethink Robotics named Paul-E. You do not need an actual robot; this lab is a simulation.

Paul-E is an integrated collaborative robot (a.k.a. cobot) with seven degrees of motion, designed with embedded vision, smart swappable grippers and high-resolution force control. The robot's purpose is to automate specific industrial repetitive tasks, and it comes with an arm that has a gripper which can be easily replaced

Bot Logic

For this lab, we use three tools: ROS, Gazebo and rviz.

ROS is a robotics middleware licensed under an open source. ROS provides libraries, hardware abstraction, device drivers, visualisers, message-passing, package management and other tools to help software developers create robot applications.

Gazebo allows you to build 3D scenarios on your computer with robots, using obstacles and other objects. This allows you to test robots in complex or dangerous scenarios without any harm to the real robot.

rviz is an Open Source 3D visualiser that uses sensor data and custom visualisation markers to develop robot capabilities in a virtual environment.

Making your robot intelligent

Microsoft Bot Framework and Cognitive Services provide a platform to develop intelligent bots. By adding cognitive services to the bot, we are able to make our bot smart and have capabilities like language understanding, image recognition, text recognition, translation, and more.

In this lab you will create a simple bot, then enable this bot to communicate with a physical robot, using natural language and computer vision for image recognition.

Projects related to Intelligent Robotics

Browse autonomous systems projects

Explore the possibilities of AI

Jump-start your own AI innovations with learning resources and development solutions from Microsoft AI.