PoseTracker uses deep learning to track the position and orientation of objects. This solution will use your phone camera to measure and track the angle, orientation, and distance of an item in real time.Learn about PoseTracker
Correctly capturing an object’s position, orientation, and identity is a major challenge—without prior information, stereo optics, or measurements—it can be hard to measure scale or distance, and object recognition requires a large labeled dataset.
Convolutional neural networks (CNN) has made significant strides in object recognition, classification, and segmentation, as used in self-driving vehicles, for example. PoseTracker leverages the power of CNN to recognize and track objects in 3D.
PoseTracker uses a patented optical marker approach to infer an object’s pose from 2D images, then tracks the position from one image to all subsequent images—based on comparisons to a predefined 3D orientation.
The complex problem of position
Tracking an object’s changing distance and position is an important challenge to solve in medical imaging, self-driving vehicles, manufacturing, drones, and many IoT applications. PoseTracker is a collaborative proof of concept to solve 3D positioning.
Technical details for PoseTracker
Convolutional neural networks, a class of deep neural network, has made significant strides in the recent years in terms of object recognition, classification and segmentation leading to significant development in self driving vehicles and a great variety of computer vision application.
However, there have been very few practical implementations of these advanced approaches in object 3D pose estimation. The ability to recognize and track the object in the 3D reference space is still a difficult problem to resolve due to some several challenging issues:
- The 3D pose information is hard to capture, requiring complicated setups involving stereo optical or magnetic localization apparatus.
- The lack of prior information about the object of interest.
- A labeled dataset with the proper pose information is very hard to obtain in large quantity. The traditional image manipulation like axis scaling and transformations will inevitably corrupt 3D pose information.
The idea is to leverage the power of CNN and implement an application to recognize and track the pose (position and orientation) of objects in 3D with a patented optical marker that will help to identify the rotation and estimate the pose of the object.
PoseTracker is a proof of concept for a simple object pose detection pipeline, integrated with rotation information based on a 3D pose tracking solution (an optical marker).
The application analyzes the 2D images taken from a camera with the optical marker always visible. The application, with a supervised training, detects the marker, that infers its orientation information from one image to all subsequent images based on comparison to a predefined 3D orientation.
This different approach to solve the pose tracker issues will help in the future, to use your phone camera get the angle, orientation, and distance that an object is from you in real time.
Clean Water AI
Clean Water AI uses deep learning to detect dangerous bacteria and particles in water. The device analyzes drinking water with real-time detection and contamination mapping.
Angel Eyes is an IOT device that monitors a baby’s sleeping position and environment. Caregivers can view a live stream from anywhere and receive notifications if the device detects any issues.
Intelligent robotics uses AI to increase collaboration between people and devices. Microsoft AI enables the next generation of robots to adapt to dynamic situations and communicate naturally with people.
AirSim - Drones
AirSim is a simulation tool that creates a 3D version of a real environment. A simulated drone “flies” to capture images, building a custom vision model. AI uses the vision model to identify objects or people.
Explore the possibilities of AI
Jumpstart your own AI innovations with learning resources and development solutions from Microsoft AI.
Innovation Developer Hub
Explore insights and behind-the-scenes technology for breakthrough AI innovations. From Tech Minutes videos to Technology Deep Dives, learn about the engineering that powers the future of AI.
Learn to create your own AI experiences with learning paths in conversational AI, machine learning, AI for devices, cognitive services, autonomous systems, AI strategy, and more.
Start building AI solutions with powerful tools and services. Microsoft AI is a robust framework for developing AI solutions in conversational AI, machine learning, data sciences, robotics, IoT, and more.