PoseTracker uses deep learning to track the position and orientation of objects. This solution will use your phone camera to measure and track the angle, orientation and distance of an item in real time.Learn about PoseTracker
Correctly capturing an object’s position, orientation and identity is a major challenge – without prior information, stereo optics or measurements – it can be hard to measure scale or distance, and object recognition requires a large labelled dataset.
Convolutional neural networks (CNN) have made significant strides in object recognition, classification and segmentation, as used in self-driving vehicles, for example. PoseTracker leverages the power of CNN to recognise and track objects in 3D.
PoseTracker uses a patented optical marker approach to infer an object’s pose from 2D images, then tracks the position from one image to all subsequent images – based on comparisons to a predefined 3D orientation.
The complex problem of position
Tracking an object’s changing distance and position is an important challenge to solve in medical imaging, self-driving vehicles, manufacturing, drones and many IoT applications. PoseTracker is a collaborative proof of concept to solve 3D positioning.
Technical details for PoseTracker
Convolutional neural networks, a class of deep neural network, have made significant strides in the recent years in terms of object recognition, classification and segmentation leading to significant development in self driving vehicles and a great variety of computer vision applications.
However, there have been very few practical implementations of these advanced approaches in object 3D pose estimation. The ability to recognise and track the object in the 3D reference space is still a difficult problem to resolve due to some several challenging issues:
- The 3D pose information is hard to capture, requiring complicated set-ups involving stereo optical or magnetic localisation apparatus.
- The lack of prior information about the object of interest.
- A labelled dataset with the proper pose information is very hard to obtain in large quantity. The traditional image manipulation like axis scaling and transformations will inevitably corrupt 3D pose information.
The idea is to leverage the power of CNNs and implement an application to recognise and track the pose (position and orientation) of objects in 3D with a patented optical marker that will help to identify the rotation and estimate the pose of the object.
PoseTracker is a proof of concept for a simple object pose detection pipeline, integrated with rotation information based on a 3D pose tracking solution (an optical marker).
The application analyses the 2D images taken from a camera with the optical marker always visible. The application, with supervised training, detects the marker that infers its orientation information from one image to all subsequent images based on comparison to a predefined 3D orientation.
This different approach to solve the pose tracker issues will help in the future, to use your phone camera get the angle, orientation and distance that an object is from you in real time.
Clean Water AI
Clean Water AI uses deep learning to detect dangerous bacteria and particles in water. The device analyses drinking water with real-time detection and contamination mapping.
Angel Eyes is an IOT device that monitors a baby’s sleeping position and environment. Caregivers can view a live stream from anywhere and receive notifications if the device detects any issues.
Intelligent robotics uses AI to increase collaboration between people and devices. Microsoft AI enables the next generation of robots to adapt to dynamic situations and communicate naturally with people.
AirSim – Drones
AirSim is a simulation tool that creates a 3D version of a real environment. A simulated drone ‘flies’ to capture images, building a custom vision model. AI uses the vision model to identify objects or people.
Explore the possibilities of AI
Make artificial intelligence real for your business today.
Create innovative AI solutions
Discover Azure AI – a portfolio of AI services designed for developers and data scientists. Take advantage of the decades of breakthrough research, responsible AI practices and flexibility that Azure AI offers to build and deploy your own AI solutions.