October 31, 2016

Data & Analytics Partners: Intelligent Kiosk demo lets you create personalized customer experiences

By Sadalit Van Buren

Sadie Van Buren, Data and Analytics Partners, Technology Solutions Professional

Introducing the Intelligent Kiosk, a collection of experiences built on top of Microsoft Cognitive Services, and now an official Cognitive Services sample and open source project. Intelligent Kiosk is an ideal demo tool for partners. In this post, I’ll explain how you can access and customize the Intelligent Kiosk resources to incorporate into your demos and offerings. And, in our next Data and Analytics Partner community call on November 15, we’ll cover this topic in depth.

The Intelligent Kiosk is a Universal Windows Platform app with a collection of demos that showcases the perceptual intelligence capability of Azure Machine Learning, Cortana Intelligence Suite, and Microsoft Cognitive Services.

The Kiosk is an example of an intelligent application that can understand human faces and predict their age, gender, and emotion. It can do so from live images from a web cam, Bing Images, or local photos. It allows you to very quickly create and train a model to perform face identification against a pre-determined set of faces.

Through the power of speech-to-text and text sentiment analysis, the Kiosk shows how an application can detect the sentiment of what someone is saying. Most of the experiences are hands-free and autonomous, using human faces in front of a web camera as the main form of input.

Intelligent Kiosk - Realtime crowd insights sample

Cognitive Services scenarios in the Kiosk demos

The Kiosk uses many of the Cognitive Services across its scenarios, including Bing News Search, Bing Image Search, Bing Autosuggest, Text Analytics, Face, Emotion, Vision, and the LUIS/Bot Framework. Here are the scenarios in the Kiosk demos, and their underlying product features.

  • Automatic Photo Capture. Capture an image and apply analytics when people approach the web camera. Features: Windows 10 Face Tracking; Face API for age & gender prediction and face identification.
  • News Analytics. Analyze search terms and detect sentiments. Features: Bing News API, Bing Autosuggest API, Text Analytics API for sentiment analysis and key phrase extraction
  • Emotion API Explorer. Analyze faces and detect emotion. Features: Windows 10 Face Tracking; Emotion API; Bing Image Search API; Bing Autosuggest API
  • Face API Explorer. Analyze faces and detect age and gender. Features: Windows 10 Face Tracking; Face API for age & gender prediction and face identification; Bing Image Search API; Bing Autosuggest API
  • Face Identification Setup. Train the machine learning model behind the Face APIs to recognize specific people. Features: Face API for face identification; Bing Image Search API; Bing Autosuggest API
  • Mall Kiosk. Makes product recommendations based on the people in front of the camera and then analyzes their reaction to it.Features: Windows 10 Face Tracking; Face API for age & gender prediction and face identification; real-time sampling using Emotion API; Windows 10 Speech-To-Text; Text Analytics API for sentiment analysis.
  • Realtime Crowd Insights – processes frames from a web camera to derive realtime crowd insights such as demographics, emotion and unique face counting. Features: Windows 10 Face Tracking; Realtime sampling; Face API for age and gender prediction, face identification, and unique face counting; Emotion API.

Get the details about each of the APIs

The Kiosk demos illustrate these scenarios in various ways. Many of them can be customized to be able to recognize a specific person or make specific recommendations based on criteria. For example, perhaps you’re going to meet with a customer CIO, and you know she loves sailing. With a few minutes of configuration beforehand, you can give her a personalized experience in the Face API Explorer, and offer her sailing-related suggestions in the Mall Kiosk.

Demos list

  • Intelligent Kiosk - Bing News AnalyticsBing News Analytics – Categorizes search results based on the sentiment of the news headlines, and uses Key Phrase extraction to create a word cloud with the top keywords of the headlines
  • CaptionBot – Captions whatever is in front of the camera
  • Crankify Kiosk – Turns the user into a Crank from the Maze Runner series
  • Emotion API Explorer – Allows the user to load images from Bing or Local Files, or from the web-camera, and displays the top three emotions on each face
  • Emotion Match (baby face edition) – Analyzes the emotion on each face and matches it with a face with similar emotion from a set of baby photos; performs real-time face tracking to detect when people are near the camera; once it detects people, it guides them through the rest of the experience
  • Emotion Photo Booth – Challenges the user to express the eight emotions that are recognized by the Emotion API; can handle many people at the same time: whoever shows the most expressive look for each emotion wins
  • Face API Explorer – Performs age and gender prediction as well as face identification by using web cam photos, local files on your hard drive, or images from Bing Images; for people who have been added to the Face Identification Setup model, it will show their names and age, for unknown people it will show their gender and ageIntelligent Kiosk - Realtime video insights
  • How Old – Guesses the age and gender of people; performs real-time face tracking to detect when people are near the camera; once it detects people it guides them through the rest of the experience; can also be trained to identify specific people
  • Mall Kiosk – Makes product recommendations based on the person’s gender and age; can be configured to make personalized recommendations when specific people are recognized
  • Realtime Crowd Insights – Analyzes emotion, age, gender, and identification while the people are looking at the camera, and displays those insights
  • Realtime Video Insights – similar to the Realtime Crowd demo, except that it uses as input a video file (from a local file or YouTube) and provides a different visualization of the emotion by slicing it for each person in the video and plotting it on a timeline
  • What Dog – similar to Emotion Match, except it matches each person with a dog, based on the age and gender of the person
  • What If – uses Cognitive Services and the Bot Framework to analyze “what-if” like questions and answer them with an image


The partner opportunity

The Intelligent Kiosk is an ideal demo tool. It engages your audience quickly, and they can easily see the relevance to their own business needs and use cases. Consumers expect personalized experiences, and the Kiosk makes the personalization opportunity clear and achievable.

You can take the demo on the road with you using any Windows 10 device that has a webcam, and you can also set it up at your office or at your next conference booth.

Most the demos included with Intelligent Kiosk are hands-free and self-guided. They require only the web cam as input, using the live camera feed to detect when people are nearby and engaging with them through UI feedback.

Business scenarios where a user encounters a screen presents the opportunity for personalized interaction with that user:

  • Digital ads and directories in public spaces like shopping malls
  • Retail store experiences
  • Hotel, resort, and casino environments
  • Waiting rooms
  • Gas pumps
  • ATMs
  • Public transit

Consider your customer portfolio and identify opportunities to customize an Intelligent Kiosk demo that provides them with a transformative solution.

Create your Intelligent Kiosk demo

Option 1

Download the source code and documentation at GitHub

Option 2

For information about a pre-built app, read the Intelligent Kiosk note in the Data Platform and Advanced Analytics Partners Yammer group (instructions to join the US Partner Community Yammer network here)

Whichever way you prefer to set up the kiosk, this powerful demo tool will help your customers see new potential to engage their own customers – one of the four dimensions of digital transformation.

Your resources

Intelligent Kiosk Sample on GitHub

Cortana Intelligence Service and Machine Learning blog

Cortana Intelligence Suite overview

Cortana Intelligence Suite Azure Services

Microsoft Cognitive Services

Microsoft Cognitive Services SDKs and Samples

Data Platform and Advanced Analytics Partner Community

Community call schedule

Yammer group

Blog series

Training and enablement

Data Analytics call Nov 15     Data and analytics Yammer group     Cognitive Services API

Other posts you may like