AI Lab projects
Learn about breakthrough AI innovation with hands-on labs, code resources and deep dives.
Snip Insights
Snip Insights helps users find intelligent insights from a snip or screenshot. AI services convert a captured image into translated text, automatically detecting and tagging image content.
Get Snip Insights source code
The need
What if users could use tools to streamline repetitive tasks? People rely on visual information to find and index things, source new ideas and learn new things – but visual information can be hard to translate to a technical environment.
The idea
Using visual information involves lots of repetitive steps. AI helps simplify how you use images. Identify famous people or landmarks, turn a scan of a book into text or find a pair of shoes based on a photo – in one app.
The solution
Snip Insights is an open source desktop utility that helps users retrieve intelligent insights from a snip or screenshot. Snip Insights leverages cloud AI services to convert images to translated text and make smart image suggestions.
AI screen capture with Snip Insights
Convert images to translated text, detect and tag image content, start an image search directly from a snip and even shop for a new discovery – all from a single app.
Technical details for Snip Insights
Snip Insights is an open source desktop utility that retrieves intelligent insights from a snip or screenshot. Snip Insights leverages cloud AI services to convert images to translated text, automatically detect and tag image content and make smart image suggestions – showcasing Azure Cognitive Services’ potential.
Imagine that you have a scan of a textbook or work report. Rather than having to manually type out the information, snipping it will return editable text from an image. Or maybe you’re scrolling through your social media feed and come across someone wearing a cool pair of shoes – you can snip to find where to purchase them. Snip Insights can also find information based on an image – including identifying famous people and landmarks. It streamlines the process of screenshotting, saving the picture, uploading it to reverse image search engines and then drawing results from there.
Installation
Windows
- Download the zip for Windows.
- Install the certificate (‘.cer’ file) according the instructions detailed on Install Certificate section.
- Install Snip Insights by double click over the .appx package file.
Linux
- Install Mono by following the official steps depending on your Linux distribution.
- Install the .deb package.
- Launch the app form the application section.
MacOS
- Download and install Mono (Stable channel). This includes GTK#, the UI toolkit used by Xamarin.Forms for this project.
- Install the .pkg as a normal MacOS application.
- Snip Insights app is available on Applications section on MacOS.
Sketch2Code
Sketch2Code converts hand-written drawings to HTML prototypes. Designers share ideas on a whiteboard, then changes are shown instantly in the browser – helping to improve collaboration between the designer, developer and customer.
Pix2Story
Pix2Story uses Natural Language Processing (NLP) for storytelling. AI scans a picture, applies a writing style and generates a story – demonstrating how AI can drive creativity.
Microsoft Math
Microsoft Math uses optical character recognition (OCR) for handwriting to extract a mathematical equation from a student’s photo of their notes. The problem is then standardised, classified, solved and returned with solution steps and similar problems.
Explore the possibilities of AI
Make artificial intelligence real for your business today.
Create innovative AI solutions
Discover Azure AI – a portfolio of AI services designed for developers and data scientists. Take advantage of the decades of breakthrough research, responsible AI practices and flexibility that Azure AI offers to build and deploy your own AI solutions.
