Vector search
Make search more accurate and capable by using vector search. It leverages approximate nearest neighbor (ANN) algorithms to search through billions of deep learning model insights in milliseconds.
Experience Bing Vector Search
The need
The challenge of search is indexing billions of entries, while also finding relevant results as quickly as possible. Most searches rely on an inverted index, which uses keyword matching as well as engineering and infrastructure.
The idea
Use vectors to improve search. Deep learning models represent data as vectors, where distance between vectors reflects similarities. Approximate nearest neighbor (ANN) algorithms search billions of vectors, returning results in milliseconds.
The solution
Vector search can be applied across many applications, such as searches across text, multimedia, images, recommendations and more. The code can be incorporated into your own applications to harness deep learning insights at scale.
AI powers smarter, faster search results
Keyword searches don’t work efficiently when words sound alike, have multiple meanings, or are ambiguous. Natural language interfaces require smarter search engines that leverage deep learning to find faster, closer matches on billions of data points.
Technical details for Bing vector search
Vector search uses deep learning models to encode data sets into meaningful vector representations, where distance between vectors represent the similarities between items. We then use approximate nearest neighbor (ANN) search algorithms to build vectors indexes that allow us to search through billions in of vectors to return the most related results in just a couple of milliseconds.
Download the architecture diagram
Vector search can be applied across a multitude of applications, such as web text search, multimedia + image search, recommendations, and many more. As an example, we’ve used vector search to power an image similarity search application across multiple image databases, such as animals, cats, and dogs. We first used an PyTorch pre-trained deep learning model to encode open source data sets (such as Stanford Dogs, Oxford Flowers, etc.) into vectors. We then used Space Partition Tree and Graph (SPTAG) algorithm to generate an approximate nearest neighbor (ANN) vector index using k-means balanced trees and nearest neighborhood graphs. When an input picture comes in, our application first uses the PyTorch model to translate the image into a vector. The query vector is then used by the SPTAG algorithm to find the most related vectors in a couple of milliseconds. The returned vectors then correspond of the images and are returned as the “most related” results.
This image similarity search is just one of many possible applications of vector search in your applications today. Try incorporating vector search in your own applications today to harness deep learning insights at scale.
Resources:
Microsoft Math
Microsoft Math uses optical character recognition (OCR) for handwriting to extract a math equation from a student’s photo of their notes. The problem is then standardized, classified, solved, and returned with solution steps and similar problems.
Celebs Like Me
Celebs Like Me uses facial recognition to match the user’s photo to similar-looking celebrities. Powered by a Deep Neural Net (DNN) model, it was trained using Bing Satori Knowledge Graph and Bing Image Graph.
Insights and Discovery Accelerator
Unlock insights and discover intelligent connections in unstructured data. The Insights and Discovery Accelerator uses Microsoft AI to speed and scale investigations and research by helping journalists find critical content in a sea of data.
JFK Files
JFK Files takes 34,000 complex files including photos, handwriting, government documents, and more, then extracts readable information. This knowledge is organized to enable new ways to explore the information.
Explore the possibilities of AI
Jumpstart your own AI innovations with learning resources and development solutions from Microsoft AI.
Innovation Tech Hub
Explore insights and behind-the-scenes technology for breakthrough AI innovations. From Tech Minutes videos to Technology Deep Dives, learn about the engineering that powers the future of AI.
AI School
Learn to create your own AI experiences with learning paths in conversational AI, machine learning, AI for devices, cognitive services, autonomous systems, AI strategy, and more.
AI platform
Start building AI solutions with powerful tools and services. Microsoft AI is a robust framework for developing AI solutions in conversational AI, machine learning, data sciences, robotics, IoT, and more.
