{"id":795500,"date":"2021-11-16T08:00:14","date_gmt":"2021-11-16T16:00:14","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-research-item&#038;p=795500"},"modified":"2021-11-16T12:52:11","modified_gmt":"2021-11-16T20:52:11","slug":"research-talk-sptag-fast-hundreds-of-billions-scale-vector-search-with-millisecond-response-time","status":"publish","type":"msr-video","link":"https:\/\/www.microsoft.com\/en-us\/research\/video\/research-talk-sptag-fast-hundreds-of-billions-scale-vector-search-with-millisecond-response-time\/","title":{"rendered":"Research talk: SPTAG++: Fast hundreds of billions-scale vector search with millisecond response time"},"content":{"rendered":"<p>Current state-of-the-art vector approximate nearest neighbor search (ANNS) libraries mainly focus on how to do fast high-recall search in memory. However, extremely large-scale vector search scenarios present certain challenges. For example, hundreds of billions of vectors coupled with limited memory creates a capacity issue. There is also a scalability issue because increasing the number of serving machines increases query latency and computation costs. This occurs as a result of the search being done in each machine, and latency increases with the increased number of aggregating candidates. To address these challenges, we propose SPTAG++, a distributed ANNS system. In this talk, we\u2019ll discuss SPTAG++, which is now integrated into production to support hundreds of billions-scale vector searches in production with millisecond response time and more than ten thousand queries per second.<\/p>\n<p>Learn more about the 2021 Microsoft Research Summit: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/Aka.ms\/researchsummit\">https:\/\/Aka.ms\/researchsummit<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Current state-of-the-art vector approximate nearest neighbor search (ANNS) libraries mainly focus on how to do fast high-recall search in memory. However, extremely large-scale vector search scenarios present certain challenges. For example, hundreds of billions of vectors coupled with limited memory creates a capacity issue. There is also a scalability issue because increasing the number of [&hellip;]<\/p>\n","protected":false},"featured_media":795503,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr_hide_image_in_river":0,"footnotes":""},"research-area":[13556,13560,13555,13547],"msr-video-type":[261263,261302],"msr-locale":[268875],"msr-post-option":[],"msr-session-type":[],"msr-impact-theme":[],"msr-pillar":[],"msr-episode":[],"msr-research-theme":[],"class_list":["post-795500","msr-video","type-msr-video","status-publish","has-post-thumbnail","hentry","msr-research-area-artificial-intelligence","msr-research-area-programming-languages-software-engineering","msr-research-area-search-information-retrieval","msr-research-area-systems-and-networking","msr-video-type-research-summit-2021","msr-video-type-the-future-of-search-recommendation","msr-locale-en_us"],"msr_download_urls":"","msr_external_url":"https:\/\/youtu.be\/1wTb7gdiMsE","msr_secondary_video_url":"","msr_video_file":"","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/795500","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-video"}],"version-history":[{"count":1,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/795500\/revisions"}],"predecessor-version":[{"id":795506,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/795500\/revisions\/795506"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/795503"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=795500"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=795500"},{"taxonomy":"msr-video-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video-type?post=795500"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=795500"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=795500"},{"taxonomy":"msr-session-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-session-type?post=795500"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=795500"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=795500"},{"taxonomy":"msr-episode","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-episode?post=795500"},{"taxonomy":"msr-research-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-theme?post=795500"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}