{"id":187928,"date":"2012-06-01T00:00:00","date_gmt":"2012-06-20T13:32:30","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/msr-research-item\/naiad-incremental-and-iterative-data-parallel-computation\/"},"modified":"2023-02-22T09:14:27","modified_gmt":"2023-02-22T17:14:27","slug":"naiad-incremental-and-iterative-data-parallel-computation","status":"publish","type":"msr-video","link":"https:\/\/www.microsoft.com\/en-us\/research\/video\/naiad-incremental-and-iterative-data-parallel-computation\/","title":{"rendered":"Naiad: Incremental and Iterative Data-Parallel Computation"},"content":{"rendered":"<div class=\"asset-content\">\n<p>Naiad is a distributed system that supports efficient incremental and iterative data-parallel computation. It extends standard batch data-parallel processing models like MapReduce, Hadoop, and Dryad\/DryadLINQ to support efficient incremental updates in the manner of a stream processing system, while at the same time enabling arbitrarily nested fixed-point iteration. This talk describes the design and implementation of Naiad, and shows that complex analyses like strongly connected components, requiring multiple nested loops, run on streams of input with subsecond update times.<\/p>\n<\/div>\n<p><!-- .asset-content --><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Naiad is a distributed system that supports efficient incremental and iterative data-parallel computation. It extends standard batch data-parallel processing models like MapReduce, Hadoop, and Dryad\/DryadLINQ to support efficient incremental updates in the manner of a stream processing system, while at the same time enabling arbitrarily nested fixed-point iteration. This talk describes the design and implementation [&hellip;]<\/p>\n","protected":false},"featured_media":196929,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr_hide_image_in_river":0,"footnotes":""},"research-area":[13562,13547],"msr-video-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-session-type":[],"msr-impact-theme":[],"msr-pillar":[],"msr-episode":[],"msr-research-theme":[],"class_list":["post-187928","msr-video","type-msr-video","status-publish","has-post-thumbnail","hentry","msr-research-area-computer-vision","msr-research-area-systems-and-networking","msr-locale-en_us"],"msr_download_urls":"","msr_external_url":"https:\/\/youtu.be\/lHBCq2AbWqc","msr_secondary_video_url":"","msr_video_file":"","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/187928","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-video"}],"version-history":[{"count":1,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/187928\/revisions"}],"predecessor-version":[{"id":921867,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/187928\/revisions\/921867"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/196929"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=187928"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=187928"},{"taxonomy":"msr-video-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video-type?post=187928"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=187928"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=187928"},{"taxonomy":"msr-session-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-session-type?post=187928"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=187928"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=187928"},{"taxonomy":"msr-episode","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-episode?post=187928"},{"taxonomy":"msr-research-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-theme?post=187928"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}