{"id":675009,"date":"2020-07-17T08:15:29","date_gmt":"2020-07-17T15:15:29","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-project&#038;p=675009"},"modified":"2023-11-16T10:47:02","modified_gmt":"2023-11-16T18:47:02","slug":"synapseml","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/synapseml\/","title":{"rendered":"SynapseML"},"content":{"rendered":"<section class=\"mb-3 moray-highlight\">\n\t<div class=\"card-img-overlay mx-lg-0\">\n\t\t<div class=\"card-background  has-background- card-background--full-bleed\">\n\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1920\" height=\"720\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/07\/MMLSpark_AI_header_07-2020_1920x720.png\" class=\"attachment-full size-full\" alt=\"MMLSpark header graphic - polygons on a purple background\" style=\"\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/07\/MMLSpark_AI_header_07-2020_1920x720.png 1920w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/07\/MMLSpark_AI_header_07-2020_1920x720-300x113.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/07\/MMLSpark_AI_header_07-2020_1920x720-1024x384.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/07\/MMLSpark_AI_header_07-2020_1920x720-768x288.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/07\/MMLSpark_AI_header_07-2020_1920x720-1536x576.png 1536w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/07\/MMLSpark_AI_header_07-2020_1920x720-1600x600.png 1600w\" sizes=\"auto, (max-width: 1920px) 100vw, 1920px\" \/>\t\t<\/div>\n\t\t<!-- Foreground -->\n\t\t<div class=\"card-foreground d-flex mt-md-n5 my-lg-5 px-g px-lg-0\">\n\t\t\t<!-- Container -->\n\t\t\t<div class=\"container d-flex mt-md-n5 my-lg-5 \">\n\t\t\t\t<!-- Card wrapper -->\n\t\t\t\t<div class=\"w-100 w-lg-col-5\">\n\t\t\t\t\t<!-- Card -->\n\t\t\t\t\t<div class=\"card material-md-card py-5 px-md-5\">\n\t\t\t\t\t\t<div class=\"card-body \">\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\n<h1 class=\"wp-block-heading h2\" id=\"synapseml\">SynapseML<\/h1>\n\n\n\n<p>Simple and Distributed Machine Learning<\/p>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n\n\n\n\n<p>Synapse Machine Learning (SynapseML) expands the distributed computing framework <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/apache\/spark\" target=\"_blank\" rel=\"noopener noreferrer\">Apache Spark<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> in several new directions. SynapseML adds several machine learning frameworks to the SparkML Ecosystem, including <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/Microsoft\/LightGBM\" target=\"_blank\" rel=\"noopener noreferrer\">LightGBM<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>,&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/vowpalwabbit.org\/\" target=\"_blank\" rel=\"noopener noreferrer\">Vowpal Wabbit<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>,&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/opencv.org\/\" target=\"_blank\" rel=\"noopener noreferrer\">OpenCV<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>,&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/linkedin\/isolation-forest\" target=\"_blank\" rel=\"noopener noreferrer\">Isolation Forest,<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> and the<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/onnxruntime.ai\/\"> Open Neural Network Exchange (ONNX)<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>. These tools allow users to craft powerful and highly-scalable models that span multiple ML ecosystems.<\/p>\n\n\n\n<p>SynapseML also brings new networking capabilities to the Spark ecosystem. With the HTTP on Spark project, users can embed <em>any&nbsp;<\/em>web service into their SparkML models and use their Spark clusters for massive networking workflows. In this vein, SynapseML provides easy-to-use SparkML transformers for a wide variety of <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/azure.microsoft.com\/en-us\/services\/cognitive-services\/\" target=\"_blank\" rel=\"noopener noreferrer\">Microsoft Cognitive Services<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>. Finally, the Spark Serving project enables high throughput, sub-millisecond latency web services, backed by your Spark cluster.<\/p>\n\n\n\n<p>Visit the SynapseML <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/aka.ms\/spark\" target=\"_blank\" rel=\"noopener noreferrer\">web site<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> or Github <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/microsoft\/SynapseML\" target=\"_blank\" rel=\"noopener noreferrer\">repository<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> to learn more.<\/p>\n\n\n\n\n\n<p><a href=\"https:\/\/www.microsoft.com\/en-us\/ai\/ai-lab-gen-studio\" target=\"_blank\" rel=\"noopener\">Learn about GenStudio ><\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/www.microsoft.com\/en-us\/ai\/ai-lab-snow-leopard\" target=\"_blank\" rel=\"noopener\">Learn about the Snow Leopard Trust ><\/a><\/p>\n\n\n\n<p>Watch these Spark Summit videos:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.youtube.com\/watch?v=OPk_bru9sf4\" target=\"_blank\" rel=\"noopener noreferrer\">The Azure Cognitive Services on Spark: Clusters with Embedded Intelligent Services<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/youtu.be\/qCpIXuMTTDc\" target=\"_blank\" rel=\"noopener noreferrer\">Spark Summit Keynote: Scalable AI for Good<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> | <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/databricks.com\/session_eu19\/scalable-ai-for-good\" target=\"_blank\" rel=\"noopener noreferrer\">Session abstract<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/medius.studios.ms\/Embed\/Video-nc\/B19-BRK3004?latestplayer=true&l=2571.208093\" target=\"_blank\" rel=\"noopener noreferrer\">The Cognitive Services for Big Data in Cosmos DB<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.youtube.com\/watch?v=OPk_bru9sf4\" target=\"_blank\" rel=\"noopener noreferrer\">The Azure Cognitive Services on Spark: Clusters with Embedded Intelligent Services Spark Summit 2019<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.youtube.com\/watch?v=dFxtDnjKJJ0\" target=\"_blank\" rel=\"noopener noreferrer\">Apache Spark Serving: Unifying Batch, Streaming, and RESTful Serving Spark Summit 2019<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.youtube.com\/watch?v=m5vAF231XZQ\" target=\"_blank\" rel=\"noopener noreferrer\">Deep Reality Simulation for Automated Poacher Detection Spark Summit Europe 2018<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.youtube.com\/watch?v=zmWrZm13VOY\" target=\"_blank\" rel=\"noopener noreferrer\">Unsupervised Object Detection Using the Azure Cognitive Services on Spark Spark Summit Europe 2018<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<\/ul>\n\n\n","protected":false},"excerpt":{"rendered":"<p>Synapse Machine Learning expands the distributed computing framework\u00a0Apache Spark in several new directions and brings new networking capabilities to the Spark ecosystem.<\/p>\n","protected":false},"featured_media":676014,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13556],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-675009","msr-project","type-msr-project","status-publish","has-post-thumbnail","hentry","msr-research-area-artificial-intelligence","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"","related-publications":[604704,672654,677550,709975,985071],"related-downloads":[],"related-videos":[671061,985086,985101,985110,985119,985149,985161,985305,985323],"related-groups":[664548],"related-events":[],"related-opportunities":[],"related-posts":[610674],"related-articles":[],"tab-content":[{"id":0,"name":"Talks and workshops","content":"<a href=\"https:\/\/www.microsoft.com\/en-us\/ai\/ai-lab-gen-studio\" target=\"_blank\" rel=\"noopener\">Learn about GenStudio &gt;<\/a>\r\n\r\n<a href=\"https:\/\/www.microsoft.com\/en-us\/ai\/ai-lab-snow-leopard\" target=\"_blank\" rel=\"noopener\">Learn about the Snow Leopard Trust &gt;<\/a>\r\n\r\nWatch these Spark Summit videos:\r\n<ul>\r\n \t<li><a href=\"https:\/\/www.youtube.com\/watch?v=OPk_bru9sf4\" target=\"_blank\" rel=\"noopener\">The Azure Cognitive Services on Spark: Clusters with Embedded Intelligent Services<\/a><\/li>\r\n \t<li><a href=\"https:\/\/youtu.be\/qCpIXuMTTDc\" target=\"_blank\" rel=\"noopener\">Spark Summit Keynote: Scalable AI for Good<\/a> | <a href=\"https:\/\/databricks.com\/session_eu19\/scalable-ai-for-good\" target=\"_blank\" rel=\"noopener\">Session abstract<\/a><\/li>\r\n \t<li><a href=\"https:\/\/medius.studios.ms\/Embed\/Video-nc\/B19-BRK3004?latestplayer=true&amp;l=2571.208093\" target=\"_blank\" rel=\"noopener\">The Cognitive Services for Big Data in Cosmos DB<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.youtube.com\/watch?v=OPk_bru9sf4\" target=\"_blank\" rel=\"noopener\">The Azure Cognitive Services on Spark: Clusters with Embedded Intelligent Services Spark Summit 2019<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.youtube.com\/watch?v=dFxtDnjKJJ0\" target=\"_blank\" rel=\"noopener\">Apache Spark Serving: Unifying Batch, Streaming, and RESTful Serving Spark Summit 2019<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.youtube.com\/watch?v=m5vAF231XZQ\" target=\"_blank\" rel=\"noopener\">Deep Reality Simulation for Automated Poacher Detection Spark Summit Europe 2018<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.youtube.com\/watch?v=zmWrZm13VOY\" target=\"_blank\" rel=\"noopener\">Unsupervised Object Detection Using the Azure Cognitive Services on Spark Spark Summit Europe 2018<\/a><\/li>\r\n<\/ul>"}],"slides":[],"related-researchers":[{"type":"guest","display_name":"Ilya Matiach","user_id":675060,"people_section":"Project Contributors","alias":""},{"type":"user_nicename","display_name":"Markus Cozowicz","user_id":32803,"people_section":"Project Contributors","alias":"marcozo"},{"type":"guest","display_name":"Anand Raman","user_id":675066,"people_section":"Project Contributors","alias":""},{"type":"guest","display_name":"Christina Lee","user_id":675069,"people_section":"Project Contributors","alias":""},{"type":"user_nicename","display_name":"Markus Weimer","user_id":33038,"people_section":"Project Contributors","alias":"mweimer"},{"type":"guest","display_name":"Dalitso Banda","user_id":675072,"people_section":"Project Contributors","alias":""},{"type":"guest","display_name":"Scott Graham","user_id":675063,"people_section":"Project Contributors","alias":""},{"type":"user_nicename","display_name":"Roy Levin","user_id":40060,"people_section":"Project Contributors","alias":"rolevin"}],"msr_research_lab":[],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/675009","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":7,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/675009\/revisions"}],"predecessor-version":[{"id":985296,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/675009\/revisions\/985296"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/676014"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=675009"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=675009"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=675009"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=675009"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=675009"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}