Announcing Spark for Azure HDInsight public preview

Lighting up interactive insights on big data

By T. K. “Ranga” Rengarajan, corporate vice president, Data Platform

Today, we’re announcing the public preview of Spark for Azure HDInsight and the upcoming general availability of Power BI on July 24th. These investments support our commitment to help more people maximize their data dividends with interactive visualizations on big data.

Big data is changing the way organizations deliver value to their stakeholders. For example, Real Madrid bring soccer matches closer to their 450 million fans and Ultra Tendency project the health impact of nuclear contamination in Japan.  Here at Microsoft, we’re thrilled to help fuel that innovation with data solutions that give customers simple but powerful capabilities. This is something we’ve done with Azure HDInsight by making Hadoop easier to provision, manage, customize, and scale. Azure HDInsight is a fully managed Hadoop service, that includes 24×7 monitoring and enterprise support across the broadest range of analytic workloads.

Introducing Spark for Azure HDInsight

Today, we go further with this vision by providing our customers the best environment to run Apache Spark. Spark is one of the most popular big data projects known for its ability to handle large-scale data applications in memory, making queries up to 100 times faster. Spark lets users do various tasks like batch and interactive queries, real-time streaming, machine learning, and graph processing – all with the same common execution model. With Spark for Azure HDInsight, we offer customers more value with an enterprise ready Spark solution that’s fully managed and has a choice of compelling and interactive experiences.

  • Choice of compelling interactive experiences: Microsoft empowers users and organizations to achieve more by making data accessible to as many people as possible.
    • We have out-of-the-box integration to Power BI for interactive visualizations over big data. Because both are powered by the cloud, you can deploy Spark cluster and visualize it in Power BI within minutes without investing in hardware or complex integration.
    • Data scientists can use popular notebooks like Zeppelin and Jupyter (iPython) to do interactive analysis and machine learning to create narratives that combine code, statistical equations, and visualizations that tell a story about the data with Spark for Azure HDInsight.
    • Microsoft offers flexibility to use BI tools like Tableau, SAP Lumira, and QlikView so you can leverage existing investments.

  • Enterprise Spark: Integrating Spark with Azure HDInsight ensures that it is ready to meet the demands of your mission critical deployments because Azure is always-on, has hyper-scale, and is enterprise-grade. With a 99.9% service level agreement at general availability, you can ensure continuity and protection against catastrophic events. As demands grow, create larger clusters with your choice of SSD and RAM allocation to process big data on demand. Microsoft also has built-in integration with other parts of Azure, like Event Hubs, for building Streaming and IoT related applications.
  • Fully Managed: With Spark for Azure HDInsight, you can get started quickly with a fully managed cluster. This includes 24×7 monitoring and enterprise support for peace of mind. You also have the elasticity of a cloud solution, so you can scale your solution up or down easily, and only pay for the power that you use.

In addition to Spark, we announced the upcoming general availability of Power BI on July 24. Power BI is a cloud-based business analytics service that enables anyone to visualize and analyze data with greater speed, efficiency, and understanding. It connects users to a broad range of live data through easy-to-use dashboards, provides interactive reports, and delivers compelling visualizations that bring data to life. Power BI has out-of-the-box connectors to Spark enabling users to do interactive visualizations on top of big data.  For more information on this announcement, read James Phillip’s blog post.

Microsoft continues to make it easier for customers to maximize their data dividends with our data platform and services. It’s never been easier to capture, transform, mash-up, analyze and visualize any data, of any size, at any scale, in its native format using familiar tools, languages and frameworks in a trusted environment on-premises and in the cloud.

Experience interactive insight on big data today with Spark for Azure HDInsight and Power BI.

–  Ranga