Microsoft Adds IoT Streaming Analytics, Data Production and Workflow Services to Azure
This blog post is authored by Joseph Sirosh, Corporate Vice President of Machine Learning at Microsoft.
Today, I am excited to announce three new services: Azure Stream Analytics, Azure Data Factory and Azure Event Hubs. These services continue to make Azure the best cloud platform for our customers to build big data solutions.
Azure Stream Analytics and Azure Data Factory are available in preview and Azure Event Hubs is now generally available. These new capabilities help customers process data from devices and sensors within the Internet of Things (IoT), and manage and orchestrate data across diverse sources.
- Stream Analytics is a cost-effective event processing engine that helps uncover real-time insights from devices, sensors, infrastructure, applications and data quickly and easily.
- Azure Data Factory enables information production by orchestrating and managing diverse data.
- Azure Event Hubs is a scalable service for collecting data from millions of “things” in seconds.
Azure Stream Analytics and Azure Event Hubs
Every day, IoT is fueling vast amounts of data from millions of endpoints streaming at high velocity in the cloud. Examples of streaming analytics can be found across many businesses, such as stock trading, fraud detection, identity protection services, sensors, web clickstream analytics and alerts from CRM applications. In this new and fast-moving world of cloud and devices, businesses can no longer wait months or weeks for insights generated from data.
With Azure Stream Analytics, businesses can gain insights in real time from data generated by devices, sensors, infrastructure, applications and other sources. Developers can easily combine streams of data – such as clickstreams, logs, metering data or device-generated events – with historic records or reference data. Complementing Stream Analytics, Azure Event Hubs is a highly scalable publish-subscribe ingestor that collects millions of events per second, allowing users to process and analyze data produced by connected assets such as devices and sensors. Stream Analytics provides out-of-the-box integration with Event Hubs – when connected, these two solutions enable customers to harness IoT by processing and analyzing massive amounts of data in real time.
One customer already using Stream Analytics and Event Hubs is Aerocrine, a medical products company focused on the improved management and care of patients with inflammatory airway diseases. The company is developing devices that include the ability to collect telematics data from clinics. The devices will connect to Azure and use Stream Analytics and Event Hubs to collect telematics information and perform near real-time analytics on top of the stream of the data from the instruments. The system will collect data about usage and performance to further improve the customer service experience and send out real-time alerts for maintenance.
Azure Data Factory
Most organizations today are dealing with a variety of massive amounts of data from many different sources: across geographic locations, on-premises and cloud, unstructured and structured. Effectively managing, coordinating and processing this data can be challenging, especially when the system needs to constantly evolve to deal with new business requirements, scale to handle growing data volume and be broad enough scope to manage diverse systems – commercial or open source – from a single place.
Azure Data Factory helps solve this problem by providing customers with a single place to manage data movement, orchestration and monitoring of diverse data sources, including SQL Server and Azure Blobs, Tables, Azure SQL Database and SQL Server in Azure Virtual Machines. Developers can efficiently build data driven workflows that join, aggregate and transform data from local, cloud-based and internet services, and set up complex data processing systems with little programming.
Milliman, an independent actuarial and consulting firm, is continuously innovating solutions for its clients and is now taking advantage of Azure Data Factory to unlock Azure HDInsight to organize and report over large and disorganized data sets. Milliman’s SaaS solution, IntegrateTM, will provide a data management environment to support both the creation of input data for the models and reporting across the vast amount of data generated from the models.
Rockwell Automation, the world’s largest company dedicated to industrial automation and information, is demonstrating IoT capabilities by offering remote monitoring services that collect data from sensors which is then securely sent to Microsoft Azure. A key component of their architecture is Data Factory. With Data Factory, Rockwell Automation is able to orchestrate critical data pipelines for time series sensor data by leveraging Microsoft Azure HDInsight so users can work with the data in Power BI and Azure Machine Learning here.
Microsoft data services
Azure Stream Analytics, Azure Event Hubs and Data Factory are just a few of the data services we’ve added to Azure recently. Just this month at Strata + Hadoop World we introduced support for Apache Storm in Azure HDInsight, and over the past few months we announced Azure SQL Database, Azure DocumentDB, Azure Search and Azure Machine Learning. We’re delivering these new services so our customers have easier ways to manage, analyze and act on their data – using the tools, languages and frameworks they are familiar with – in a scalable and reliable cloud environment.
###
