343 Industries Gets New User Insights from Big Data in the Cloud
The Halo franchise is an award-winning collection of properties that has grown into a global entertainment phenomenon. To date, more than 50 million copies of Halo video games have been sold worldwide. As developers prepared to launch Halo 4, they were tasked with analyzing data to gain insights into player preferences and support an online tournament. To handle those requests, the team used a powerful Microsoft technology called Windows Azure HDInsight Service, based on the Apache Hadoop big data framework. Using HDInsight Service to process and analyze raw data from Windows Azure, the team was able to feed game statistics to the tournament’s operator, which used the data to rank players based on game play. The team also used HDInsight Service to update Halo 4 every week and support a daily email campaign designed to increase player retention. Organizations can also take advantage of data to quickly make business decisions.
Although it initially considered building its own custom BI solution, 343 Industries ultimately decided to use HDInsight Service, which is based on Apache Hadoop, an open-source software framework created by Yahoo! Hadoop, which is ideal for running complex analytics, can analyze massive amounts of unstructured data in a distributed manner. HDInsight Service is a big data solution for Windows Azure that empowers users to gain new insights from unstructured data, while connecting that data to familiar Microsoft BI tools. “Even though we knew we would be one of the earliest customers of HDInsight Service, it met all our requirements,” says Tamir Melamed, Development Manager on the Halo Services Team. “It can run any possible queries, and it is the best format for integration with Azure.”
The team was particularly attracted to the flexibility of HDInsight Service, which allowed for separating the amount of the raw data from the processing size needed to consume that data. “With previous systems, we never had the separation between production and raw data, so there was always the question of how running analytics would affect production,” says Mark Vayman, Lead Program Manager for the Halo Services Team. “Hadoop solved that problem.”
HDInsight Service was also instrumental in changing the team’s focus from data storage to useful data analysis. That’s because Hadoop applies structure to data when it’s consumed, as opposed to traditional data warehouse applications that structure data before it is placed into a BI system.
The team wrote Azure-based services that convert raw game data collected in Azure into the Avro format, which is supported by Hadoop. This data is then pushed from the Azure services in the Avro format into Windows Azure binary large object (BLOB) storage, which HDInsight Service is able to utilize with the ASV protocol. The data can then be accessed by anyone with the right permissions from Windows Azure.
Every day, Hadoop handles millions of data-rich objects related to Halo 4, including preferred game modes, game length, and many other items. With Microsoft SQL Server PowerPivot for SharePoint as a front-end presentation layer, Azure BLOBs are created based on queries from the Halo 4 team.
Microsoft SQL Server PowerPivot for Excel loads data from HDInsight Service using the Hadoop Hive ODBC driver. A PowerPivot workbook is then uploaded to PowerPivot for SharePoint and refreshed nightly within SharePoint, using the connection string stored in the workbook via the Hive ODBC driver to HDInsight Service. The team uses the workbooks to generate reports and facilitate their viewing of interactive data dashboards.
- Increases Agility and Speeds Response Time
- Helps Halo 4 Team Make Weekly Game Updates
- Provides In-Game Analysis and Helps Identify Cheaters
- Contributes to Player Retention
- Uses Familiar Tools to Simplify Decision Making
Microsoft Big Data Solution Datasheet
Get answers to technical questions about SQL Server for implementation and development.
Microsoft Big Data Solution Brief
Learn how SQL Server 2012 provides customers with large-scale data warehousing and analytical solutions backed by IT management & insights.