Government catching the Big Data train

19 June 2012 | Parul Bhandari, Government Lead, Open Data and Big Data

With shrinking budgets and limited resources, many governments worldwide are facing a Big Data dilemma: deciding when the time is right to invest. In my opinion, the train has left the station. Today, many countries are already backing Big Data initiatives in order to mine greater business insights from their operations, spot opportunities for cost savings, and run more sustainably and efficiently in the long term. Take, for example, the United States, which recently announced plans to invest in Big Data across the public sector, including $200 million in research grants and infrastructure building for analyzing Big Data using a public cloud infrastructure.

As the U.S. moves forward with its ambitious plans for Big Data, I suspect that we’ll soon see similar commitments popping up abroad. The potential value of Big Data is just too great to ignore. According to a recent research report from McKinsey Global Institute, the estimated value of Big Data to Europe’s public sector could be as much as €300 billion. That’s value derived from reducing costs, improving efficiency, and enhancing tax revenue collection by leveraging Big Data analytics. For more on Big Data adoption in Europe and how it stacks up with the United States, I highly recommend checking out this article on O’Reilly Radar, which provides some insightful comparisons.

While there is clear value in investing in Big Data, the challenge becomes how to tame it. According to a MeriTalk survey of 151 federal government CIOs and IT managers in the United States, government agencies will add a petabyte (PB) of stored data over the next two years. To put that into perspective, 1PB of data is equal to 20 million four-drawer filing cabinets filled with text!  This exponential growth in new data types and volumes presents a similar challenge in Europe. Looking forward, having the right partnerships and expertise in place will be critical to harness and tap into the full potential of governments’ Big Data.

At Microsoft, we’ve been riding the “Big Data train” long before it began sweeping international headlines. In fact, every time you perform an online search using Bing, you’re witnessing Big Data in action. Through our Bing search engine, we typically analyze more than 100PB of data to deliver high-quality search results over the web. It’s just one example of our Big Data initiatives, which have enabled us to provide a range of solutions to help customers tackle their own Big Data challenges.

For more information, I encourage you to check out our dedicated Big Data solution website to learn how we help organizations unleash actionable insights from both structured and unstructured data. There, you will also find out how our analytics solutions leverage familiar tools such as Office and SharePoint and integrate with new ones like Hadoop, as well as our ability to scale with analytics in the cloud provided by Windows Azure—all to enable governments to unlock hidden value in their data.

Big Data is a goldmine of an opportunity that is here to stay. Now is a great time to get started.

Have a comment or opinion on this post? Let me know @Microsoft_Gov. Have a question for the author? Please e-mail us at

Parul Bhandari
Government Lead, Open Data and Big Data

About the Author

Parul Bhandari | Government Lead, Open Data and Big Data

Parul Bhandari works closely with governments worldwide to develop IT strategies that promote greater insights and accountability, as well as to support forming more transparent and open government.