The next flood gate of Big Data? Think small - then even smaller

27 June 2012 | Alan Merrihew, Senior Director of Government Technology Strategy, Worldwide Public Sector

When people talk about Big Data, they talk about the volume, variety, and velocity of data, or the "three V's" of Big Data. With the global volume of data growing at an estimated 59 percent year over year according to Gartner, many organizations are ill-equipped to analyze this amount data, causing them to be overrun with data, literally. If you think it's overwhelming now, however, we've likely only seen the tip of the iceberg.

As pointed out in a recent article from MIT's Technology Review, there is a convergence of factors taking place that will create a new tidal wave of Big Data in years to come:

  • The cost of computing continues to decline
  • The energy efficiency of electronics is going up
  • An emerging class of super-low power devices is entering the market

Combined, many experts predict that these factors will give rise to billions-if not trillions-of low-cost, highly efficient sensors and smart devices that people and organizations will use to better understand their surrounding environments. From using a swarm of sensors to analyze immediate changes in weather patterns to embedding refrigerators with sensors to alert a household when it's out of milk, these devices will collectively shape an Internet of Things—bringing our physical world and virtual world closer together.

Needless to say, when almost anything can be connected to the Internet to report information, there will be massive volumes of data accompanying it. The challenge then becomes how to interpret, analyze this information, and improve our ability to predict outcomes based on this information. For the public sector, this will be especially important for governments seeking to tackle some of their biggest challenges. From using sensors to better understand phenomena such as weather, pollution, or traffic patterns to analyzing massive sets of "nanodata" to model the societal effects of policy, to the fast and low-cost mapping of the human genome to deliver better health outcomes. Big Data analytics has the potential to one day make this all possible.

Today, we're working hard to get there. In fact, we recently set a new world record for data sorting—an important capability of Big Data analytics. It's just one small step to realizing a brighter future with Big Data.

Have a comment or opinion on this post? Let me know @MicrosoftOnGov. Have a question for the author? Please e-mail us at

Alan Merrihew
Senior Director of Government Technology Strategy, Worldwide Public Sector

About the Author

Alan Merrihew | Senior Director of Government Technology Strategy, Worldwide Public Sector

As chief technology strategist for the government team, Alan regularly engages with senior government officials and policy makers worldwide to help them leverage technology to help make government more efficient and effective.