Microsoft Research helps hydrology scientists make effective use of information

To help researchers gain an accurate picture of the health of a watershed, Microsoft Research collaborates with the Berkeley Water Center—located on the campus of the University of California at Berkeley—and the Lawrence Berkeley National Laboratory to build a "digital watershed."

The project goal: acquire and curate existing hydrologic data to understand historic conditions on key watersheds in California.

The historical data can be used with current data collected from sensors and new sensor network deployments. Researchers also hope to develop new models that predict what occurs within a watershed and use those models to guide watershed policy and uses.

A Flood of Data

For many environmental scientists, the challenge today is not collecting data. The challenge comes in making sense of what can be a flood of information. It can be difficult and slow work to sort through enormous quantities of information from disparate data sources.

Researchers who work on water-related projects in California often have access to large amounts of current and historical data. In California, hundreds of sensors and dozens of agencies have been collecting data on state rivers for decades—in some cases records go back before 1900.

To cite just one example of immense data collection, in California, regional water quality boards, the California State Department of Water Resources, the U.S. Geological Survey, and the Environmental Protection Agency each collect different sets of water quality measurements.

Says Deb Agarwal of Lawrence Berkeley National Laboratory: "People assume that once they have their source of data, it’ll be great, and they can do whatever they want. But the data can be frustrating. I knew one researcher who spent three months trying to get the data set he wanted from an environmental database."

Browsing for Science

Microsoft researcher Catharine van Ingen is a member of Microsoft Research’s eScience group, which is working to help how scientists deploy computer technology to make better use of data.

Van Ingen and computer science collaborators such as Agarwal are developing ways to use relational databases and database tools such as data cubes to extract information from a variety of sources.

The data cubes allow simple browsing of datasets for data availability, data quality, and data relevance along "dimensions" such as location, variable time, or time period. The use of data cubes helps researchers who don’t have programming skills to add their own data.

Van Ingen also uses Microsoft Virtual Earth technology to help researchers visualize spatial data sources and their relation to the landscape.

"A lot of the data we deal with is spatial, such as a river network, so it helps to place it on a map," says van Ingen. "With Virtual Earth you can really get the rivers to pop out at you, then overlay that with data taken in from the field and see the availability of data for different areas."

A Bridge that Connects Science, Policy, and Technology

One of the digital watersheds is the Russian River, an important northern California river that runs through rich wine country as well as fast-growing suburban areas. It faces many of the challenges of rivers in the western United States: risks to populations of salmon, competition between agricultural and urban water users, concerns about pollution, and flood control.

By combining publically available data, digital watershed researchers such as Dr. James Hunt of the Berkeley Water Center hope to create models and forecasts that can be used by a wide range of water interests in a manner that has not previously been possible.

Says van Ingen: "Our work with the Berkeley Water Center is just beginning, but it’s starting to show us how we can bridge science, policy, and technology."