South Florida Water Management District

Florida Boosts Productivity, Cuts Run Times with High-Performance Computing Cluster

Sixteen counties in Florida depend on South Florida Water Management District (SFWMD) to manage and protect their water resources. Since 2002, SFWMD has been testing Linux-based and Windows-based high-performance computing clusters. That testing was taken over in 2003 by the Interagency Modeling Center (IMC), which performs complex numerical modeling and analysis in support of SFWMD restoration projects. In 2005, the IMC built a five-node computer cluster and began testing Windows® Compute Cluster Server 2003 in the field. Offering 64-bit processor support, Windows Compute Cluster Server 2003 has helped the IMC boost productivity and decrease the run times of its Windows-based modeling software. Positive test results have encouraged the IMC to consider consolidating its multiple operating systems into a single, unified Windows-based environment.

Situation

As a regional agency of the state of Florida, South Florida Water Management District (SFWMD) manages and protects water resources by balancing and improving water quality, flood control, natural systems, and water supplies. With a staff of more than 1,800, SFWMD serves 16 South Florida counties, covering 18,000 square miles. SFWMD is working with the U.S. Army Corps of Engineers on the most ambitious environmental restoration project in the country: restoration and protection of the Everglades ecosystem.

Windows Compute Cluster Server 2003 allows us to take full advantage of the dual processors and run many jobs at the same time. That’s a big deal in this type of production environment.    Akin Owosina Program Manager, Interagency Modeling Center SFWMD

The Interagency Modeling Center (IMC), a division of the Hydrologic and Environmental Systems Modeling department of SFWMD, is charged with performing and coordinating numerical modeling in support of this restoration effort. The IMC uses a number of numerical modeling applications that run in three operating environments: UNIX, Linux, and the Microsoft® Windows Server® operating system. As the agency’s projects continued to become more complex, the IMC required significant computational resources that it didn’t have in-house. Although universities and research facilities offered supercomputer resources, they were expensive, and priority for agency projects could not be guaranteed.

In 2002, prior to the formation of the IMC in 2003, SFWMD decided to explore the use of high-performance computing clusters. A research group set up a Linux-based computer cluster to test its Linux-based numerical modeling applications. The group also had some Windows-based modeling applications and decision support tools that it wanted to test in a clustered environment. To test its Windows-based modeling applications, SFWMD teamed up with Cornell Theory Center, which had been working on a project to implement Windows-based clusters.

SFWMD set up a five-node cluster based on the Microsoft Windows 2000 Server operating system. Cornell Theory Center provided a simple management interface that allowed administrators to submit and manage batch jobs. The head node in the cluster was a 32-bit dual processor running at 2.6 gigahertz (GHz) with 2 gigabyte (GB) of memory. The compute nodes were similarly configured but with 3.2-GHz processors. The group used this cluster primarily to test a 32-bit modeling application called Mikeshe/Mike11, developed by the Danish Hydraulic Institute (DHI).

After extensively testing and achieving positive performance results, the IMC determined that computer clusters were worth pursuing further, but with more aggressive performance criteria. “One of our challenges was that we had stitched together a lot of software to make the first test cluster work,” says Akin Owosina, Program Manager for the IMC. The group wanted a more cohesive solution that was user friendly and easy to manage. In addition, the IMC wanted a solution that would enable:

• Existing 32-bit versions of its modeling applications to take advantage of 64-bit versions of Windows.
• Several instances of a Windows-based application to be launched on each node in the cluster.
• Windows-based applications to be rewritten to take advantage of all processors in the computer cluster (referred to as parallelization).
• Developers to port Regional Simulation Model (RSM), a next-generation modeling tool developed by SFWMD, from its native Linux platform to Windows.

A long-term goal for Owosina is to consolidate the range of computers, operating systems, and applications into a single, unified operating environment. “Our short-term goal is to move from three [UNIX, Linux, and Windows] platforms to two [Linux and Windows]. But ultimately, I’d like to get down to just Windows,” says Owosina. 

Solution

In 2005, Cornell Theory Center put the IMC in touch with people at Microsoft who were working on Windows Compute Cluster Server 2003, a new product that would support 64-bit processors and

On the cluster, we saw run time drop from about 25 hours to 13—almost half. We are quite pleased with that.  Akin Owosina Program Manager, Interagency Modeling Center SFWMD

high-performance computing solutions on multiple nodes. “Based on early discussions with Microsoft, we realized that if we beta-tested the solution and worked together with them, we could improve our existing cluster environment,” says Owosina.

In December 2005, the group took its eight-node Linux computer cluster offline and reconfigured the three 64-bit computers as a Windows-based cluster. The dual AMD Opteron (248) 2.2-GHz server computers were reloaded with Windows Compute Cluster Server. Two servers had 16 GB of random access memory (RAM), one had 8 GB of RAM, and all had a local hard disk drive. The IMC configured the 8-GB RAM computer as both a head node and a compute node; this combined head and compute node runs the Compute Cluster Administrator and the Compute Cluster Manager. The other two nodes were configured as compute nodes only. All three nodes were joined to the existing Microsoft Active Directory® service domain.

The IMC tested several applications in the clustered environment, among them, the 32-bit version of Mikeshe/Mike11 running in the 64-bit environment. South Florida Water Management District also uses a groundwater flow modeling application called Lower East Coast sub Regional Model (LECsR), evolved from the U.S. Geological Survey’s MODFLOW model. The IMC was able to recompile the 32-bit LECsR code for a 64-bit environment and tested it on the cluster. In addition, the IMC continues to work on porting the RSM application from Linux to the 64-bit Windows-based environment.

In May 2006, the IMC conducted more rigorous testing on a five-node cluster. Following the test period, SFWMD purchased Dell 1950 server computers, each with two Dual Core Xeon 5130 processors and 4 GB of RAM, to support its ongoing work.

Benefits

In the 12 weeks that the IMC has been testing Windows Compute Cluster Server 2003, overall productivity has increased due to faster run times and the ability to run several jobs simultaneously. In addition, the cluster software is more affordable than alternative solutions and paves the way for the IMC to consolidate its multiple operating environments in the future.

Increased Productivity

For production shops like the IMC, the volume of jobs that can be run is often as important as the speed at which they run. A primary goal for the IMC was to be able to run more jobs simultaneously. Owosina explains that the previous cluster environment allowed the group to queue up multiple jobs, but each server ran only one job at a time, even though the servers were dual processors. “Windows Compute Cluster Server 2003 allows us to take full advantage of the dual processors and run many jobs at the same time. That’s a big deal in this type of production environment, because we need to churn out models,” says Owosina.

Our short-term goal is to move from three [UNIX, Linux, and Windows] platforms to two [Linux and Windows]. But ultimately, I’d like to get down to just Windows.  Akin Owosina Program Manager, Interagency Modeling Center SFWMD

The ability to run multiple jobs simultaneously allows the IMC to do certain types of analyses that weren’t possible before. Owosina offers the following as an example.

In lieu of running a model many times to generate probabilistic outcomes, the group often did “deterministic” runs of models. They made a number of assumptions, ran the model once, and based their interpretations on that one run. “We could tell the planners and design engineers that they needed a pump of x capacity,” says Owosina. “But we could not tell them any distribution around that number for risk-based decision making. That’s the kind of information that the design engineers and decision makers really need.”

The IMC is also finding that jobs are running faster with Windows Compute Cluster Server. “In a test of LECsR, we got a significant speed improvement on the 64-bit version running in the cluster environment,” says Owosina. He attributes part of the speed improvement to the 64-bit support, but he points out that Windows Compute Cluster Server enabled the group to implement the 64-bit version of the code. “On the cluster, we saw run time drop from about 25 hours to 13—almost half. We are quite pleased with that,” says Owosina.

The group is still testing its ability to parallelize models. “When we are developing and calibrating a new model, it might take three days to complete a run on a complicated model. If we parallelize that model so that it takes advantage of all the processors that are available, it might complete a run in, say, three hours instead of three days,” says Owosina. Parallelizing Windows-based models like Mikeshe/Mike11—one of the group’s most critical modeling tools—would enable the group to run models of greater complexity and show more detail in its analyses. Owosina predicts that when a 64-bit version of Mikeshe/Mike11 is available and is modified to support parallel processing on the cluster, it will yield significant performance gains.

Reduced Costs

Compared to the alternative of purchasing time on supercomputers at universities and research facilities, Windows Compute Cluster Server is an affordable solution for SFWMD and the IMC. “It also lets us, not the university, determine the priority of our runs,” says Owosina.

Windows Compute Cluster provides an operating environment that will enable the group to move toward its goal of a single, less costly operating environment. “For the IMC, the first step is to consolidate from three environments down to two. Ultimately, if we can consolidate to just the Windows platform, then I won’t have to buy specific hardware, such as Sun workstations, to run just one legacy modeling application,” says Owosina. “Ultimately, I’d like to see one, maybe two machines on each desktop instead of three or four.”  

Improved Operational Efficiency

Windows Compute Cluster Server helps the IMC to make the most of its processing power because modelers are able to submit, schedule, and manage batch jobs around the clock. In the future, a single, unified Windows-based environment will be easier for developers to use, for Owosina to manage, and for his staff to support. “If I can get everything in my production environment to look the same, work the same, and run the same, it will run much smoother and more efficiently,” says Owosina.

For More Information

For more information about Microsoft products and services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada Information Centre at (877) 568-2495. Customers who are deaf or hard-of-hearing can reach Microsoft text telephone (TTY/TDD) services at (800) 892-5234 in the United States or (905) 568-9641 in Canada. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information using the World Wide Web, go to:
www.microsoft.com

For more information about South Florida Water Management District, Interagency Modeling Center, call (561) 686-8800 or visit the Web site at:
www.sfwmd.gov 

For more information about Everglades restoration, visit the Web site at:
www.evergladesplan.org

Microsoft Server Product Portfolio

For more information about the Microsoft server product portfolio, go to:
www.microsoft.com/servers/default.mspx

This case study is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.

Document published October 2006