4-page Case Study - Posted 8/11/2011
Views: 362
Rate This Evidence:

Relic Entertainment

Game Studio’s Use of Parallel Programming Yields Eightfold Reduction in Processing Time

Video game studio Relic Entertainment needed to parallelize its code to speed up the package-creation process—the last step in converting the output of content creators into a playable format for test. Early work to parallelize package-creation tools was complex and time-consuming, resulting in thousands of lines of code. By taking advantage of new parallel programming tools in the Microsoft Visual Studio 2010 development system and the .NET Framework 4, Relic was able to quickly and easily reduce that number to just 12 lines of code. The package-creation process now takes 30 minutes instead of four hours, enabling the company to more quickly begin testing new changes. Relic was able to begin using the parallel processing libraries provided in the .NET Framework 4 quickly and easily, with a negligible learning curve.

Situation

Founded in 1997, video game studio Relic Entertainment has 150 employees and is based in Vancouver, British Columbia, Canada. Homeworld, the company’s first game, was released in 1999. Since then, Relic has followed with notable titles such as Impossible Creatures, Warhammer 40,000, Company of Heroes, and Dawn of War II. The company’s latest game, Warhammer 40,000: Space Marine, is currently in development.

With a mission to be “the leader in strategic games the world can play together,” Relic must balance discipline, teamwork, and attention to detail with creative freedom and risk tolerance—as required to continually reevaluate itself without sacrificing momentum. To achieve this, the company employs an agile, Scrum-based development methodology that enables employees to take chances on new ideas and see what works, without affecting time-to-market.

*
* By taking advantage of the parallel libraries provided in the .NET Framework 4, I was able to reduce the number of lines of code required for parallel processing from 5,000 to just 12. *

James Harrison
Senior Software Developer, Relic Entertainment

*
To that end, Relic must ensure that the tools it uses for game development won’t slow it down. In the past, some of the tools did. For example, it took four hours to complete the package-creation process—the last step in converting the output of graphic artists and content creators into a playable format. The process is typically completed over and over again as the creative team iterates on changes and hands them off for test.

“In the past, I would spend the morning babysitting the package-creation process before handing the package off for test midday,” says James Harrison, a Senior Software Developer at Relic, who works on the tools that artists and content creators use, as well as the tools used to convert their output to a playable format. “We build a new package one or two times per day as the release date draws near, and, with the process taking four hours, it was a major source of lost momentum. Testers would lose half a day waiting for the new package, staying late into the evening or coming in at two in the morning to make up lost time.”

Harrison realized that parallel processing could be used to speed up the package-creation process, which involves several steps that can be done at once: retrieving files from build or source control servers, calculating checksums, converting Photoshop files to DirectDraw Surface (DDS) format, converting High Level Shader Language (HLSL) to byte code (FX) format, and so on.

“With a standard software build process, you can break it apart and distribute the workload across a server farm,” says Harrison. “You can’t do that well with package creation, which is why parallelism is a good approach. It provides a way to keep all the cores on a server as busy as possible, instead of one core running at 100 percent while all the other cores sit idle.”

At first, Harrison attempted to parallelize his package-creation tools by hand—an effort that turned out to be far from trivial. “It was lots of work, resulting in more than 5,000 lines of code spanning multiple methods and several files,” he recalls. “Writing the code went pretty fast, but debugging it was slow and painful, requiring extensive focus on low-level concepts such as thread management, locks, and scheduling. It was hard enough for me to trace through my own code; anyone else attempting to work on it would have had an even harder time.”

Solution

Harrison modified his package-creation code to take advantage of new parallel processing capabilities that are included in the Microsoft Visual Studio 2010 development system and the .NET Framework 4. “I started using the Parallel Extensions to the Microsoft .NET Framework 3.5,” says Harrison. “When the parallel libraries became fully released and supported in .NET 4, the decision to upgrade was an easy one.”

In modifying his code for parallel processing, Harrison now uses the new parallel programming libraries provided in the .NET Framework 4, which are supported by new features in Visual Studio 2010. The parallel programming libraries provided in the .NET Framework 4 include:

Task Parallel Library (TPL), which includes parallel implementations of for and foreach loops (For and For Each in the Visual Basic language) as well as lower-level types for task-based parallelism. Implemented as a set of public types and APIs in the System.Threading.Tasks namespace, the TPL relies on a task scheduler that is integrated with the .NET ThreadPool and that scales the degree of concurrency dynamically so that all available processors and processing cores are used most efficiently.

Parallel Language-Integrated Query (PLINQ), a parallel implementation of LINQ to Objects that combines the simplicity and readability of LINQ syntax with the power of parallel programming. PLINQ implements the full set of LINQ standard query operators as extension methods in the System.Linq namespace, along with additional operators to control the execution of parallel operations. As with code that targets the Task Parallel Library, PLINQ queries scale in the degree of concurrency according to the capabilities of the host computer.

Data Structures for Parallel Programming, which introduces several new types that are useful in parallel programming—including a set of concurrent collection classes that are scalable and thread-safe, lightweight synchronization primitives, and types for lazy initialization. Developers can use these new types with any multithreaded application code, including that which uses the Task Parallel Library and PLINQ.

The parallel programming libraries in the .NET Framework 4 are complemented by new parallel diagnostic tools in Visual Studio 2010 Professional, Premium, and Ultimate, which include new Parallel Stacks and Parallel Tasks windows for debugging code. Visual Studio 2010 Premium and Ultimate also have a new Concurrency Visualizer, which is integrated with the profiler to provide graphical, tabular, and numerical data about how a multithreaded application interacts with itself and other programs. With these tools, developers at Relic can quickly identify areas of concern and navigate through call stacks and to relevant call sites in the source code.

*
* With the parallel programming tools provided in Visual Studio 2010 and the .NET Framework 4, we’ve been able to reduce package build times from four hours to about 30 minutes. *

James Harrison
Senior Software Developer, Relic Entertainment

*
In parallelizing his code, Harrison made extensive use of the new data structures and synchronization primitives included in the .NET Framework 4, which gave him a way to easily manage concurrent tasks. “With Tasks, I can keep all cores on the server busy: one processing requests to the source control system, one performing an MD5 hash on a file, and the others staying busy with other types of work, such as generating DDS or FX output,” he says.

Harrison is now applying the parallel libraries in the .NET Framework 4 to speeding up the tools used by others. “Today, we use Lua as a scripting language,” he says. “By switching to .NET 4 and applying parallelism, we can get some huge performance gains—in ways that are otherwise transparent to content creators. I’ve already started to do a bit of prototyping, and everything seems to be working great.”

He is also evaluating the Visual Studio Async CTP, which extends Visual Studio 2010 with a new, streamlined syntax for asynchronous development. “I downloaded the CTP and took five minutes to write some code to break a file apart and calculate its MD5 hash in parallel,” he recalls. “Compared to before, the asynchronous code was far more readable and performance was much better, taking about oneeighth the time to process a 4gigabyte file.”

Benefits

Through its use of new parallel programming tools provided by Microsoft, Relic was able to vastly simplify the code required to parallelize its package-creation tools. The package-creation process now takes 30 minutes, enabling quality assurance personnel to much more quickly begin testing the new changes that are checked in by content creators. Harrison was able to take advantage of the parallel programming libraries provided in .NET 4 quickly and easily, with a negligible learning curve.

Significant Performance Gains

Through parallel programming, Relic realized an eightfold decrease in package-creation times. Those results were obtained on a Dell PowerEdge R510 server running the Windows Server 2008 R2 Standard operating system, which is configured with two quad-core Intel Xeon processors, a solid-state system disk, and 24 gigabytes of RAM. “With the parallel programming tools provided in Visual Studio 2010 and the .NET Framework 4, we’ve been able to reduce package build times from four hours to about 30 minutes,” says Harrison.

Strong Scalability

*
* When we find a must-fix issue, depending on its nature, we can fix and retest it quickly versus losing four hours and picking up the next day. *

James Harrison
Senior Software Developer, Relic Entertainment

*
Should additional performance be needed, Relic can now cost-effectively meet that need. “The code scales well,” says Harrison. “If I run it on a system with more cores, it will execute faster. It’s less expensive to put more cores in the server than to replace its existing CPU with a newer and faster one, and we’ll probably get better performance gains with additional cores than we would by replacing existing CPUs with new ones that have a faster clock speed.”

Reduced Developer Effort

With the parallel programming libraries in the .NET Framework 4, Harrison was able to replace several thousand lines of low-level code with a dozen lines of higher-level code, leaving all of the low-level details to the libraries themselves. “By taking advantage of the parallel libraries provided in the .NET Framework 4, I was able to reduce the number of lines of code required for parallel processing from 5,000 to just 12,” says Harrison. “The code we have today is much easier to comment and maintain, and is readily understood by other developers.”

Ease of Adoption

Harrison was able to begin taking advantage of the parallel programming libraries from Microsoft in a matter of hours, with a minimal learning curve. “Developers can start writing parallel code quickly and easily by taking advantage of the extensive resources provided by Microsoft, such as the Channel 9 videos and the Parallel Programming with .NET blog on MSDN,” says Harrison. “It only took a few hours for me to come up to speed.”

Positive Business Impact

Reduced package-creation times have yielded significant benefits for Relic, which no longer must wait a half-day to begin testing the changes that are checked in by content creators. “When we find a must-fix issue, depending on its nature, we can fix and retest it quickly versus losing four hours and picking up the next day,” says Harrison. “The parallel programming tools provided by Microsoft have enabled us to improve quality by enabling at least twice as many iterations, resulting in a higher-quality game. It’s like having another month to polish the product.”

Microsoft Visual Studio 2010

Microsoft Visual Studio 2010 is an integrated development system that helps simplify the entire development process from design to deployment. Unleash your creativity with powerful prototyping, modeling, and design tools that help you bring your vision to life. Work within a personalized environment that helps accelerate the coding process and supports the use of your existing skills, and target a growing number of platforms, including Microsoft SharePoint Server 2010 and cloud services. Also, work more efficiently thanks to integrated testing and debugging tools that you can use to find and fix bugs quickly and easily to help ensure high-quality solutions.

For more information about Visual Studio 2010, go to:

www.msdn.microsoft.com/vstudio

For More Information

For more information about Microsoft products and services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada Information Centre at (877) 568-2495. Customers in the United States and Canada who are deaf or hard-of-hearing can reach Microsoft text telephone (TTY/TDD) services at (800) 892-5234. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information using the World Wide Web, go to:

www.microsoft.com

For more information about Relic Entertainment, visit the website at: www.relic.com

This case study is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.
Solution Overview



Organization Size: 150 employees

Organization Profile

Relic Entertainment, a video game studio, has 150 employees and is based in Vancouver, British Columbia, Canada.


Business Situation

Relic needed to speed up the package-creation process—the last step in converting game content to a playable format—so that changes by content creators could be tested more quickly.


Solution

Relic used the new parallel programming libraries in the Microsoft .NET Framework 4, which enabled the company to parallelize its package-creation tools with just 12 lines of code.


Benefits

  • Significant performance gains
  • Strong scalability
  • Reduced developer effort
  • Ease of adoption
  • Positive business impact


Software and Services
  • Microsoft Visual Studio 2010 Premium
  • Windows Server 2008 R2 Standard
  • Microsoft .NET Framework 4

Vertical Industries
Amusement/Gambling/Recreation/ Perf Arts/ Spectator Sports

Country/Region
Canada

IT Issue
  • Development
  • Parallel Computing

Languages
English

RSS