MarketWatch.com

MarketWatch.com Deploys SQL Server 2005 to Enhance Scalability, Availability

MarketWatch.com, a wholly-owned subsidiary of Dow Jones & Company, helps more than five million unique visitors a month stay abreast of market conditions with its business and investing news, personal finance information, and analytical tools. The company was using Microsoft® SQL Server™ 2000 database, which had served it well, but needed a solution that would support more than two gigabytes of physical memory. It also wanted to enhance high availability. MarketWatch upgraded to SQL Server 2005 Enterprise Edition (64-bit), which supports up to 1 terabyte of RAM. The company is evaluating SQL Server 2005 Database Mirroring which allows continuous streaming of the transaction log from a source server to a single destination server to provide complete database synchronization. The company develops its own applications using Microsoft Visual Studio® 2005 and the Microsoft .NET Framework.

Situation
MarketWatch.com, which was acquired by Dow Jones & Company in January 2005, is a leading provider of business and investing news, personal finance information, and analytical tools. Founded in 1997, the product lineup includes the popular MarketWatch.com web site, the award-winning BigCharts.com charting application, and the online trading simulation game VirtualStockExchange.com.

SQL Server 2005, especially with the 1 terabyte of RAM supported in the 64-bit version, opens the door for substantial memory expansion. Martin Petersen Director of Operations Dow Jones & Company

In recent months, MarketWatch.com has launched a number of redesigns, journalist blogs, podcasts and videos produced onsite that range from CEO interviews to hot stocks to personal finance trends. In June 2005, MarketWatch.com was named Best Internet Business Service Over 1 Million unique monthly users for the 2005 EPpy Awards.

Additionally, MarketWatch Information Services is a global leader offering content and state-of-the-art tools and technology for license by financial-services companies, media sites, and corporate services.

The IT infrastructure supporting MarketWatch was based upon Microsoft® SQL Server™ 2000 database running on a Windows Server® 2003 Enterprise Edition operating system. The earlier version of SQL Server had served the company well, but looking toward the future, the company needed a solution that would support more than 2 gigabytes (GB) of physical RAM, and wanted to find a more powerful tool than SQL Server Data Transformation Services (DTS) for coordinating extract, transform, and load (ETL) processes.

MarketWatch’s technology team also needed to enhance availability of its disaster recovery site, to provide a real-time alternative to the manual process the company had been using for its geographically distributed redundancy.

Solution
Working with Scalability Experts, a Microsoft Gold Certified Partner, MarketWatch upgraded its key market data master database to run on three instances of Microsoft SQL Server 2005 Enterprise Edition (64-bit), hosted on an active\active\passive three-node cluster hosted on Dell PowerEdge 2950 servers with two dual-core Intel Xeon processors and 4 GB of RAM. The nodes are deployed using Windows Server 2003 Clustering Services. Storage is on an EMC SAN with 10 terabytes of raw storage.

An identical infrastructure for the market data master database is deployed at the company’s active redundant data center in New Jersey. Each site also hosts several subscription servers with data replicated from the market data master database. The Load Balancing feature of Windows Server 2003 is used to coordinate loads between the subscription servers at each site. The data centers are scaled to an “n-1” model, each data center has enough capacity to serve all site needs in case one data center should be unavailable.

The solution makes use of a number of SQL Server features, including:
SQL Server 2005 Log Shipping. The two data center market data master databases, each holding about 200 GB, are synchronized using the Log Shipping feature of SQL Server 2005. The company is considering deployment of the Database Mirroring feature of SQL Server 2005 to enhance high availability.
SQL Server 2005 Integration Services. Information is imported into the market data master database from financial markets and other reporting services around the world. The updates are made using a collection of ETL servers that receive raw data from a wide range of sources and perform ETL processes on the data before loading it into the market data master database. Each data center is served by three ETL servers. The company is in the process of upgrading its ETL servers from SQL Server 2000 to SQL Server 2005, at which point it will upgrade from DTS to SQL Server 2005 Integration Services, which provides enterprise-grade ETL capabilities.

Visual Studio and the .NET Framework, along with SQL Server 2005, have been very important to us in creating scalable solutions based on a services oriented architecture. Kent Tangen Director of Engineering and Product MarketWatch.com

SQL Server 2005 Database Snapshot. MarketWatch uses the Snapshot feature of SQL Server 2005 to run reports from a separate SQL Server instance, so reporting can be done without impacting performance on the production servers. SQL Server 2005 introduces the capacity for administrators to generate and use a read-only, stable view of a database. The database snapshot provides this capability without the overhead of creating a complete copy of the database or storage requirement. As the primary database diverges from the snapshot, the snapshot gets its own copy of original pages when they are modified. The snapshot may be used to recover an accidental change to a database by simply reapplying the pages from the snapshot to the primary database.
SQL Server 2005 Replication. MarketWatch uses a publisher/subscriber model of distribution in which information from around the world is first loaded into the market data master databases in New York and New Jersey and then distributed to the company’s subscribing databases using SQL Server 2005 Replication. The subscribing databases are in turn used to respond to customer requests and services. Replication is also used between the two master market data databases in New York and New Jersey.
SQL Server 2005 Online Indexing. The company plans to use the Online Indexing feature of SQL Server 2005 to reduce the need for scheduled downtime. Online Indexing allows reorganizing data stored in SQL Server 2005 without taking the system offline, which is crucial for maintaining high availability and uptime with a system that's as mission-critical to MarketWatch as its market data master database.

The MarketWatch.com site supports more than five million unique visitors per month, integrating market data from more than 120 sources around the globe.

Benefits
Upgrading to SQL Server 2005 is giving MarketWatch the database platform it needs to deliver its high-value financial news and investment tools and position itself for the future. In addition to looking ahead to using SQL Server 2005 Database Mirroring, the site is enjoying benefits including the ability to support more memory, scalability, and an efficient development environment.

Enhanced Availability with SQL Server 2005 Database Mirroring
MarketWatch is planning on using the Database Mirroring feature of SQL Server 2005 to enhance high availability. Database mirroring allows continuous streaming of the transaction log from a source server to a single destination server. In the event of a failure of the primary system, applications can reconnect to the database on the secondary server almost immediately, without waiting for recovery to finish. The secondary instance detects failure of the primary server within several seconds and accepts database connections almost immediately after detecting a failure. Unlike failover clustering, the mirrored server is fully cached and ready to accept workloads because of its synchronized state.

“Database Mirroring offers total synchronization,” says Martin Petersen, Director of Operations at Dow Jones & Company. “One of the difficulties we have with replication is making sure that we stay perfectly in-sync with all of the stored procedures, rules, and tables. As we move forward, we’re constantly having to update table structures and add new information tools to meet the needs of our customers. There’s always a challenge when you get into large databases to make sure that you’ve replicated all the table changes, stored procedures, rules, and other elements to your alternate master. Database Mirroring solves that problem for us.”

Database Mirroring offers total synchronization. Martin Petersen Director of Operations Dow Jones & Company

Ability to Support More Memory
MarketWatch.com prides itself on providing the best possible performance for its users. This means that the company needs the best database running on the fastest operating system, and hosted on the most efficient hardware. The company felt memory constrained with the earlier version of its database, which was limited to supporting    2 GB of physical memory.
The 32-bit version of SQL Server 2005 supports up to 64 GB of RAM, while the 64-bit version, used by MarketWatch for its market data master databases, supports up to 1 terabyte of RAM. “With the old limit of two gigabytes of physical memory we felt constrained,” says Petersen. “SQL Server 2005, especially with the 1 terabyte of RAM supported in the 64-bit version, opens the door for substantial memory expansion.”

Scalability
Immediacy is crucial for MarketWatch.com users making financial investment decisions, which makes scalability a critical element of success for the company. The company uses Microsoft products across its operations to enhance scalability, and takes advantage of the Clustering Service and Load Balancing built into the Windows Server 2003 operating system. MarketWatch can scale physical memory up to 1 terabyte.

From a development standpoint, MarketWatch benefits from using Microsoft Visual Studio® 2005 development system and the Microsoft .NET Framework version 2.0. The .NET Framework is an integral component of the Microsoft Windows operating system that provides a programming model and runtime for Web services, Web applications, and smart client applications.

“Visual Studio and the .NET Framework, along with SQL Server 2005, have been very important to us in creating scalable solutions based on a services oriented architecture (SOA),” says Kent Tangen, Director of Engineering and Product at MarketWatch.com. “The .NET Framework gives us what we need to efficiently implement services.”

Moving to an SOA approach to development has helped the company meet scalability needs by providing more precisely targeted solutions. “We’ve been able to break up our resources to target them more efficiently,” says Petersen. “Instead of having multiple homogenous Web severs each serving up the all aspects of a product, for example, we now have groups of servers that are load-balanced for the different modules that make up a product. This allows us to more precisely scale to meet traffic demands. We can specifically react to the scaling needs of the news service, or to the needs of the market data service, page layout, service, authentication service, and so on.”

Efficient Development Environment
MarketWatch‘s internal development team creates its own applications. The need to constantly meet customer needs by providing breaking news, dynamically-updating stock information and innovative community tools means that the team requires an efficient development environment—which is what the company has found with Visual Studio and the .NET Framework.

“The .NET Framework does a really good job of implementing services,” says Tangen. “With the .NET Framework and ASP.NET, creating Web services is fairly seamless.”

For More Information
For more information about Microsoft products and services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada Information Centre at (877) 568-2495. Customers who are deaf or hard-of-hearing can reach Microsoft text telephone (TTY/TDD) services at (800) 892-5234 in the United States or (905) 568-9641 in Canada. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information using the World Wide Web, go to: http://www.microsoft.com/

For more information about Scalability Experts products and services, call (800) 304-8221 or visit the Web site at: http://www.scalabilityexperts.com/

For more information about MarketWatch products and services, visit the Web site at: http://www.marketwatch.com/

Microsoft Server Product Portfolio
For more information about the Microsoft server product portfolio, go to:
www.microsoft.com/servers/default.mspx

Microsoft SQL Server 2005
Microsoft SQL Server 2005 is comprehensive, integrated data management and analysis software that enables organizations to reliably manage mission-critical information and confidently run today’s increasingly complex business applications. By providing high availability, security enhancements, and embedded reporting and data analysis tools, SQL Server 2005 helps companies gain greater insight from their business information and achieve faster results for a competitive advantage. And, because it’s part of Windows Server System, SQL Server 2005 is designed to integrate seamlessly with your other server infrastructure investments.

For more information about SQL Server 2005, go to:
www.microsoft.com/sqlserver

© 2007 Microsoft Corporation. All rights reserved. This case study is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. Microsoft, Visual Studio, Windows, the Windows logo, Windows Server, and Windows Server System are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. All other trademarks are property of their respective owners. Document published January 2007