4-page Case Study - Posted 6/11/2007

Views: 4164

Rate This Evidence:

Newegg.com

Leading Internet Retailer Builds an Enterprise Service Bus Using In-House Technology

With plans to replicate its existing e-commerce infrastructure in China, Newegg.com wanted to avoid reusing the point-to-point, database-level integration techniques that it had used in the past, which had made accommodating new business processes slow and costly. Instead, the company is building an enterprise service bus (ESB), which combines service-oriented architecture principles with an event-driven, standards-based messaging engine to provide a comprehensive integration framework that will improve all aspects of the software development life cycle. By building its ESB using technology built into the Windows® platform instead of buying one based on Java and/or open source software, the company will realize the benefits of a comprehensive integration solution with lower initial costs, reduced long-term complexity and costs, and minimal additional developer training.

Situation

Since shipping its first order in 2001, Newegg.com has grown to become a leading online retailer of computer, communications, and consumer electronics products. Its Web site, which receives some 11 million unique visitors per month, generated more than U.S.$1.3 billion in sales in 2005. A 30 percent increase over the previous year, that figure earned Newegg.com the ranking of the tenth-largest retail Web site for 2005, according to Internet Retailer Magazine.

The company’s funding and management come primarily from China, where Newegg.com maintains a team of more than 100 software developers, who handle most such work for the company’s U.S. operations. The Newegg.com IT infrastructure runs primarily on the Windows Server® 2003 operating system, and, with the exception of the company’s SAP financial system, all key business applications have been developed in-house. Newegg.com is now focused on expansion into China, where the company will need to duplicate its U.S. IT infrastructure, including new e-commerce sites and more than a dozen supporting back-end systems.

We realized that, with Windows, we already had all the components of an ESB in-house, and that we just had to assemble them. James Wu Principal Architect, Newegg.com

“Most integration so far has been point-to-point and done primarily at the database level,” says James Wu, Principal Architect at Newegg.com. “Even relatively simple changes were hard to implement, usually involving several different development teams sitting down to coordinate and make sure that any necessary changes at the database level wouldn’t break anything else. In building out our new overseas presence, we wanted a more scalable architecture that would allow us to implement or change business processes in hours or days, not weeks or months.”
 
Given its integration needs, Newegg.com first began looking at a service-oriented archi-tecture (SOA). Ultimately, however, the com-pany decided that a more comprehensive approach would be an enterprise service bus (ESB) that combined SOA principles with an event-driven, standards-based messaging engine to provide a comprehensive inte-gration framework—in essence, an “SOA accelerator,” as Wu puts it.

Newegg.com spent several months evaluating potential ESB vendors, including BEA, TIBCO, Sonic, and IBM. However, none offered a good fit with respect to the underlying technology platform or the company’s budget. “Most of the solutions were based on Java, Linux, or both, but we wanted to stay with Microsoft® .NET technology, which we already know and trust,” says Wu. “In addition, the Java-based solutions all were very expensive to implement.”

Solution

A better solution presented itself in the fall of 2006, when conversations with the company’s platform strategy adviser at Microsoft revealed that Newegg.com could easily build its own ESB on the Windows® operating system platform. “We realized that, with Windows, we already had all the components of an ESB in-house, and that we just had to assemble them,” says Wu. “An approach based on Windows was preferable for several reasons, including lower costs, a shorter learning curve, and the ability to get every-thing we need from a single, current vendor.”

Components of the new ESB—currently in development in China as part of the effort to build out the company’s IT infrastructure for that region—will include:

• Windows Server 2003, which provides a solution foundation with built-in support for messaging technologies such as Message Queuing and Web services.
  
  
• Microsoft BizTalk® Server 2006, which is used for workflow orchestration and enter-prise application integration (as provided by the Microsoft BizTalk Adapter for SAP).
  
  
• Microsoft SQL Server™ 2005 database software, which provides for data storage and the logging of all transactions.
  
  
• Windows Communication Foundation, a part of Microsoft .NET Framework version 3.0, which provides a consistent, extensible programming interface for all messaging technologies.
  
  
• The Microsoft Visual Studio® 2005 Professional Edition development system, which provides an integrated software develop-ment environment for all ESB solution components.

Proof of Concept

Before beginning full-scale development, Newegg.com chose to validate its newfound approach with a proof of concept. The com-pany enlisted the aid of Neudesic, a Microsoft Gold Certified Partner with a track record of implementing ESB solutions based on Microsoft technology. During the three-week proof-of-concept effort, in which Newegg.com used its existing return merchandise authorization (RMA) process as a use case, the company validated that an ESB approach based on the aforementioned Microsoft technologies could meet three criteria:

• Synchronous submittal of an RMA request. Developers implemented a Web page based on Asynchronous JavaScript and XML (AJAX), on which a customer could submit an RMA request. When the user clicks “Submit,” the Web page packages the request as a Web service call and sends it to an endpoint based on Windows Communication Foundation exposed by the ESB. Windows Communication Foundation unpackages the serialized XML message and creates an RMA request object, which is handed off to a Windows service that validates the RMA request and creates an update in the company’s RMA system. In the proof of concept, the workflow to process the RMA request object was left incomplete; in the production implementation, the workflow is orchestrated by BizTalk Server 2006.
  
  
• Asynchronous status updates to the user interface. On the Web page, an AJAX control periodically sends status update requests to a second Windows Communication Foundation endpoint. The endpoint invokes code to get the status of the RMA request from the RMA system and returns that information back through the same endpoint to the AJAX control. By using AJAX in this way, Newegg.com can decouple the RMA process from the Web-based user interface. The user is able to continue to use the Web site as the RMA request is processed yet remain aware of the status of the RMA request.
  
  
• Multicast messaging to supporting systems. Following validation of the RMA request, the ESB must coordinate updates to a warehouse system, the company’s financial system, and an e-mail notification system. The RMA request is passed to a multicast message service exposed through a third Windows Communication Foundation endpoint, which uses Communication Foundation callback operations to send messages to each of the three supporting systems and employs Message Queuing as the transport mechanism to ensure reliable message delivery.

Figure 1. In the company’s ESB implementation, Windows Communication Foundation exposes workflows driven by BizTalk Server.

“With Windows Communication Foundation, we were able to implement multicast messaging with just 100 or so lines of code,” says Curt Peterson, an Enterprise Solution Architect and Distinguished Engineer at Neudesic. “Developers don’t need to know much about message queuing, and, because the underlying transport mechanism to be used is specified in Windows Communication Foundation configuration files, they can switch to a different mechanism, such as a peer channel, without changing a single line of code.”

Implementation

After completing the proof of concept, the developers who had traveled from China returned home to share with their colleagues what they had learned. Full-scale development began in March 2007 and is expected to be completed in June 2007, during which the 30 to 40 developers working on the project will use the ESB architecture and programming techniques from the proof of concept to integrate the various systems. Newegg.com will then begin to migrate its U.S. presence onto the new ESB architecture, replacing existing point-to-point integrations one business process at a time.

As part of the project, Newegg.com also will switch from using SAP Exchange Infrastructure (XI) technology to the BizTalk Adapter for SAP as a means of integrating with SAP. “BizTalk Server is already a core component of our new ESB architecture, so it makes great sense to use it to integrate with SAP as well,” says Wu. “Besides, we’ve found the BizTalk Adapter for SAP to be both easier to use and better-performing than the SAP XI technology we adopted last year.”

Benefits

With its new approach to integration, Newegg.com will have an IT infrastructure that can better support future growth and change. “With an ESB, our entire infrastructure will become simpler and scale more efficiently,” says Wu. “By implementing our ESB using in-house Microsoft technology, we’ll be able to realize those benefits with significantly lower costs and complexity—both in terms of initial deployment and ongoing development, training, support, and so on. Had we chosen a packaged solution based on Linux, we would have had to pay at least several hundred thousand dollars to acquire it and then spend a good deal more to learn how to use and support it.”

Improved Software Development Life Cycle

By switching to an ESB-based approach, Newegg.com will be able to integrate its systems faster and less expensively than with point-to-point integration. In addition, the move will ease application maintenance because changes to one system won’t be as likely to break another system. Finally, the ESB will enable business system analysts who work in the United States to more easily coordinate the implementation of new or changed business processes with overseas developers.

“From a short-term perspective, it would have been faster and cheaper to stay with point-to-point integration, duplicating our existing IT infrastructure in China,” says Wu. “We view our ESB as a long-term investment—one that will benefit the entire development life cycle, from defining integration requirements and coding those changes, to their formal testing and ongoing support. In the past, we had to focus on where the data resided and in what form. In the future, we’ll be able to work at a business process level, supporting change through workflows that are designed graphically using BizTalk Server instead of with extensive coding.”

Lower Initial Costs

By building its own ESB using the Windows platform, Newegg.com will realize the benefits provided by its new integration solution but will incur significantly lower up-front costs. “Many of the packaged, Java-based ESB solutions we examined required purchasing the required modules and adapters separately—and that added up fast,” says Wu. “For example, one of the leading J2EE-based solutions we were looking at would have cost several times more to deploy a two-processor, failover system, along with the necessary SAP and Web service adapters. With a Windows-based ESB, our initial costs will be a fraction of what we would have paid for a packaged, Java-based ESB.”

Reduced Complexity and Long-Term Costs

With its Windows-based ESB, Newegg.com will avoid introducing new technology into its IT infrastructure, thereby keeping both complexity and associated long-term costs to a minimum. “A key benefit of our decision to use only Microsoft technology is that we get everything we need from a single vendor with whom we already have a relationship,” says Wu. “With Microsoft software, we get a preintegrated set of software, a common programming model, and an integrated development environment, none of which would have been provided by an open source solution. That not only will help accelerate the initial development of the solution, but also will minimize costs and complexity in other areas, such as ongoing support, operations, and vendor management.”

Reuse of Existing Skills and Tools

Newegg.com’s decision to stay with a Windows-based solution also will maximize the company’s investment in developer skills. “We’re able to work with a consistent, familiar set of software, programming model, and development environment—all things on which our developers are already trained,” says Wu. “Had we introduced a Java-based or Linux-based solution, we would have had to train both developers and IT operations people on the use of that new technology. For example, we’ve found the BizTalk Adapter for SAP to be easier to use than SAP XI technology. Along with increased performance, that ease of use is another reason that we’ve adopted the native capabilities of BizTalk Server for that aspect of our ESB architecture.”

Microsoft Visual Studio 2005
Microsoft Visual Studio 2005 is the world’s most popular development environment for designing, developing, and testing next-generation Windows-based solutions and Web applications and services. By improving the development experience for Windows, the Web, mobile devices, and Microsoft Office, Visual Studio 2005 helps organizations deliver a variety of solutions more productively than ever before. Visual Studio Team System expands the product line with new software tools that enable greater communication and collaboration throughout the development life cycle. With Visual Studio 2005, businesses can deliver modern service-oriented solutions more efficiently.

For more information about Visual Studio 2005, go to:
msdn.microsoft.com/vstudio

The Newegg.com solution is based on the Microsoft ESB Guidance, which was developed in conjunction with Neudesic and can be found at:
www.microsoft.com/biztalk/solutions/soa/esb.mspx

In June 2007, Microsoft Patterns and Practices will make the guidance broadly available through CodePlex at:
www.codeplex.com/esb

For More Information

For more information about Microsoft products and services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada Information Centre at (877) 568-2495. Customers who are deaf or hard-of-hearing can reach Microsoft text telephone (TTY/TDD) services at (800) 892-5234 in the United States or (905) 568-9641 in Canada. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information using the World Wide Web, go to:
www.microsoft.com

For more information about Neudesic products and services, visit the Web site at:
www.neudesic.com

For more information about Newegg.com products and services, visit the Web site at:
www.newegg.com