FileControl Partners, with its hosted FileControl Knowledge & Litigation Management System, helps Fortune 500 companies and other organizations around the world work with millions of pages of documents. Attorneys needing to research millions of pages of documents need incredibly fast full-text search capabilities to identify relevant information. To enhance search speeds across its more than 2 billion rows of data, FileControl upgraded its combined 1 terabyte of databases to Microsoft® SQL Server™ 2005 (64-bit), replacing its earlier deployment of SQL Server 2000. Since upgrading to SQL Server 2005, the company has enjoyed full-text searches that run 25 times faster than before, and has seen a 300 percent increase in the rate at which it can insert new data. The company also enjoys easier maintenance, as re-indexing that once took three hours is completed in less than 10 minutes.
FileControl Partners, creators of the FileControl Knowledge and Litigation Management System, helped pioneer the field of Web-based knowledge and litigation management technology, with its solution that provide a paperless and powerful data search and retrieval mechanism with controlled access to critical corporate and legal documents.
||It used to take two or three hours for us to re-index our terabyte of data. With SQL Server 2005, the same re-indexing takes less than 10 minutes.
President and Chief Technology Officer
The FileControl solution, used by a number of Fortune 500 corporations, resolves one of the great challenges that large organizations face: How to store, search, and work with litigation documentation that can total millions of pages. The FileControl solution enables users to securely store, search, annotate, produce, print and email selected documents without fear that they can be accessed by unauthorized users. “We have some search functions that took 20 to 25 seconds to complete using SQL Server 2000, which now take less than a second since moving to SQL Server 2005.” Denis Lui, Technical Director, FileControl Partners.
Full-text search performance is one of the most critically important elements of the FileControl solution because of the immense volume of information that must be searched and the time constraints that often affect attorneys and other members of an organization. FileControl initially deployed its solution using Microsoft® SQL Server™ 2000 database, running on the Microsoft Windows® 2000 Advanced Server operating system, and hosted on a Dell PowerEdge 2850 with two processors and 16 megabytes (MB) of RAM.
FileControl offers its solution as an application service provider giving each customer its own instance of SQL Server, though some of its customers host their own FileControl deployment behind their corporate firewall. SQL Server 2000 proved itself as a stable and enterprise-grade database, however FileControl sought even faster full-text search performance.
FileControl Partners upgraded its server infrastructure to SQL Server 2005 Enterprise Edition (64-bit) running on the Microsoft Windows Server® 2003 Enterprise Edition (64-bit) operating system. The decision to upgrade was made after FileControl developers visited the Microsoft SQL Server Customer Labs in Redmond, WA, where the company, using its own data, saw the improvements in full-text search that SQL Server 2005 provides.
The company hosts and manages the entire application from its maximum-security, fully-redundant, carrier-class data center, which runs over an OC-192 backbone, and is Department of Justice Compliant. The solution can also be hosted from within a corporation's facilities.
The company also upgraded its FileControl solution using the Microsoft Visual Studio® version 6.0 development system and COM+, writing code using Microsoft Visual Basic® and Microsoft Visual C++® and is in the process of upgrading to Visual Studio 2005 and the Microsoft .NET Framework version 2.0.
The FileControl solution enables attorneys to work with organize and produce millions of documents, including email, native files, and TIFF images efficiently by uploading and indexing them into FileControl’s Cabinets. Cabinets like physical folders are hierarchical structures, containing case specific information that can be easily identified and categorized—based on its significance, priority and relevance to the case. Key evidence can then be easily located using FileControl’s powerful search tools and shared with other law firms in the form of productions, whenever the need arises. Once the useful information has been extracted, it is stored within FileControl folders for permanent storage. Users can also manage associations from FileControl Cabinets to FileControl folders by mapping category properties.
||We have some search functions that took 20 to 25 seconds to complete using SQL Server 2000, which now take less than a second since moving to SQL Server 2005.
FileControl’s Knowledge and Litigation Management Solution provides interfaces for uploading and manipulating files within the FileControl document repository. The users with appropriate security rights can then access, move, copy, delete, annotate, print and email these document files. Search results can be refined by specifying search criterion based on contents, cabinets, cabinet properties (such as system and custom, categories and date ranges).
The solution also supports online collaboration for multiple reviewers. It provides them the ability to store, review, analyze and search electronically produced documents in their native formats, saving time and effort associated with the review process. That critical case specific information can then be sorted, centralized and shared securely with different litigation firms involved.
The company used an n-tier architecture in creating its solution, which includes:
Presentation Tier. FileControl created a browser-based presentation tier that enables users to log onto the site across the Internet, using security built into the Windows Server operating system. The presentation tier is co-hosted with the application tier.
Application Tier. The application tier integrates with the Full-Text Search feature of SQL Server 2005, adding additional functionality that, among other things, gives the user the ability to specify metadata as part of the search criteria, including any data elements outside of the content search. The FileControl solution can include text documents, PDF, e-mail, spreadsheets, presentations, and more than 350 other file types. In the FileControl application, scanned documents are parsed by an OCR module and the resulting text pages are stored in SQL Server. The application tier is hosted on a Dell PowerEdge 1850 with two processors and 1 GB of RAM.
Data Tier. The text pages are classified into documents and then organized into user-definable folders, and this information is stored in SQL Server relational tables. The resulting folder structures are hierarchical: although each page belongs to only one document, documents may be placed in more than one folder. Users can create their own custom metadata to store information and notations concerning folders, documents, and pages. This information is stored in SQL Server tables. Data is stored on a number of databases hosted on two instances of SQL Server 2005. Total SQL Server 2005 storage is about 1 terabyte. The largest table has more than two billion rows. Each SQL Server instance is hosted on a Dell PowerEdge 6850 computer with four processors and 32 GB of RAM.
Document Storage. Documents that aren’t already in digital format are scanned and then processed by optical character recognition (OCR) software, and then stored as TIFF image files. The original scanned document images are stored outside SQL Server as TIFF files on a file server. FileControl has more than 5 terabytes of TIFF images on a storage array with attached file server. The TIFF files are stored using the FileControl directory structure with reference pointers in the database.
The system takes advantage of security features built into Windows Server to help protect the intellectual property of its customers, allowing only authorized users access. In addition, audit trails and user logs are maintained to trace document access.
Upgrading to SQL Server 2005 running on Windows Server 2003 has given FileControl Partners several benefits, including 25 times faster full-text search, three-times faster data inserts, easier and faster maintenance, and enterprise-grade dependability. The company has also benefited from the ease of development inherent to working with Microsoft development tools.
25 Times Faster Full-Text Search
Moving to SQL Server 2005 gave FileControl the boost it needed in full-text search speeds. “We have some search functions that took 20 to 25 seconds to complete using SQL Server 2000, which now take less than a second since moving to SQL Server 2005,” says Denis Lui, Technical Director at FileControl Partners. “SQL Server 2005 seems to be much faster in doing joins, which cuts the speed of performing full-text searches. As soon as we migrated to SQL Server 2005, our performance problems disappeared.”
Any reduction in searching speed makes a difference to users.
“When an attorney is searching through millions of pages of documents, any time savings is a big deal,” says Ahmad Mian, President and Chief Technology Officer of FileControl Partners. “Waiting 15 seconds for a response is too long when you are spending days going through 10 to 20 million pieces of [digitized] paper. Reducing a search from 20 seconds to 1 second, results in a huge performance gain.”
Three-Times Faster Data Inserts
||When we leave work in Texas, our customers in Singapore are just arriving. There is no downtime. Our global customer base is always working, and SQL Server has never let us down.
President and Chief Technology Officer
FileControl has enjoyed more than a 300 percent improvement in the speed of inserting new rows into its database since upgrading to SQL Server 2005. “With our old SQL Server 2000 deployment we could insert about 100 million rows of OCR data into the database in 24 hours,” says Ahmad. “Moving to SQL Server 2005, we have tripled our OCR data inserts to 360 million rows in 24 hours.”
Easier and Faster Maintenance
SQL Server 2005 has proven easier to maintain than the earlier database, including faster re-indexing and faster backups. Re-indexing is important to FileControl because tables must be kept updated to maximize search performance. “We have thousands of users and records are constantly being added and deleted, so table re-indexing is an important maintenance issue,” says Mian. “It used to take two or three hours for us to re-index our terabyte of data. With SQL Server 2005, the same re-indexing takes less than 10 minutes.”
The company is also enjoying faster data backups. “Backing our terabyte of data used to take about three hours,” says Lui. “With SQL Server 2005, our backup only takes about an hour.”
FileControl has always been able to depend upon SQL Server to provide enterprise-grade reliability, which is essential to its operations. “We serve very large clients around the globe, and the services we offer are mission-critical to their operations,” says Mian. “We cannot afford to be down, at all. We have no maintenance windows because we serve customers in 10 different countries through our hosted ASP platform. When we leave work in Texas, our customers in Singapore are just arriving. There is no downtime. Our global customer base is always working, and SQL Server has never let us down.”
Ease of Development
Efficient development is important to FileControl because the company creates its own applications. Based on its success using earlier versions of Visual Studio, the company is in the process of upgrading its development environment to Microsoft Visual Studio 2005 Team System, which extends Visual Studio’s integrated and productive experience from the developer to the entire development team by providing powerful new role-based tools for software architects, developers, testers and project managers.
“When we evaluated the Team System, we were especially impressed by how it ties test-use cases and unit testing into the full development cycle,” says Lui. “We think that moving to Visual Studio Team System will help us to develop our applications more quickly while enabling us to retain our high standards for quality. We are also looking forward to working with the Microsoft .NET Framework, and see special value for us in taking advantage of the ASP.NET component.”
Microsoft Server Product Portfolio
For more information about the Microsoft server product portfolio, go to:
Microsoft SQL Server 2005
Microsoft SQL Server 2005 is comprehensive, integrated data management and analysis software that enables organizations to reliably manage mission-critical information and confidently run today’s increasingly complex business applications. By providing high availability, security enhancements, and embedded reporting and data analysis tools, SQL Server 2005 helps companies gain greater insight from their business information and achieve faster results for a competitive advantage. And, because it’s part of Windows Server System, SQL Server 2005 is designed to integrate seamlessly with your other server infrastructure investments.
For more information about SQL Server 2005, go to:
© 2006 Microsoft Corporation. All rights reserved. This case study is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. Microsoft, Visual Basic, Visual C++, Visual Studio, Windows, the Windows logo, and Windows Server are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. All other trademarks are property of their respective owners. Document published September 2006