The Associated Press (AP) strives to be “the essential global news network.” Every day, AP member organizations deliver news to more than half of the world’s population. With the shift in the journalism industry to an online viewership, the AP needed to help its member organizations capitalize on the content they publish online. The not-for-profit designed a news tracking system, and using Microsoft SharePoint Server 2010, gave publishers the power to track where and how their content is used so that they can best meet audience demand and make better decisions as to how to license, sell, protect, and extend their content. The system will allow the AP to help members interpret massive amounts of data in easy-to-understand reports. The AP created the portal in just 70 days for a very fast time-to-market, and expects ongoing development and administration to be highly streamlined.
The Associated Press (AP) is a not-for-profit news cooperative owned by its 1,500 newspaper and broadcast member organizations, which, collectively, serve thousands of daily newspaper, radio, television, and online customers with text, photos, graphics, and videos. AP reporting is known for its objective, accurate, high-quality coverage, provided by approximately 3,700 employees—two-thirds of them journalists—working from more than 300 locations worldwide. The AP has received an incomparable 49 Pulitzer Prizes, and 30 photo Pulitzers.
Over the past decade, the AP and the member organizations it comprises have been working to adapt to an online world. “The news delivery paradigm has evolved, as leading daily newspapers have shifted to online models,” says Riyad Omar, Business Director for the News Registry at the AP. News today is delivered through a wide range of digital formats and protocols, over the internet and over satellite. News is viewed on a number of digital devices, including PCs and mobile phones, internet-connected televisions, and electronic readers.
“The AP has unique visibility into the nation’s most-read 1,500 newspapers, and a unique insight into what frustrates them,” continues Omar. “Essentially, these publishers are seeing viral publication of their content.” For example, a third-party news provider can sign up for a publisher’s news feed, and deliver the news content in that feed to its users, through its own custom-built news application. “Publishers cannot always see where their content is ending up, on the Web,” Omar says. “We want to help publishers understand how people are getting access to their news, so that they can optimize the delivery of it to satisfy consumer demand, and so that they can track it and monetize it.”
The AP, while known for its excellent news coverage, has also historically taken advantage of technology to propel its journalism and bolster the industry. The AP wanted to use its technology expertise and leadership to give its members the ability to track content online so that members can make better decisions about how best to deliver, enhance, and extend the content, and how to license, sell, and protect it for broader monetary gain.
The AP designed a news tracking system to help publishers identify each news piece by type, track it to determine where it was located on the Internet, which sites were repurposing it for viewer consumption, which devices viewers were using to access the content, how often it was being viewed, and where the viewers were located. The not-for-profit planned to make this information available through a Web-based portal that offered easy-to-understand, graphical analysis tools. Developing a News Tracking System
The IT team received approval from AP management to implement the News Registry, and quickly went to work assembling a development team, designing an architecture, building the service, and launching it in just 70 days. “To attack this challenge iteratively, we needed to choose an underlying technology that would let us develop efficiently, and that would accommodate tremendous scale,” says Todd Martin, Vice President and Director of Technology at the AP. Current usage statistics indicate that 75 to 150 million unique visitors a month view AP content online. “That translates into billions of story views each month.”
|Figure 1. Overview of publisher’s content impressions through|
Web-based News Registry portal.
The AP used Microsoft products and technologies to create the storage and tracking foundation for its news tracking system, by which it could gather data about publishers’ online content. The AP had already used Microsoft Office SharePoint Server 2007 with success in other areas of the company, so it chose the product to help users access and interpret the volumes of data stored in the news tracking system. When prerelease code for SharePoint Server 2010 became available, the AP felt that it could take advantage of many of the products’ advancements
. The organization had already started using the newer version to implement a new, newsroom text-editing application, and so it transitioned to SharePoint Server 2010 for the News Registry project as well. “We believed that using SharePoint Server 2010 as the basis for our portal would allow us to tap into more out-of-the-box functionality, rather than building each component ourselves—including data visualization, user authentication and authorization, and content management,” says Martin. “Also, the monitoring and administrative capabilities would help us efficiently operate the platform once it was in production.”
Between July 2009 and November 2009, the AP developed and implemented the new system. The AP uses the Microsoft SQL Server 2008 data management software as the underlying data repository for the system. The AP can extract online analytical processing cubes containing publishers’ data from SQL Server 2008, and, using the data visualization features of SharePoint Server 2010, make the data accessible through a Web-based portal. (See Figure 1.) “SharePoint Server 2010, with its many built-in features, helped us deliver the first release in 70 days,” says Jonathan Malek, Director of Research at the AP.
Enhanced integration between SharePoint Server 2010 and the Microsoft Visual Studio development system tools made it possible for AP developers to vastly accelerate their development process. “The developer experience has been thought out very carefully in SharePoint 2010 and Microsoft Visual Studio 2010. Developers can write code and start the debugging process, and Visual Studio 2010 bundles everything up and deploys it. It’s really a powerful set of tools,” says Malek.
The AP used the Microsoft Silverlight 3.0 browser plug-in as the foundation for the portal’s user interface. “Silverlight helps us process information quickly and present it to users in a meaningful way. They can drill into the data to form a better understanding of where their content is being used,” says Malek.
The AP is also using Microsoft PerformancePoint Services, a performance management service included in SharePoint Server 2010, to transform raw data into visual dashboards that are accessible through the Silverlight-enabled portal. The not-for-profit is also taking advantage of SQL Server PowerPivot for SharePoint to give the News Registry staff the ability to analyze these massive amounts of data to identify trends and publish reports. “Our next step is for our analysts to use PowerPivot technology to uncover data and relationships hidden in the data warehouse,” explains Malek.
|Figure 2. Content impressions reported in the|
News Registry portal, shown by geographic region.
Security is enabled through the authentication and authorization capabilities of SharePoint Server 2010, specially, the claims-based security model. “SharePoint Server 2010 provides a great deal of out-of-the-box functionality in terms of access control, management of users and groups, and implementation of rules,” says Malek. “This was key to meeting our deadline, because it meant that we didn’t have to build a separate identity management application.”
Finally, the AP, which has relied on Microsoft System Center Operations Manager 2007 to monitor its original Office SharePoint Server 2007-based sites, also uses this management software to monitor and maintain the health and performance of the Web-based portal. Giving Users Insight into How Content Is Used
The solution, called the AP News Registry portal, gives publishers unprecedented visibility into how their content is being used on the Web, which, in turn, helps them to make better publishing and licensing decisions, and helps them discover new business opportunities.
All content published online by News Registry participants includes a usage tag. Online interaction with that content is processed and analyzed in the News Registry warehouse. Participants logging on to the portal have access to visual reports (see Figure 2) that show where each news article is being used on the Web, and how many times each article has been viewed. Reports include detailed statistics, such as how many unique visitors are accessing any given article. “If an article has been accessed 1 million times,” explains Martin, “a user can determine whether this means 1 million unique readers or repeated views by a smaller number of users.” Users can see the timeframe in which page views occur, the site where the article was published, identify when content licensed to a site is not being used, determine which type of content is being consumed on which type of mobile device, and much more.
The News Registry, currently in private beta and being tested by a small group of participants, tracks text-based content, but the AP plans to add photo and video tracking capabilities soon. The AP plans to make the solution available to all 1,700 member organizations—approximately 20,000 users—and open the News Registry to all news content publishers. Once in broader use, the AP expects to be adding 6 billion rows of content tagging information each month to the News Registry database, which runs on Microsoft SQL Server 2008 data management software.
Using SharePoint Server 2010, the AP developed a News Registry portal through which it can provide member organizations with self-service access to critical data. Members can use this information to make business decisions about how to produce and license their content. The system was deployed quickly, and the AP is pleased with the improved development experience, the anticipated savings on development, operational, and maintenance costs, and the fact that the tools it is providing will help each member improve the bottom line.Self-Service Usage Data for News Publishers
||With SharePoint Server 2010, we can deliver simple yet powerful tools to help content publishers … make informed decisions about how they produce and license content.
Business Director for the News Registry, the Associated
With its News Registry portal, the AP can provide news publishers with an easy way to analyze and understand
how their content is being used. With highly detailed, near real-time usage data in hand, these publishers have better insight into consumer behavior, and are thereby better equipped to meet consumer demand. They are also better equipped to spark business-oriented conversations with Web site owners to discuss licensing terms. “With the SharePoint Server 2010–based solution, our members have, for the first time, a clear view into where and how their content is used online. This information is essential to making decisions about how to publish and how to sell that content,” says Malek. “Before, they were operating in the dark.”
“Publishers can now make better decisions about coverage, licensing, staffing, and many other business considerations,” Omar adds. “The solution provides them with aggregate information about their content, and also with highly specific drill-down capabilities. For Example, using the tool, a publisher can see not only how much of its sports content is being consumed on mobile devices, it can see how, for instance, how many times an article covering NCAA activity is viewed. This gives publishers new insight into the habits and behavior of viewers—information that was previously unavailable. “With SharePoint Server 2010, we can deliver simple yet powerful tools to help content publishers visualize and understand how their content is used, thus helping them make informed decisions about how they produce and license content,” says Omar.
Fast Time-to-Market, Lower Costs
The AP was able to create the News Registry portal in just 70 days by taking advantage of the out-of-the-box functionality of SharePoint Server 2010. Tight integration between Microsoft Visual Studio 2010 development tools and SharePoint Server 2010 also helped the AP accelerate time-to-market for its new application. “With SharePoint Server 2010, we have made deadlines that we otherwise would not have,” says Malek.
Deeper Publishing Insights for the AP
Initial deployment was fast, but just as important, the enhanced development environment will allow the not-for-profit to continue to add features and functionality to the application with a great deal of efficiency. “With SharePoint Server 2010 as the underlying development platform, each time we add to or modify the system, we will experience a much shorter life cycle from development to production. Because of this, we think it will take fewer developers and a smaller IT staff to grow and maintain the system,” says Martin.
Further, says Malek, “PerformancePoint Services in SharePoint Server 2010 enables us to easily provide rich, interactive dashboards and key performance indicators for users so that they can quickly gain an understanding of their data.”
Management costs are also low for the News Registry portal because the AP can take advantage of the software that it uses to manage the rest of its SharePoint Server–based environment. “We came to this project with Microsoft System Center Operations Manager 2007 already in place. The software will help us keep the portal highly available at a very low cost, because of the savings we expect from the consolidated platform and shared infrastructure, especially contrasted against building the platform in-house,” says Martin.
While AP is the operator of the News Registry, it is also a participating publisher. AP analysts—like any other News Registry participant—will be able to use the portal to sift through enormous amount of data for business intelligence. “Our own analysts are asking the same questions as our member organizations, about how and where AP content is being used,” says Martin. “Through the News Registry portal, AP editors, journalists, and salespeople will get the same benefits as news publishers from our member organizations.”
The anticipated 6 billion rows of content-tagging information amounts to approximately one terabyte of data that will flow into the AP data warehouse each month. “We’re confident that we can use our SharePoint Server 2010–based portal to break that information down and present it to our users in intelligible, helpful visuals and reports,” says Martin.
Microsoft SharePoint Server 2010
Microsoft SharePoint Server 2010 is the business collaboration platform for the Enterprise and the Internet.
For more information about Microsoft SharePoint Server 2010, go to:For More Information
For more information about Microsoft products and services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada Information Centre at (877) 568-2495. Customers in the United States and Canada who are deaf or hard-of-hearing can reach Microsoft text telephone (TTY/TDD) services at (800) 892-5234. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information using the World Wide Web, go to:
For more information about Associated Press products and services, visit the Web site at: