Publishers finesse multiple metrics

The result of this abundance has been that unlike their counterparts in traditional media, publishers working online routinely subscribe to both panel‐based and census‐based measurement services — that is, to multiple, incompatible estimates of audience size. Each of the newspapers interviewed for this report (though notably not the online‐only news site, TPM) subscribed to either Nielsen or comScore, or to both, while also relying on a server‐side Web analytics package, usually Omniture. Most also incorporate at least one additional source of audience data, such as Scarborough, Hitwise, Google Analytics, or Alexa, into their internal analysis and their pitches to advertisers. However, impressions of the relative merits of these data sources vary widely. Some publishers find server‐side data much more reliable. The Providence Journal subscribes to comScore, but sees hard‐to‐credit fluctuations in its online audience from quarter to quarter. As a result, the paper relies on audits of its server traffic, collected via Omniture, to come

up with its official online readership. The Journal relies on comScore data mainly for “product development” — to gauge the success of niche sites among particular demographic targets. Similarly, the Miami Herald uses comScore and also used to subscribe to Nielsen. But Hirsch reports that his paper’s position in either of the panelbased rankings varies for no apparent reason. “I don’t know when to believe them,” he says. Meanwhile traffic recorded by the Herald’s own servers, analyzed with Omniture, tends to match his own editorial sense of when certain stories, or entire editions, are commanding a great deal of attention in Miami. As an example, Hirsch points to January of 2010, the month of the Haitian earthquake that claimed an estimated 230,000 lives. “We know our traffic went through the roof, because of our history of coverage in the region,” Hirsch says. The paper’s internal figures matched expectations: as recorded by Omniture, traffic spiked 36 percent over December, to 35 million pageviews, while unique visitors jumped 11 percent, to almost 6 million people. Meanwhile, though, comScore recorded less than half as much traffic for January, and fewer than an third as many unique visitors. percent the month of the earthquake — and falling again in February, despite the fact that Miami hosted the Super Bowl that month. At larger national papers, the story is somewhat different. The Washington Post relies on both comScore and Nielsen data to understand how it fares against major competitors online, while also using server‐side traffic figures for internal strategic analysis. Managing editor Raju Narisetti acknowledges that the cacophony of competing measurements has been a serious issue, with both the panels undercounting the Post’s audience. “However, over time you can recognize the strengths and weaknesses of each and start to understand how one approach relates to the other,” he explains. “It is less of an issue now” — in part because of the hybrid measures in the works from comScore and Nielsen, which bring these audience estimates closer to internal data. The Wall Street Journal subscribes to Nielsen, comScore, and Hitwise, in addition to using Omniture for server‐side analysis. Kate Downey, the paper’s director of “audience analytics & insights,” observes that the Nielsen and comScore ratings of wsj.com rarely agree with each other, or with the Journal’s own records. However, she emphasizes that server data is also unreliable and prone to double‐counting; to make their case to advertisers, salespeople rely mainly on demographic data from the panels and on the Journal’s own registration records (all the more valuable since much of the site is behind a paywall). She appreciates having multiple sources of data at her disposal, each with its own strengths and weaknesses, and suspects that many of her peers at other papers agree. “People use whatever numbers look good that month. It gives publishers some flexibility,” Downey explains. “I think if everybody had the same numbers, we would hate that even more.” Talking Points Memo sees the same dramatic divergence in audience estimates. Google Analytics counted 1.8 million unique visitors for a recent 30‐day span, while comScore typically gives it in the neighborhood of 300,000 visitors per month. But unlike its peers in the newspaper business, TPM’s response is to ignore the panels outright — the site subscribes to neither comScore nor Nielsen, counting on advertisers and agencies to supply panel‐based figures if they consider them necessary to the conversation. (For demographic data, TPM relies on its own, voluntary audience surveys; every six months or so founder Josh Marshall issues an appeal to readers, culling about 1,000 responses within 12 hours.) “The panel‐based numbers are atrocious,” says Karimkhany flatly, pointing out that most of TPM’s traffic comes from the workplace, which the panels don’t capture well. “But as long as they’re equally inaccurate for our competitors, it’s okay. It’s something we live with.”