A Moment of Convergence on Metrics?

On May 15, 2014, BuzzFeed published a leaked version of what has become known as The New York Times’s Innovation Report. The document spanned nearly 100 pages and was the product of six months of reporting and research by a team of Times staffers tasked with assessing the paper’s transition to the digital age and crafting recommendations for moving forward. The report painted a picture of an organization struggling mightily to reconcile its storied print past (and present) with its digital future. Many of its findings overlapped with those of this research, such as the online edition’s persistent prestige deficit relative to that of the print edition, and the organization’s difficulties melding the online and print editions into a cohesive entity.

Metrics are an important patch of terrain where these struggles are playing out. The report urged greater use of analytics in editorial decision-making, though did not devote much space to the issue. This is because, in advance of the report’s internal release, the research team had already successfully made the case for more metrics-driven decision-making in an extensive presentation to executive editor Dean Baquet. Since then, the organization has made significant investments in growing The Times’s audience. During the fall of 2014, Alexandra MacCallum was appointed assistant managing editor for outreach. MacCallum formed a 23-person audience development team (consisting mostly of existing staffers, along with some new hires), which, significantly, set up shop in the newsroom.²⁷nascent efforts, spearheaded by the newly appointed internal “strategy team,” to broaden access to metrics among a wider swath of editorial staffers, including some reporters.

In addition, executive editor Dean Baquet is working to diminish the newsroom’s focus on the front page. In February of 2015, Baquet sent a memo to newsroom staff explaining that instead of jockeying for page one placement, desks would instead compete to make “Dean’s list,” a group of enterprise stories selected by masthead editors that would receive “the very best play on all our digital platforms.”²⁸indicates that the paper’s allocation of prestige will soon start to follow, though the process is likely to be gradual.

Meanwhile, Gawker Media underwent a series of major changes at the end of 2014. On December 2, Denton announced the ouster of Joel Johnson as editorial director, writing that if Gawker hoped to beat well-funded competitors BuzzFeed and Vox in 2015, “our talent selection and development, and our editorial plays, must be as shrewd and accomplished as the baseball management popularized by Moneyball.” This reference seemed to suggest Gawker would move in an even more metrics-driven direction. But in a subsequent post, Denton cited Gawker’s traffic-chasing as one of the reasons he felt the quality of the company’s content had suffered in 2014: “Editorial traffic was lifted but often by viral stories that we would rather mock. We—the freest journalists on the planet—were slaves to the Facebook algorithm.” Just over a month later, Denton published another blog post, this one announcing that the traffic chart displaying unique visitors to Gawker sites over time that had long adorned the wall of Gawker’s editorial floor would be taken down. Instead, the screen would show a blog of the best stories across the Gawker network, as chosen by newly appointed executive editor Tommy Craggs and his Politburo, a small group of senior editorial staffers. The Politburo would also determine sites’ bonuses based on its evaluation of their content. “A layer of subjective editorial judgment will return,” Denton wrote. “Newspaper traditionalists will no doubt see this as vindication.”

Denton’s post should be taken with a grain (if not a hunk) of salt; in the past he has indicated a desire to diminish the company’s focus on traffic without meaningfully changing company incentive structures or HR policies.^xv These changes seem more substantial than those the company has adopted in the past, but Gawker is known for its willingness to experiment; the new approach may turn out to be short-lived, especially if traffic (and advertising revenue) takes a major hit. It is also likely that years of working with metrics-based incentives have conditioned Gawker writers to pursue the kinds of stories that will draw uniques; indeed, Craggs told me this was one of the reasons he was not worried about traffic plummeting as a result of the new policies. Nor has the company completely turned away from traffic: The Big Board still adorns the wall on Gawker’s editorial floor; counts of uniques and page views still appear on posts; and the widely loathed Kinja leaderboard—while no longer displayed in the office—is still publicly available online, as are the traffic numbers for individual Gawker writers. The company also still collects eCPM numbers for writers, though Craggs said he “get[s] hives when people start talking about that stuff here.” Whatever future direction Gawker takes, the fact that Denton invoked Moneyball—arguably the best known parable about the superiority of quantitative performance data to the judgment of entrenched experts—shortly before announcing the replacement of a traffic chart with a “handcrafted blog” that would vindicate “newspaper traditionalists” is telling—both about company’s internal contradictions and challenges it will face moving forward.

These recent developments suggest that Gawker and The Times, once polar opposites in their orientations toward metrics, are moving closer to one another. This is not all that surprising, given that both organizations are trying to work through basically the same dilemma with regard to metrics. Journalists—even nontraditional, analytics-savvy ones—are hesitant to fully equate an article’s audience size with its quality and importance. Yet given that journalism is, by definition, a public-facing profession, indifference to audience interests and behaviors likewise seems inappropriate. It is also, practically speaking, impossible: at a time when home page visits are falling and readers increasingly get their news via social media, no commercial media organization is exempt from playing the traffic game.

What are we to make of this tension? It is not one that can be resolved simply by the creation of better metrics.^xvi While online metrics are relatively new, they raise fundamental questions about journalism that are anything but. What is, ultimately, the purpose of journalism? What should it try to accomplish? And perhaps most importantly: what does it mean to have a media system that is simultaneously a primarily commercial, for-profit industry and one of our most vital democratic institutions? For much of the twentieth century, the high profit margins enjoyed by major media companies provided something of a reprieve from thinking deeply about these questions. Nowadays, the economic reality of the contemporary media field (of which metrics are a particularly vivid manifestation) forces us to confront them head on.

The intention of this report is to play a role in this process, by stepping away from dire (or bullish) predictions about the impact of metrics on journalism to consider how this data is actually produced, interpreted, and used by individuals and organizations. Below are some of the central findings, followed by suggested directions for future research in this line of inquiry.