Key Findings
Analytics dashboards communicate powerful social and emotional messages. Conversations about metrics, as well as analytics companies’ own marketing materials, tend to focus on the ways in which metrics are a rationalizing force in newsrooms, allowing journalists to access the unvarnished truth about their audience’s behavior and make decisions accordingly. But this view overlooks the ways in which analytics dashboards are designed to do more than simply communicate data. For instance, Chartbeat’s dashboard is designed to convey deference to journalistic judgment and provide opportunities for positivity and celebration. Chartbeat employees also downplay disappointing metrics in their conversations with clients. These actions help to engender positive feelings in clients about Chartbeat’s product.
Chartbeat faces a similar dilemma to one of its news organization clients: its most popular metrics are not always the ones it’s most proud of. Chartbeat has invested considerable time and effort into building metrics that attempt to quantify audience loyalty and engagement—both because the company believes that engagement metrics will incentivize higher-quality content and because these metrics serve an important prestige function. Still, staffers acknowledged that for many clients, the dashboard’s most popular feature is its real-time, speedometer-style dial of concurrent users. Thus, the dashboard’s success is partly due to its combination of prestige metrics—which allow Chartbeat to claim allegiance to journalistic values and position itself against rival analytics companies—and highly addictive metrics from which journalists struggle to tear their eyes away.
Organizational culture heavily shapes the use of metrics. Efforts to examine the ways in which metrics are changing journalism should keep in mind that the reverse can also be true. While Gawker’s norms and practices are very much influenced by the company’s historical emphasis on metrics, The Times’s longstanding organizational culture and structure significantly shapes the role of metrics in the newsroom. For instance, instead of finding their authority diminished by metrics, Times editors controlled the circulation and interpretation of metrics in the newsroom such that they used data to serve preexisting editorial and managerial ends.
Journalism’s multiple goals can make metrics difficult to interpret. Most news organizations—even Gawker Media—are not solely aiming to amass as much traffic as possible. This can lead to confusion about how to read analytics data. There is a general sense that, for instance, a story on drone strikes should not be held to the same traffic expectations as one on Game of Thrones. The two stories are qualitatively different—but metrics, by definition, rank pieces of content according to uniform standards. When interpreting metrics, some journalists therefore found it difficult to determine what constituted a fair comparison or an appropriate standard for a given article.
Metrics can be a source of intense stress for writers and editors, but also one of validation and solace. The relentlessness of metrics, the competitiveness of rankings, and the somewhat unpredictable nature of web traffic caused considerable anxiety among writers and editors at Gawker. However, many staffers reported far more stress about commenters and online trolls than they did about metrics. Indeed, at times, high traffic numbers served as reassurance for staffers that negative comments, vitriolic as they were, represented only a tiny minority of a vast audience. In this way, some staffers used metrics to psychologically buffer against an onslaught of negative feedback.
Traffic pressures can coexist with a strong perception of editorial autonomy. This was the case at Gawker, where writers and editors felt free to do “what they wanted,” even as they also knew they would be evaluated based on metrics.
Metrics fuel internal competition. Because analytics tools rank individual stories and authors, they can have the effect of turning competition further inward. Rather than focusing on surpassing blogs owned by rival media companies (whose traffic numbers they usually did not have access to), editorial staffers at Gawker often expressed a desire to beat the traffic of their sister sites.
A metrics-driven culture can be just as sticky as a legacy one. Gawker’s longtime emphasis on metrics—and especially metrics-based pay incentives—made it difficult for the company to shift its focus to interaction, which was not as easily quantified. This raises questions about the widespread tendency to conceptualize editorial judgment and metrics as two entirely distinct—and often incompatible—entities. At a place like Gawker, where a post’s quality and the size of its audience were often considered to be at least somewhat correlative, staffers’ editorial judgment was inextricably linked to their notions (based on past metrics) about what would hit a nerve with audiences. The management’s directive to shift partially away from that model was thus interpreted as a threat, not only to traffic but also to editorial staffers’ judgment and professional autonomy.