Sheltering Clients from Bad News and Providing Opportunities to Celebrate Good News

As Chartbeat and other analytics tools become more common and influential in newsrooms, journalists’ sense of dignity, pride, and self-worth are increasingly tied up in their traffic numbers. Metrics have earned a reputation as ego-busters, as journalists discover that their readership is considerably smaller and less engaged than they imagined. In meetings and calls I observed, clients called Chartbeat data points on their dashboard “sad” and “super pride-crushing.” Clients’ feelings about their data can become inseparable from their feelings about Chartbeat. An employee who works with smaller, lower-trafficked publishers said the experience of looking at the dashboard can be “harsh,” adding that some elect to stop using Chartbeat before finishing their free trial.

A few Chartbeat employees saw themselves as providers of tough love. One said he tells clients that “facing the low numbers is the first step towards boosting them.” However, in conversations with clients, Chartbeat generally tended to shy away from the role of a purveyor of uncomfortable truths, in large part because the company didn’t want clients to develop negative associations with the product. In meetings and trainings, employees often took a positive tone, even when discussing disappointing or unimpressive numbers. During one training, an employee displayed a graph showing that very few visitors went to a second page on the client’s site after the one on which they’d landed. The employee said that, despite appearances, this wasn’t actually bad, because the client’s landing page had so much to do and look at. “People stay on your landing page,” he encouraged. There were several other instances in which employees were quick to reassure clients that a particular metric wasn’t as bad as it seemed or to redirect clients’ attention from an underwhelming metric to a stronger one.

In addition to trying to take the edge off disappointing metrics, Chartbeat also actively facilitates opportunities for optimism—both in interactions with clients and in the dashboard itself. When describing the considerations that went into the new dashboard, one employee said:

You know, celebrate was one thing. That needs a place in [the product].

CP: When something goes well—

Yeah, yeah! Should we, in our product, be helping them celebrate more, or telling them when to?

Nowhere is the importance of celebration clearer than in the case of the “broken dial.” Chartbeat is priced so that each client has a cap of concurrent visitors that it has paid for; if the number of visitors on its site exceeds that number, the dial “breaks”—that is, the client will see that the dial and the number of concurrents have hit the cap, but the dashboard will not show by how much the cap has been exceeded. The broken dial sometimes results in an upsell, as clients who repeatedly hit their cap can pay to raise it. But even when the broken dial doesn’t directly lead to greater revenue, it has a valuable emotional impact on clients. When the cap is exceeded, one employee explained, “the dial looks incorrect …The product is broken, but in a fun way. If you didn’t have that sort of excitement, it wouldn’t work.”

One client, during a product testing session, articulated just how powerful such excitement can be and how central it is to Chartbeat’s appeal. She said that her news organization had briefly considered discontinuing its account with Chartbeat in favor of a rival analytics service, but within a few days had “crawled [their] way back” to Chartbeat. The Chartbeat employee conducting the session asked, “what is it, do you think, that was the most compelling aspect of Chartbeat?” The client answered:

First of all, if you’re into traffic as most sites are, seeing that big number [recites number of concurrents on her site—750], that’s a really good number. And we’re capped at 2,000 concurrent users, so if we always think—if we’re at 1,999, we always imagine we’re at 2,450 or 5,150, but we probably [just] can’t see it ’cause we’re capped. So we always have that kind of illusion, like that optimism going on.^vi

For this client, the broken dial did exactly the opposite of what we typically think of as the role of metrics: Rather than providing unvarnished information about her audience, it allowed her and her team to imagine an audience size that was more than twice the size of their cap, regardless of whether or not the imagined number had any basis in reality. In this instance, Chartbeat’s role as a facilitator of optimism and celebration has taken precedence over its self-described mission to provide readers with an accurate picture of their traffic. Importantly, this is not an oversight on the company’s part but a strategic choice that renders the product more appealing. As an employee put it:

When [a client] does hit a home run, and you see the Chartbeat dial go up, and you see the red line go up, then that euphoria—Chartbeat gets to be part of that experience. So as long as we’re in the room, like high-fiving you, then we get a lot of positive association with that moment.

These findings all contribute to a larger point: While we tend to think of analytics dashboards as a rationalizing force in newsrooms, a tool that applies rigorous, objective quantitative methods in place of the unreliable, unscientific guesswork of the past, it turns out that Chartbeat’s dashboard is designed to play a social and emotional role in newsrooms just as much as a rational one. Everything—from the types of data the dashboard provides, to the features it doesn’t include, to the way in which data is presented, to the interactions between staff and clients—is profoundly influenced by Chartbeat’s need to earn journalists’ trust, prevent them from becoming demoralized, and provide them with an appealing product. Indeed, the company’s very survival depends on its ability to be something editors actually want to use. This is not to say that Chartbeat’s methods for measuring audience behavior are in some way flawed or not objective, or that the company intentionally or cynically manipulates its clients. To the contrary, Chartbeat’s methods are widely considered to be quite rigorous, and most of the staff members with whom I spoke seemed wholly genuine in their belief that Chartbeat’s metrics support and reward high-quality journalism. Rather, the goal here is to illustrate that Chartbeat’s metrics—indeed, all metrics!—are the outcome of human choices and that these choices are in turn influenced by a range of organizational, economic, and social factors.