Case Studies and Methods

By looking closely at one analytics company and two media organizations that use its tool, this research is able to follow both how an analytics dashboard moves from the company that produces it to those that use it, and also how very different types of editorial groups integrate the same tool into their work.

Chartbeat is an important and ideal analytics company to study for several reasons. It has tremendous reach—its clients include 80 percent of the most-trafficked publishers in the United States, as well as media outlets in 35 other countries—yet is small enough that it is possible for an ethnographer to get a feel for the company as a whole. It was also one of the first analytics companies to make a dashboard specifically designed for use by journalists, rather than by advertising sales departments. During my time at Chartbeat, the company was building, marketing, and launching a brand new version of Chartbeat Publishing, its flagship editorial product; I was able to witness much of this process take place. From August 2013-January 2014, I spent time as a “fly on the wall” in Chartbeat’s offices, observing internal meetings, user experience research, client trainings, and the rhythms of daily office life. I also conducted 22 interviews and in-depth conversations with 16 employees^v who worked on a mix of teams across the company—from sales and marketing, to engineering, to product development and tech support.

Gawker Media is a highly popular and visible network of blogs that covers topics ranging from gaming to sports to women’s issues. Gawker is widely known as a metrics-driven organization. In the early days of the company, owner Nick Denton developed a reputation for paying writers partly based on the page views their posts generated. The company also devised the Big Board (subsequently built by Chartbeat), a constantly updating screen displaying the top stories by traffic across all Gawker Media sites. During the time of my fieldwork, Gawker gave bonuses based on its sites’ unique visitor counts and had started publicly ranking individual writers by traffic. Gawker is a longtime client of Chartbeat and the two companies have a close relationship; while most of Chartbeat’s clients require that their data be kept confidential, Gawker allows Chartbeat to make its data public for training and other purposes. I conducted 30 interviews with writers and editors from six of Gawker’s eight core titles, as well as with a small number of editorial and business development executives. From February-July 2014, I also spent a total of five days observing the online group chats (conducted on the collaboration software Campfire) of two of the company’s core sites and attended occasional staff meetings. In addition, I analyzed the company’s internal memos (some of which staffers provided to me, others of which were leaked online).

If Gawker is known for being metrics-driven in all decision-making, The New York Times, at least at first glance, seems to have the opposite relationship to metrics. For years, representatives of The Times newsroom were publicly dismissive, even scornful, of the idea of using metrics to inform editorial processes. In the face of the paper’s ongoing financial troubles⁶The Times’s culture around metrics may be changing. Though The Times is a longtime Chartbeat client, it only recently upgraded from the most basic iteration of the dashboard to the more sophisticated tool Gawker and most other large newsrooms employ. The Times’s complex and fraught relationship to analytics makes the organization an excellent case study through which to examine the interactions between metrics and legacy media practices and values. Between 2011 and 2015, I conducted 23 semi-structured interviews with 20 reporters, columnists, editors, bloggers, and analysts at The Times.