Effective impact measurement must be tied to an organization’s goals. No amount of technology can help an organization measure what it hasn’t defined as important. Should a newsroom’s reporting seek to change the narrative around an issue? Does it want to reach certain stakeholders or affect lasting reform? Only after an organization has understood what it wants to achieve can quantitative and qualitative tools assess how close the organization is to that goal.
Both quantitative and qualitative metrics have a place in impact measurement. While quantitative metrics are often vilified as leading journalism astray from its true purpose, the researchers found they do help tell the story of a newsroom’s performance. Although this project began with an interest in giving more visibility to qualitative measurements, its founders repeatedly heard reports from newsrooms that quantitative measurements play an important role for organizations wanting to tell a long-term story of audience growth.
Newsrooms should better tag their articles. Newsrooms that want to properly understand their own performance over time should put more care into tagging and cataloging their stories. These practices can give an organization a better understanding of its own operations and how much space it devotes to each subject. Tags also offer staff the ability to perform myriad analyses comparing stories and packages. Without differentiating and labeling content, it is difficult to understand patterns in traffic or impact.
Newsrooms have metrics, but they also still have many questions—particularly about audience. As one newsroom put it, “Google Analytics feels both too complicated and not powerful enough for the questions we want to answer about readers.” Many existing metrics aren’t designed to help analyze metrics from readers’ perspectives—as in, what did they think about the story? Did they leave after the fifth graf because they understood the newsy part of it and didn’t need anymore, or was the site design wrong or the prose too dense? Nor do common tools provide enough insight into that relationship between the news organization and its audience.
Custom analytics solutions have recently become more feasible. With the continuing maturation of open source analytics pipelines, it is now possible for news organizations to own their entire analytics stack and not rely on third-party vendors for the data-collection portion of their metrics. In other words, the next few years could see newsrooms access much more diverse offerings, providing faster analysis and greater detail more relevant to journalism. That being said, these pipelines are largely for data collection, so most newsrooms will need to design and implement their own custom interfaces to interpret this data for the average reporter and editor.