Two projects aimed at the qualitative aspects of impact are the Center for Investigative Reporting’s (CIR) impact tracker and Chalkbeat’s tool called MORI (Measures of Our Reporting’s Influence).34
Lindsay Green-Barber, a post-doctoral ACLS Public Fellow brought on to serve as the organization’s first media impact analyst, designed CIR’s Impact Tracker as a simple online form that journalists and editors can fill out when they believe an investigation has led to a real-world impact. The form prompts its users to describe what happened, when it happened, optional links or documents associated with the event, and to which CIR story it relates. Users then assign the event to one of 17 carefully curated categories, which represent, in Green-Barber’s experience, the full range of potential outcomes from CIR’s work:
Law change
Government investigation
Reader/viewer/listener contact
Award
Advocacy organization uses report
Screenshot of CIR story in media outlet
Public official refers to report
Institutional action (firing, reorganization, etc.)
Change of policy or regulation
CIR staffer does a public appearance or interview
Localization of story using CIR data
Lawsuit filed
Editorial
Screening
Professional organization cites reporting
Social network share
Other
The process also classifies impact across three levels of the event’s effect:
Macro: Stories that have a concrete effect on things like legislation, changes in staffing involving those in power, or allocation of resources to a subject. Examples might include the prototypical impact event: the passage of a new law addressing an investigation’s findings.
Meso: Stories that influence the general discourse and awareness around a subject. Examples of this could mean increased coverage on a topic at other media outlets or the public organization of a protest.
Micro: Stories that lead to changes in individuals’ behavior or actions. Examples of this include an individual who writes a letter to his or her congressman or stops buying products revealed to be harmful.
Green-Barber has been able to use the data generated to create analyses35reports36on CIR’s impact. To date, other organizations, like The Seattle Times, have already started using the Impact Tracker.
Chalkbeat, an education-focused publication, centers its impact collection around an open source WordPress plugin called MORI that combines article-tagging, event-tracking, and goal measurement.
Before an article is published, staff must categorize the story by type—Analysis, Curation, Enterprise, Quick Hit, etc.—as well as identify the post’s audience—Education Participants, Educational Professionals, General Public, or Influencers and Decision Makers.
If a story is related to a meaningful offline event, staff users can go to that article’s page in their CMS and add a narrative description and an impact tag of either “informed action, the actions that readers take based on our reporting” or “civic deliberation, the conversations readers have based on our reporting.”
Rather than simply reporting raw metrics, MORI works by first requiring editors to predefine goals. In turn, all numbers are displayed in the context of progress rather than performance. This choice was a conscious decision on the part of Chalkbeat’s creators, who were wary of the utility of placing decontextualized metrics in front of journalists or requiring them to track the impact of their stories without being clear as to why they were doing it in the first place.
MORI users can set goals in any of these categories as well: Content Production (e.g., for the number of stories they’ve written across certain focus areas such as teacher evaluations or common core), Content Consumption (e.g., unique visitors, newsletter subscribers), and Engagement (e.g., Facebook fans, offline events hosted by the organization, etc.).
While Chalkbeat had initial concerns about whether its journalists would adopt MORI, its founders were pleasantly surprised by its reception:
Within a week, we were watching conversations unfold in our newsrooms about whether this or that thing constituted an impact. People were eager to tally up the results of our stories. Indeed, reporters and editors quickly began asking how they could sort the data by the stories they had individually produced, a feature we had planned to roll out more slowly.
For more information, their video walkthrough and white paper on the topic are very much worth review.37