A major goal of our initial research was to investigate the feasibility of an impact taxonomy—shared terms that could make sense for multiple organizations.
Taxonomies are notoriously difficult to create because real-world data does not necessarily fall into discrete buckets. Take, for instance, Jorge Luis Borges’s Celestial Emporium of Benevolent Knowledge, which divided animals into the following categories:53
Belonging to the emperor
Embalmed
Tame
Suckling pigs
Sirens
Fabulous ones
Stray dogs
Included in the present classification
Frenzied
Innumerable
Drawn with a very fine camelhair brush
Et cetera
Having just broken the water pitcher
That from a long way off look like flies
Although humorous, creating any taxonomy inherits the same absurdity and measure of futility. In the news context, we face constantly shifting content types as well as desired outcomes that differ on a per-project basis. In developing our framework, instead of implementing a strict taxonomy, we intentionally left the question of what constitutes an impactful event up to the discretion of the newsroom.
One strategy that gives more comparative power to qualitative taxonomies involves assigning values to each category—making it more like a quantitative measure. Since it’s one of the simplest examples of impact to visualize, let’s see how this type of impact classification would play out around legislative activity.
For example, let’s say that any article that led to a law creation might earn a rating of 10. An article that gets cited by an influencer (however defined) would earn a 6, and so on. This strategy gets tricky, however, since one must quantify just about everything. How many points separate a bill passing, one being proposed, a watered bill that passed but didn’t fully solve the problem, and a bill that didn’t pass and yet spurred vigorous public debate and changed the narrative around an issue? How would one assign points around different assumptions of causality? If Congress is moved to investigate an industry, what of that can you contribute to any one piece of reporting on the topic that came from any one news organization? Would the value-scale seek to capture the strength of that causal link?
To borrow a term from statistics, we think this type of thinking “over-fits” the model to the data. It might perfectly describe one scenario, but it loses all generalizability to new events or events at other newsrooms. To borrow another image from Borges, “On Exactitude in Science” describes the uselessness of a one-to-one mapping between a subject and a frame used to understand it:
…In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast map was Useless, and not without some Pitilessness was it, that they delivered it up to the Inclemencies of Sun and Winters. In the Deserts of the West, still today, there are Tattered Ruins of that Map, inhabited by Animals and Beggars; in all the Land there is no other Relic of the Disciplines of Geography.54