While philanthropists began adopting the mantle of measurement, they also became increasingly interested in the importance of media for communicating and amplifying the message of their missions. Here we begin to see how the M&E framework that aid and development communities established connects directly with the present topic of measuring media impact.
With the release of high-profile, social-issue documentaries such as Bowling for Columbine (2002) and An Inconvenient Truth (2006), the power of mass media to steer public debate around a topic became readily apparent. In the years following, prominent funders like the Bill & Melinda Gates Foundation, the Ford Foundation, and Open Society Foundations latched onto documentary films as potential means of raising awareness and prompting action on widespread societal problems. From educational reform (Waiting For Superman) to schoolyard bullying (Bully) to fracking (Gasland), many of the most resonant documentary films of the past decade have received foundation support for their production, distribution, and/or associated outreach programs. Whether for purposes altruistic, financial, or both, it is now standard practice for documentary filmmakers to attach social-issue campaigns to their creative works.
In turn, the foundations that supported these films—influenced by their concurrent involvement in aid and development interventions—began requiring filmmakers to provide detailed reporting on the impact of their work. In practice, these reports initially relayed traditional metrics like viewership, ratings, and box-office returns. Yet, over time, they increasingly adopted more sophisticated social science methodologies, employing pre- and post-surveys, frame analysis, and monitoring of mass and social media mentions. BritDoc,14to exclusively support social-issue documentaries, lists over 30 of these impact reports published since 2008 in its Impact Field Guide & Toolkit.15 Yet a fundamental difference exists between assessing the impact of an aid intervention versus a documentary film. If your goal is to eradicate polio—as is one mission of the Gates Foundation—it is (relatively) easy to measure the effectiveness of your intervention; simply counting the number of polio cases over time provides a reliable metric of success. If you’re concerned about the influence of confounding factors, like simultaneous development initiatives in the same region, you might design a randomized control trial to test the varying effectiveness of different vaccines, treatments, or educational campaigns. Documentary filmmakers, however, seek more abstract goals like raising awareness, shifting societal norms, or advancing the art form. While academics have attempted to design randomized studies to isolate the effect the mass media has in driving such outcomes, these approaches are limited to highly specific interventions and do not address the need for making comparisons across a variety of contexts.16challenges as it increasingly moves toward business models driven by institutional and philanthropic support.