News Organization: The New York Times
Person: Emily Goligoski, user experience research lead.
Goligoski was an arts and culture reporter in San Francisco. She has a bachelor’s degree in journalism, a master’s degree in learning design, and has taught at the Stanford d.school. Before joining The New York Times, she worked for the Mozilla Foundation.
Setting: As user experience lead, Goligoski is based in the Consumer Insight Group at The Times. In the spring of 2015, however, McCallum embedded Goligoski in the newsroom to work directly with desk editors. “It’s really first of all to help them understand who their audiences are,” Goligoski said. “If you’re not doing audience research, you risk just taking shots in the dark.”
It’s worth noting that Goligoski’s work does not represent the entirety of The Times’s consumer insights efforts. Her work is meant to complement other modes of research, such as real-time reader information, data analytics, syndicated tools like comScore, and more. The difference is that her work is directly alongside editorial.
The Process: Earlier in 2015, two editors approached Goligoski with a basic question: What can we learn about what people need in breaking news moments? In order to define the question, Goligoski first scheduled a series of informal lunches with people across the newsroom involved with breaking news. Each lunch was one-hour long, attended by ten to fifteen people. Goligoski acted as facilitator. “We were trying to understand what constitutes a breaking news win,” she said. “What’s an example of us or a competitor really meeting someone’s information needs in those moments?” Goligoski framed her project with three questions: What do readers seek most in breaking news moments? What role do devices play in shaping their news gathering decisions? How important is social media as a news discovery mechanism?33 After gathering information inside the newsroom, Goligoski did the same outside it. She worked with other consumer insights staff to find twenty New York Times users representing different levels of engagement, from the casual user who just stumbles across the news, to regular readers. Some of the recruits spent hours a day on NYTimes.com; others hadn’t read The Times since the Boston Marathon bombing. Goligoski’s team also selected across socioeconomic, racial, and professional lines. (Goligoski sometimes uses third-party audience research firms to help with selection, although in this case she did not. The team recruited twenty people, and ended up with fifteen, due to dropout.)
Goligoski did not ask recruits to participate in a focus group; she argues that these “encourage groupthink” and give undue prominence to the most outspoken. Goligoski is not alone in her distaste for focus groups. Steve Jobs famously said, “It’s really hard to design products by focus groups. A lot of times, people don’t know what they want until you show it to them.”34ii Goligoski advocates one-on-one work and more time spent understanding the context of a person’s life. For this project, she gave each of her recruits access to an online dashboard to record their media interactions throughout the day, as well as how they felt about those interactions and whether they shared what they consumed. She also encouraged recruits to take snapshots of what they were viewing whenever possible and had them keep diaries of their experiences over the week. The dashboard and diary are a way to see what people actually do, rather than just what they wish they were doing. What people report on surveys is notoriously different than actual behavior. “We’re trying to get towards what’s really happening and not a person’s idealized self,” Goligoski said. After seven days of this, Goligoski set up interviews with each person, in their own houses when she could arrange it. As much as possible, she seeks to interview people in their real-life environments; she can learn more seeing people in their living rooms than in a Times conference room, she said. This is actually a key principle of human-centered design, which advocates that users be understood as full humans, not just consumers. It addresses the ever-present need to consider the larger system in which a person exists and will use your product.iii Goligoski’s interviews are at least sixty minutes each. “I’m not listening for sound bites,” Goligoski said, but for meaning and context. After the interviews, Goligoski and her team spent another week analyzing and synthesizing their findings. “Basically, we lock ourselves in a room and Post-it like crazy,” she said. “What you’re looking for are patterns and surprises.”iv Like Majeed and her team, Goligoski relies heavily on visual tools in her work. The team put up portraits of the people with whom they’d spoken, alongside the images people took of themselves and the media they’d consumed over the week. Goligoski said this helps her identify with her subjects and gain a richer understanding of them as individuals. On a more prosaic level, visuals can be extremely useful tools of communication in team environments.
After completing her research, Goligoski condensed the work into a thirty-minute presentation that she showed to different desk editors. So far, she has presented her findings to design teams, product teams, and the international news teams, among others. Mostly, she said, news of the research has spread by word of mouth, and she presents to people who are curious. The Times does not mandate that everyone see her presentation. Some of what Goligoski found in the breaking news project was in line with what the data analytics team had already suggested: People are on their computers during the day, their tablets on weekends, radio during commutes, cable TV at night, and mobile all the time. There were also surprises, however. For example, if people didn’t catch a story when it broke, they didn’t care about new developments until it took up so much space in their social media channels that they felt a sense of personal responsibility to catch up. Then, when that moment came, a surprising number turned to Wikipedia. Another finding that emerged was that people were annoyed by alerts. “The volume is too high, and lot of them are irrelevant to what people want,” Goligoski said. Was the exercise useful? Goligoski’s presentation is still making the rounds at The Times, so it’s hard to give a firm answer on this. But it is considered a success, and the research is being used to further understand reader needs regarding SMS alerts, mobile live-blogging, and news personalization. I asked Goligoski if this sort of design work was essentially a business function, or if it actually made for better journalism. “To be on the editorial side and not think about business considerations, that feels like a dated approach to me,” Goligoski said. “I think design and design thinking are tools we have to best meet readers where they’re at. And given how competitive things are right now, why wouldn’t you employ that?” This comment may raise the hackles of some journalists. The breakdown of the so-called “Chinese wall” between editorial and business is a major topic of conversation in future of journalism circles. Since the 1920s, when the industry began to develop a code of ethics—and had successfully moved away from a model of political allegiance to one of advertiser support—a complete separation of business and editorial was seen as a priority.35 But since the collapse of the print business model, many have begun to wonder if this insistence is counterproductive. Bob Steele, the Nelson Poynter Scholar for Journalism Values at the Poynter Institute, suggested that the division between editorial and business should be a “picket fence” rather than a wall. “You can talk over the picket fence. If there’s a gate, you can go back and forth,” he said.36
The question, of course, is can you turn a wall into a picket fence without sacrificing, consciously or unconsciously, editorial staff’s capacity to remain uninfluenced by business interests. This is not to imply that reporters shouldn’t care if their news organizations survive—presumably, they do—but rather that they should not be making decisions about what to cover, or not cover, based on what advertisers want.
When asked if she worried that ideas generated in consumer insights might unduly affect the work of editorial, Goligoski said:
It’s delicate, because we should never get in the way of editorial judgment. I will make recommendations around, say, “opportunities on mobile alerts to do X, Y or Z.” But I really try to avoid being prescriptive. My worst nightmare ever would be that we didn’t cover something editorially valuable because we didn’t think it would be financially valuable.
Takeaways:
Bring in user research early and often. Goligoski likens her work to that done in R&D labs. “It’s finding out if there’s a hunger,” she said. This goes back to human-centered design’s insistence on creating around people’s needs.
Don’t rely on focus groups. With focus groups, there’s always the danger that the loudest person in the room will end up dominating. Because of this, Goligoski prefers one-on-one interviews, asking people to keep diaries, and usability testing. Besides, as Steve Jobs said, people often don’t know what they want in any way they can articulate—one-on-one time can help you tease out people’s actual needs.
Focus on the individual. Seek to understand how what you are providing might fit into a particular person’s life. While members of Goligoski’s group were selected across socioeconomic lines, the focus was on them as whole people. According to Goligoski and other designers, viewing people as individuals rather than generalities leads to better design.
Qualitative research matters. Design research fills important gaps in knowledge that quantitative research may not answer fully.