If you’ve ever taken an elevator, you’re familiar with this experience: you step in, press the floor you want to go to, and, because you’re in a rush, you hit the “close doors” button… and nothing happens. You press it again. You hit it repeatedly. Still nothing seems to happen. While something may be happening behind the scenes, our experience as elevator riders is that the elevator has a will of its own, and it’s not going to go anywhere until it’s good and ready.

This is the experience that came to mind for Mozilla researchers reviewing their findings of their experimental audit of YouTube, powered by crowdsourced data from over 22K participants. Like the elevator buttons, YouTube offers viewers a variety of ways for users to control what types of content they are recommended. On a platform known for recommending harmful content, negative rabbit holes or toxic bubbles, these controls are a critical way for viewers to express their preferences. What Mozilla’s research found, however, was that these controls did little to note viewer preferences and factor them into recommendations.

Turning research into a visual story

As the design team within Mozilla Foundation, we often collaborate with colleagues that come with dense, info-rich findings with the goal of being shared with distinct groups of people. On the one hand, as a movement building organization we want to influence policy makers and companies to make better consumer protection laws and products. On the other hand we want to educate and mobilize the public about issues that affect them and that they should take action on.

Our mission is often to design and execute a strategy where our work can reach both these audiences. In this instance, the research really lended itself to this goal. The YouTube User Control study, led by Becca Ricks and Jesse McCrosky, was actually two studies combined. Becca led the qualitative study, surveying study participants and talking with users who had consented to being interviewed. Jesse, working with Dr. Chico Camargo’s computer science group at the University of Exeter and data scientists from ThoughtWorks in Finland, dug into the data donated by participants through Mozilla’s YouTube add-on, RegretsReporter. Together, the studies painted a rich picture of two sides to one story: People were frustrated with YouTube’s user controls, and that frustration was being caused by a real problem backed up by data.

This insight became the linchpin that guided our design for the study’s landing page. We wanted to meet people where they were and guide them into the findings that proved they weren’t imagining things. YouTube really wasn’t listening to them and no one really knew how the company’s recommendation system worked. As a design team, we wanted to take these findings out of the research report, and share them in a compelling, engaging visual way that would provoke understanding, as well as empower supporters to take action on the issue. And importantly to us - we wanted to elevate the story beyond algorithms and controls, and integrate the key human element - how do we share the stories of frustration, anxiety and fear that people experienced using the platform, and make that paramount?

Our design strategy consisted of several approaches:

Different websites

It was agreed upon early on between the researchers and our team that the findings needed to be split between websites. The first, a landing page, would be an executive summary for anyone who didn’t want or have time to read a multi-page report. The second, a digital publication, would contain the full study findings, meant for people who wanted to delve deep into the details and data.

Layout and content design

Since the landing page was essentially the executive summary and on-ramp to the study findings, it needed to be succinct, visually engaging and informative. We broke down the research into digestible chunks, designed illustrations to complement the text, and simplified data into what it meant and why it was important. Our goal was to tell a story that resonated with people and present easy ways to take action.

To show the human aspect of the research and how it impacts real people, we included a lot of quotes and stories highlighted from the qualitative report. After, the content segued into key data points from the quantitative study illustrated by graphs and an animated timeline using real data. We purposely used animation to show how overwhelming it can be to get the same type of content recommended to you even after expressing disinterest.

Finally, the page concluded with recommendations on how Mozilla was calling on YouTube to improve its product, what policy makers can do, and how people could take action with us.

Visual metaphor

To further flesh out the landing page and other web properties, we wanted to develop illustrations with visual motifs that would tie everything together. Taking cues from our previous YouTube Regrets work to build upon, our team set to work beginning with the broken elevator button analogy. The idea of broken buttons led to broken inputs and controls such as remotes, old movie projectors, and broken wires. All ideas with the important through line that YouTube was projecting a false perception of user control when actually, preferences were falling on deaf ears.

Illustrations for the YouTube User Control Study were created by Mozilla Foundation designer Sabrina Ng.

Additional Challenges

Along with the design challenge of developing a visual story out of the report, and reaching the right audiences, we also faced a tight project deadline. From the outset we had to balance creative ambition (both our own as well as our colleagues) with feasibility. Through good communication, trust and collaboration between Design, Engineering, and Advocacy Research, we were able to move from briefing to concept and final product within a month. In the end, while each team let go of aspects of their original vision, we all ended up happy with the final product.

Impact

This project and its impact was the result of the cumulative effort of multiple teams at Mozilla Foundation. Through everyone’s hard work, the landing page has seen 19,590+ page views and counting, which is a 455% increase from our last report landing page. The campaign page collected over 10,000 petition signatures. The digital publication, hosting the full study findings, has 7,444 page views and counting with 864 PDF downloads. The study has garnered 125 press stories across 26 geographies and 38 Tier 1 publications. We’ve talked to YouTube about the research, and have met with EU lawmakers in Brussels who are looking at the broader platform algorithmic auditing ecosystem.

It’s always an interesting challenge creating something that has to cater to multiple audiences and is so full of rich research. As the design team, it’s really important for us to be the voice of the everyday person and throughout this project we were always thinking about how we can put things in simpler terms or display the right information at the right pace. All of this is important because it makes technical information less intimidating and accessible to more people and everyone deserves to understand how their lives are impacted by tech and by these large companies.