Does This Button Work? Investigating YouTube's ineffective user controls
Powered by 22,722 volunteers, Mozilla scrutinized YouTube to determine how much control people actually have over the platform’s recommendation algorithm. This is what we learned.
Overview
In this report, we present a large-scale audit Mozilla conducted on YouTube’s user controls: the tools and features people use to manage their video recommendations on YouTube. To evaluate the effectiveness of these controls for real users of the platform, we collected data from 22,722 volunteers about their interactions with YouTube. We also surveyed 2,757 participants to better understand their feelings of control on the platform. Building on our previous investigation into YouTube, this study is likely the largest experimental audit of YouTube by independent researchers, powered by Mozilla’s RegretsReporter.
Main Findings
People don’t feel they have much control over their YouTube recommendations — and our study demonstrates they actually don’t. We determined that:
1. People feel that using YouTube’s user controls does not change their recommendations at all. We learned that many people take a trial-and-error approach to controlling their recommendations, with limited success.
2. YouTube’s user control mechanisms are inadequate for preventing unwanted recommendations. We found that YouTube’s user controls influence what is recommended, but this effect is meager and most unwanted videos still slip through.
Recommendations
1. The controls should be easy to understand and access. People should be provided with clear information about the steps they can take to influence their recommendations, and should be empowered to use those tools.
2. YouTube should design its feedback tools in a way that puts people in the driver’s seat. Feedback tools should enable people to proactively shape their experience, with user feedback given more weight in determining what videos are recommended.
3. YouTube should enhance its data access tools. YouTube should provide researchers with access to better tools that allow them to assess the signals that impact YouTube's algorithm.
4. Policymakers should protect public interest researchers. Policymakers should pass and/or clarify laws that provide legal protections for public interest research.
