29 January 2020: You're doing it wrong

The coverage: "Auditing Radicalization Pathways on YouTube" by the Anti-Defamation League, and "YouTube’s algorithm seems to be funneling people to alt-right videos" by MIT Technology Review

The excuse: New research by the Anti-Defamation League revealed that "YouTube is a pipeline for extremism and hate," in the words of MIT Technology Review, which covered the study.

YouTube's response to the study and story? “Over the past few years ... We changed our search and discovery algorithms to ensure more authoritative content is surfaced and labeled prominently in search results and recommendations and begun reducing recommendations of borderline content and videos that could misinform users in harmful ways."

YouTube continued: "We strongly disagree with the methodology, data and, most importantly, the conclusions made in this new research."

31 October 2019: You’re generalizing

The coverage: “YouTube’s algorithm apparently helped a Chinese propaganda video on Hong Kong go viral” -- Quartz

The excuse: In a Twitter thread an official YouTube account responded to the Quartz investigative piece writing that: “1/ While AlgoTransparency, created by former Google ads engineer Guillaume Chaslot, raises interesting questions, the findings cannot be used to generalize about recommendations on YouTube for a few reasons. 2/ The majority of YouTube recommendations are personalized. You are likely to get recommendations for content similar to what you’re watching, or that other people watching that content have also enjoyed. The recommendations on AlgoTransparency are based on incognito mode. 3/ AlgoTransparency assumes all recommendations are equally viewed. Every recommended video gets counted, even those that are farther down in the ranking. It simulates a user who would simultaneously select ALL recommendations (and no user can do this). 4/ AlgoTransparency also relies on a set of channels that don't represent all of YouTube. In fact, if they would use popular content as an indicator, the conclusions would be different. 5/ As with most assertions made by AlgoTransparency, we've been unable to reproduce the results here. 6/ For example, a Vox video about the Hong Kong protests has 10x more views than the video mentioned in this story, and general queries and recommendations about the Hong Kong protests are showing results from authoritative sources.” Note, Guillaume Chaslot is currently a Mozilla Fellow.

27 September 2019: You’re doing it wrong

The coverage: “YouTube is experimenting with ways to make its algorithm even more addictive” -- MIT Technology Review

The excuse: In a Twitter thread an official YouTube account responded that “This is not accurate. We looked into the claims brought forward in this article and removing position bias will not lead to filter bubbles nor lead users to extreme content. On the contrary, we expect this change to decrease filter bubbles and bring up more diverse content. This paper primarily explored objectives beyond engagement like satisfaction and diversity. So, it actually does the opposite of what the experts in this article suggest that it does. In fact, we have been very public about the fact that our goal is long term satisfaction rather than engagement and that we’re actively reducing the spread of borderline content on our site - whether it’s engaging or not.”

Researchers quickly jumped in to criticise YouTube’s response, asking them to provide data that proves their claims made in the previous tweets. Critical users also chimed in with story after story of how they had experienced this radicalisation/recommending extreme content first hand and suggesting that they did not believe YouTube’s claims and were not going to ‘take their word for it’.

22 August 2019: We’re working on it 😉 Also, you’re doing it wrong

The coverage: “Auditing Radicalization Pathways on YouTube” -- École polytechnique fédérale de Lausanne (Switzerland) and Universidade Federal de Minas Gerais (Brazil)

The excuse: In a statement sent to Rolling Stone, Farshad Shadloo, a YouTube spokesperson, says: “Over the past few years, we’ve invested heavily in the policies, resources and products needed to protect the YouTube community. We changed our search and discovery algorithms to ensure more authoritative content is surfaced and labeled prominently in search results and recommendations and begun reducing recommendations of borderline content and videos that could misinform users in harmful ways. Thanks to this change, the number of views this type of content gets from recommendations has dropped by over 50% in the U.S. While we welcome external research, this study doesn’t reflect changes as a result of our hate speech policy and recommendations updates and we strongly disagree with the methodology, data and, most importantly, the conclusions made in this new research.”

A statement given to The Verge reads: “While we welcome external research, this study doesn’t reflect changes as a result of our hate speech policy and recommendations updates and we strongly disagree with the methodology, data and, most importantly, the conclusions made in this new research.”

VICE reported that “In a statement a YouTube spokesperson said they're constantly working to better their ‘search and discovery algorithms’ and ‘strongly disagree with the methodology, data and, most importantly, the conclusions made in this new research.’ The spokesperson, as well as the information provided on background, did not address the majority of the study and instead focused solely on the section that touched upon channel recommendations.”

11 August 2019: Our data disproves this. But, you can’t see that data

The coverage: “How YouTube Radicalized Brazil” -- New York Times

The excuse: Regarding efforts to study YouTube’s influence in causing a rise of the far-right in Brazil, YouTube challenged the researchers’ methodology and said its internal data contradicted their findings. The company declined the Times’ requests for that data, as well as requests for certain statistics that would reveal whether or not the researchers’ findings were accurate.

Farshad Shadloo, a spokesman, said that YouTube has “invested heavily in the policies, resources and products” to reduce the spread of harmful misinformation, adding, “we’ve seen that authoritative content is thriving in Brazil and is some of the most recommended content on the site.”

With regards to claims of medical misinformation about the Zika virus spreading, “A spokesman for YouTube confirmed the Times’ findings, calling them unintended, and said the company would change how its search tool surfaced videos related to Zika.”

6 August 2019: Our data disproves this. But, you can’t see that data

The coverage: “The Making of a YouTube Radical” -- New York Times

The excuse: In interviews with the NYT, YouTube officials denied that the recommendation algorithm steered users to more extreme content. The company’s internal testing, they said, has found just the opposite — that users who watch one extreme video are, on average, recommended videos that reflect more moderate viewpoints. The officials declined to share this data, or give any specific examples of users who were shown more moderate videos after watching more extreme videos. The officials stressed, however, that YouTube realized it had a responsibility to combat misinformation and extreme content.

“While we’ve made good progress, our work here is not done, and we will continue making more improvements this year,” a YouTube spokesman, Farshad Shadloo, said in a statement.

25 July 2019: We’re working on it 😉

The coverage: “Most YouTube climate change videos 'oppose the consensus view'” -- The Guardian

The excuse: According to the Guardian, a YouTube spokesperson said: “YouTube is a platform for free speech where anyone can choose to post videos, as long as they follow our community guidelines. Over the last year we’ve worked to better surface credible news sources across our site for people searching for news-related topics, begun reducing recommendations of borderline content and videos that could misinform users in harmful ways, and introduced information panels to help give users more sources where they can fact-check information for themselves.”

30 June 2019: We changed the algorithm. Trust us

The coverage: “Google directing people to extreme content and conspiracy theories” -- Sky News

The excuse: YouTube strongly denies these claims. Its spokesperson told Sky News it “no longer used watch time as a metric, that it was not in its ethical or financial interest to recommend harmful content, and that it had changed its secretive algorithm to reduce the impact of what it described as "borderline content", content which did not break its rules, but "comes right up to the line".

25 June 2019: We changed the algorithm. Trust us

The coverage: “They turn to Facebook and YouTube to find a cure for cancer — and get sucked into a world of bogus medicine” -- Washington Post

The excuse: WaPo asked YouTube about the changes to videos previously reported in the story, which occurred just before they reached out to the company for comment. They were told that YouTube has started to treat search results for different types of topics differently: When its algorithms decide a search query is related to news or information-gathering on a topic like cancer, they will attempt to populate the results with more authoritative sources. The company said it is working with experts on certain health-related topics to improve results.

A spokesman told the WSJ that it was removing advertising on "bogus cancer treatment channels." According to the investigation, ads for legitimate pharmaceutical companies appeared on channels that touted, for example, ways to treat cancer with diet. Like Facebook, YouTube is limiting the reach of these videos – recommending them to other users less often – rather than removing them or banning them outright. (no data provided on results of this)

3 June 2019: We threw some machine learning at it

The coverage: “On YouTube’s Digital Playground, an Open Gate for Pedophiles” -- New York Times

The excuse: YouTube took several actions following the publication of this study, detailed in this blog post. Namely, they 1.) Restricted live features 2.) Disabled comments on videos featuring minors and 3.) Reduced recommendations—with no data provided for this last point. They outright refused to stop recommending videos featuring minors entirely because they said it would hurt their content creators. Gizmodo reported that YouTube claimed that they “improved their machine learning to better identify videos that might put minors at risk.” In nearly every request for comment by journalists who covered this, YouTube referred them to the blog post about how they’re doing more.

According to the researchers who carried out this study, YouTube shut down the “related channels” feature during the time when the NYT asked for comments; according to YouTube it was because they “weren’t frequently used.” This was the main feature that the researchers used for their research and which they now cannot continue.

From the NYT: “‘It’s not clear to us that necessarily our recommendation engine takes you in one direction or another,’ said Ms. O’Connor, the [YouTube] product director. Still, she said, ‘when it comes to kids, we just want to take a much more conservative stance for what we recommend.’”

20 February 2019: We changed the algorithm. Trust us

The coverage: “YouTube Continues To Promote Anti-Vax Videos As Facebook Prepares To Fight Medical Misinformation” -- BuzzFeed

The excuse: When BuzzFeed reported on YouTube’s recommendation engine suggesting anti-vaccine content, YouTube said they were working on the problem -- but didn’t share hard data. YouTube wrote in an email: “Over the last year we’ve worked to better surface credible news sources across our site for people searching for news-related topics, begun reducing recommendations of borderline content and videos that could misinform users in harmful ways, and introduced information panels to help give users more sources where they can fact check information for themselves. Like many algorithmic changes, these efforts will be gradual and will get more and more accurate over time.”

24 January 2019: We’re going to dodge the question

The coverage: “We Followed YouTube’s Recommendation Algorithm Down The Rabbit Hole” -- BuzzFeed

The excuse: Despite BuzzFeed’s in-depth reporting, YouTube provided a brief and largely irrelevant reply: “Over the last year we’ve worked to better surface news sources across our site for people searching for news-related topics,” a spokesperson told BuzzFeed over email. “We’ve changed our search and discovery algorithms to surface and recommend authoritative content and introduced information panels to help give users more sources where they can fact check information for themselves.”

18 September 2018: We’re ‘open.’ Now leave us alone

The coverage: “YouTube's 'alternative influence network' breeds rightwing radicalisation, report finds” -- The Guardian

The excuse: In response to Becca Lewis’ research, YouTube told the Guardian that “YouTube is an open platform where anyone can choose to post videos to a global audience, subject to our community guidelines, which we enforce rigorously.” The spokeswoman added that the company has tightened the rules for which channels have access to monetisation features and deployed machine learning technology to identify hate speech in comment features.

7 February 2018: We’re working on it 😉

The coverage: “How YouTube Drives People to the Internet’s Darkest Corners” - Wall Street Journal

The excuse: After the WSJ provided examples of how the site still promotes deceptive and divisive videos, YouTube executives said the recommendations were a problem. “We recognize that this is our responsibility,” said YouTube’s product-management chief for recommendations, Johanna Wright, “and we have more to do.”

2 February 2018: You’re doing it wrong

The coverage: “'Fiction is outperforming reality': how YouTube's algorithm distorts truth” & related methodology from Guillaume Chaslot & Graphika “How an ex-YouTube insider investigated its secret algorithm” -- The Guardian

The excuse: First statement: “We have a great deal of respect for the Guardian as a news outlet and institution. We strongly disagree, however, with the methodology, data and, most importantly, the conclusions made in their research. The sample of 8,000 videos they evaluated does not paint an accurate picture of what videos were recommended on YouTube over a year ago in the run-up to the US presidential election. Our search and recommendation systems reflect what people search for, the number of videos available, and the videos people choose to watch on YouTube. That’s not a bias towards any particular candidate; that is a reflection of viewer interest. Our only conclusion is that the Guardian is attempting to shoehorn research, data, and their incorrect conclusions into a common narrative about the role of technology in last year’s election. The reality of how our systems work, however, simply doesn’t support that premise.”

After it emerged that the Senate intelligence committee wrote to Google demanding to know what the company was doing to prevent a “malign incursion” of YouTube’s recommendation algorithm – which the top-ranking Democrat on the committee had warned was “particularly susceptible to foreign influence,” YouTube asked The Guardian to update its statement: “Throughout 2017 our teams worked to improve how YouTube handles queries and recommendations related to news. We made algorithmic changes to better surface clearly-labeled authoritative news sources in search results, particularly around breaking news events. We created a ‘Breaking News’ shelf on the YouTube homepage that serves up content from reliable news sources. When people enter news-related search queries, we prominently display a ‘Top News’ shelf in their search results with relevant YouTube content from authoritative news sources. We also take a tough stance on videos that do not clearly violate our policies but contain inflammatory religious or supremacist content. These videos are placed behind a warning interstitial, are not monetized, recommended or eligible for comments or user endorsements. We appreciate the Guardian’s work to shine a spotlight on this challenging issue. We know there is more to do here and we’re looking forward to making more announcements in the months ahead.”