Edited image of Pope Francis in a Balenciaga puffer jacket in a black and yellow overlay
Image courtesy of Futurism, as seen in AI Incident Database

This is a profile of Mozilla Technology Fund awardee Responsible AI Collaborative.

A Pope, a Balenciaga puffer jacket, and an AI image-generating tool were the ingredients chosen by one Pablo Xavier to create the perfect viral joke. What didn’t occur to Xavier was the extent to which people would think it was real.

The image had duped thousands of people, making it one of the early cases of an AI-generated deep fake going viral. As the image and the subsequent stories flooded news feeds, a distributed team of staff and volunteers at the Responsible AI Collaborative debated whether satire for comedic expression passed as AI harm.

“Detecting AI harms has never been so onerous. The volume of incidents that we're getting for generative AI and for deep fakes is so large that we're needing to quickly advance our database and editing processes,” says Sean McGregor, founder of the Responsible AI Collaborative, and a member of the Mozilla Technology Fund’s 2023 cohort.

Whether it is a bizarre malfunction of a rogue robot breaking the finger of a seven-year-old chess player or a much more devastating occurrence of an AI chatbot that encouraged the suicide of a Belgian man, “The scale and impacts of unintended consequences of these systems can be very profound,” McGregor explains.

Detecting AI harms has never been so onerous... The scale and impacts of unintended consequences of these systems can be very profound.

Sean McGregor, Responsible AI Collaborative

The Responsible AI Collaborative is the team behind the open-source project AI Incident Database, which records and classifies AI harm events known as "incidents". Driven by the objective of mitigating societal harms, the database documents and shares incidents with the public and auditing practitioners. It elucidates speculation of “what could go wrong” and instead highlights “what is going wrong” with AI systems.

The centerpiece in creating an AI risk checklist is to propel the process of AI auditing and risk analysis a step further; “From identifying systems that are most capable of significant harms to using these incidents to inform the design, development, and deployment of AI,” McGregor adds.

Reported incidents are grouped per the Center of Security and Emerging Technology taxonomies, detailing the nature of the harms including the varying scale in which AI harms manifest.

Devastatingly, AI systems are still often racist. Racial bias encoded into AI models is still the most prevalent source of harm, accounting for at least 29% of classified incidents in the database, followed by sex which featured about 18% of the classified incidents. “Establishing the rate at which different incidents are occurring is immensely important in enabling us to identify what industrial and policy responses may be required,” McGregor explains.

Although the tech policy landscape moves slower than the rate that these technologies are being deployed, McGregor has witnessed how recording these incidents is indeed motivating companies to build better — but still stresses that effective regulation is necessary.

The Mozilla Technology Fund (MTF) supports open source technologists whose work furthers promising approaches to solving pressing internet health issues. The 2023 MTF cohort will focus on an emerging, under-resourced area of tech with a real opportunity for impact: auditing tools for AI systems.