TRK Wage Calculator

Caroline Sinders' Technically Responsible Knowledge is a tool and advocacy initiative spotlighting unjust labor in the machine learning pipeline which includes an open source data labeling and training tool and a wage calculator, and is a part of the Feminist Data Set project. This project was made in collaboration with Rainbow Unicorn (graphic design and branding), Cade Diehm (creative direction), and Ian Ardouin Fumat (developer), with support from the Mozilla Foundation.

The open source tool can be used to label and train image and text datasets for machine learning. It can be used by gig workers themselves, but also artists and researchers who would prefer an open-source alternative to Amazon’s Mechanical turk. The wage calculator isn’t just calculating a price for tasks. Instead, the calculator reveals how under priced data labeling or data training tasks really are. Unlike other tools, this calculator takes into account workdays and a living wage — not just pricing a bunch of tasks in aggregate. The calculator defines a living wage by the standards of Washington State, where Amazon is headquartered. Washington’s minimum wage is around $11 — much more than Amazon Turkers are generally paid. I spoke with more than one dozen gig economy workers from CrowdFlower, Mechanical Turk, and Fiverr, as well as researchers and professors who analyze labor and worker inequality on Mechanical Turk.

Furthermore, after interviewing artists, researchers in labs, startups, and employees of big technology companies, I realized a major takeaway: Even those that wanted to be “ethical” or create equity in their practices and pipelines, still had a hard time understanding pricing in Mechanical Turk. Meaning, even if they wanted they weren’t pricing fairly, because tools like Mechanical Turk don’t show a calculation in relationship to time spent on a task. People were accidentally under-pricing work, even when they were trying to give a good wage.