Docs show how freelancers test AI by creating 'harmful' prompts
Apr 04, 2025 - businessinsider.com
Leaked documents reveal that freelancers are paid to stress-test AI models with harmful prompts to improve their safety, a process known as AI "red teaming." Companies like Outlier and Scale AI employ freelancers to create prompts on sensitive topics such as suicide, abuse, and terrorism. These workers are instructed to be creative but must avoid certain lines, particularly involving child sexual abuse material. Despite the disturbing nature of the work, freelancers are offered wellness support, though they cannot opt out of specific risk categories once assigned. Scale AI faces scrutiny, including lawsuits and a Department of Labor investigation, over its treatment of workers and business practices.
The training materials show how contributors are taught to embed harmful requests in fictional setups to test AI safety filters. Examples include prompts about bomb-making and domestic violence. Scale AI, a leading data labeling service provider, works with major tech companies like OpenAI, Meta, and Google. Despite legal challenges and investigations, Scale AI is seeking a valuation of up to $25 billion. The company defends its business model and claims to offer flexible earning opportunities, while critics allege inadequate mental health support and worker misclassification.
Key takeaways:
Freelancers are paid to stress-test AI with harmful prompts to make AI safer, as revealed by leaked training documents.
Scale AI and Outlier use freelancers to create prompts about sensitive topics like suicide, abuse, and terrorism.
Contributors are encouraged to be creative with harmful prompts but must avoid content related to child sexual abuse material.
Scale AI faces lawsuits and a Department of Labor investigation over worker treatment and misclassification.
