OpenAI Tests Its AI's Persuasiveness By Comparing It to Reddit Posts - Slashdot
Feb 02, 2025 - slashdot.org
OpenAI has utilized the subreddit r/ChangeMyView to test the persuasive abilities of its AI reasoning models, as reported by TechCrunch. The company collects user posts from the subreddit and asks its AI models to craft replies intended to change the original poster's mind. These AI-generated responses are then evaluated by testers for their persuasiveness and compared to human replies. OpenAI clarified that this evaluation process is separate from its content-licensing deal with Reddit, which allows it to train on Reddit posts. The company has no plans to release the evaluation publicly and aims to ensure its AI models do not become overly persuasive, addressing concerns about AI's potential for persuasion and deception.
The ChangeMyView subreddit, with its 3.8 million subscribers, provides a rich source of human interactions for OpenAI's evaluations. Despite the licensing deal, it's unclear how OpenAI accessed the subreddit's data for this specific purpose. The article also notes that Reddit CEO Steve Huffman mentioned difficulties in negotiating with companies like Microsoft, Anthropic, and Perplexity, highlighting the challenges Reddit faces in managing its data partnerships.
Key takeaways:
OpenAI used the subreddit r/ChangeMyView to test the persuasive abilities of its AI reasoning models by having them write replies to change users' minds.
OpenAI has a content-licensing deal with Reddit, but the ChangeMyView-based evaluation is reportedly unrelated to this agreement.
The goal of OpenAI's evaluation is to ensure AI models do not become overly persuasive, addressing concerns about persuasion and deception.
Reddit CEO Steve Huffman mentioned difficulties in negotiating with companies like Microsoft, Anthropic, and Perplexity regarding access to Reddit data.
