The Tow Center's evaluation of ChatGPT Search's ability to identify sources revealed that out of 200 queries, 153 responses were incorrect, and the AI rarely acknowledged its mistakes. The study also found that ChatGPT Search sometimes cites copied or syndicated articles instead of original sources due to publisher restrictions or system limitations. OpenAI responded by stating its support for publishers through clear attribution and content discovery, and its commitment to improving search results. However, the report suggests that OpenAI needs to ensure accurate representation of publisher content in ChatGPT Search to earn user trust.
Key takeaways:
- OpenAI’s ChatGPT Search has been found to frequently misquote and incorrectly attribute sources, according to a study by Columbia University’s Tow Center for Digital Journalism.
- The study found that even when publishers specify their participation choice via the robots.txt file, they still face the risk of misattribution and misrepresentation.
- ChatGPT Search sometimes cites copied or syndicated articles instead of original sources, which raises concerns about OpenAI’s content filtering and its approach to journalism.
- OpenAI responded to the findings by stating that it supports publishers through clear attribution and helps users discover content, and remains committed to improving its search product.