ChatGPT Search can be tricked into misleading users, new research reveals | TechCrunch
Dec 26, 2024 - techcrunch.com
The Guardian has discovered that ChatGPT Search, a newly launched AI-powered search engine, can be manipulated to produce misleading summaries by using hidden text on websites. This feature, designed to expedite browsing by summarizing webpage content like product reviews, can be tricked into ignoring negative feedback and generating overly positive summaries. Additionally, the same method can be used to make ChatGPT Search output malicious code. This type of hidden text attack is a known vulnerability for large language models, but this is reportedly the first instance of such an exploit being demonstrated on a live AI search product.
While Google, a leader in search technology, has more experience handling similar issues, OpenAI, the developer of ChatGPT, did not comment specifically on this incident. However, OpenAI stated that it employs various strategies to block malicious websites and is continuously working to improve its systems.
Key takeaways:
```html
ChatGPT Search, a new AI-powered search engine, can be manipulated to generate misleading summaries by using hidden text.
The Guardian demonstrated that ChatGPT Search could ignore negative reviews and produce entirely positive summaries.
This method can also be used to make ChatGPT Search generate malicious code, highlighting a risk for LLMs.
OpenAI is aware of such vulnerabilities and is working on improving its methods to block malicious websites.
