I'm Spartacus, No, I'm Spartacus: Measuring and Understanding LLM Identity Confusion
Dec 29, 2024 - arxiv.org
The article discusses the widespread use of Large Language Models (LLMs) in various fields such as education, business, and creative industries, highlighting their capabilities in tasks like text generation and data analysis. However, it raises concerns about the originality and trustworthiness of LLMs, particularly focusing on a phenomenon called identity confusion, where LLMs misrepresent their origins or identities. The study investigates the prevalence of identity confusion, its causes, and its impact on security and trust. An automated tool was developed to analyze documentation, test self-identity recognition, and compare output similarities to assess identity confusion among 27 LLMs, revealing that 25.93% exhibit this issue.
The study found that identity confusion primarily results from hallucinations rather than model reuse or plagiarism. A structured survey conducted via Credamo indicated that identity confusion significantly undermines user trust, especially in critical areas like education and professional use, more so than logical errors or inconsistencies. Users attributed these trust issues to design flaws, incorrect training data, and perceived plagiarism, highlighting the systemic risks posed by identity confusion to the reliability and trustworthiness of LLMs.
Key takeaways:
```html
Large Language Models (LLMs) are widely used across various domains but face issues of originality and trustworthiness.
Identity confusion in LLMs, where models misrepresent their origins, is a significant concern, affecting 25.93% of the models studied.
Identity confusion primarily arises from hallucinations rather than model reuse or plagiarism.
This confusion significantly undermines user trust, especially in critical areas like education and professional use.
