Almost all leading AI chatbots show signs of cognitive decline - BMJ Group
Dec 20, 2024 - bmjgroup.com
A study published in the Christmas issue of _The BMJ_ reveals that leading AI chatbots, including ChatGPT and Claude, exhibit signs of mild cognitive impairment when assessed using the Montreal Cognitive Assessment (MoCA) test, a tool typically used to detect early signs of dementia in humans. The study found that older versions of these chatbots performed worse, similar to older patients. While ChatGPT 4o scored the highest with 26 out of 30 points, Gemini 1.0 scored the lowest with 16. All chatbots struggled with visuospatial skills and executive tasks, such as the trail making and clock drawing tests, and failed to show empathy or interpret complex visual scenes accurately.
These findings challenge the notion that AI will soon replace human doctors, highlighting significant weaknesses in tasks requiring visual abstraction and executive function. The study suggests that neurologists are unlikely to be replaced by AI in the near future and may instead find themselves addressing cognitive impairments in AI models. The research underscores the essential differences between human brains and large language models, emphasizing the limitations of AI in clinical settings.
Key takeaways:
```html
Leading AI chatbots show signs of mild cognitive impairment, similar to early signs of dementia in humans.
ChatGPT 4o scored the highest on the Montreal Cognitive Assessment (MoCA) test, while Gemini 1.0 scored the lowest.
All chatbots struggled with visuospatial skills and executive tasks, highlighting a significant area of weakness.
The study suggests that AI models may present with cognitive impairments, challenging the notion that they will soon replace human doctors.
