Despite these flaws, some programmers still prefer ChatGPT, with a small sample of 12 programmers preferring the AI's answers 35% of the time and failing to catch AI-generated mistakes 39% of the time. The researchers suggest that the AI's polite language and comprehensive responses may make its answers seem more convincing, leading users to overlook inaccuracies.
Key takeaways:
- A team of researchers from Purdue University found that 52 percent of programming answers generated by OpenAI's ChatGPT are incorrect.
- The study also found that 77 percent of the AI's answers are more verbose than human answers, and 78 percent suffer from inconsistency compared to human responses.
- Despite these inaccuracies, many programmers seem to prefer ChatGPT's answers, possibly due to its polite language and comprehensive, textbook-style responses.
- The study highlights that while AI platforms like ChatGPT are increasingly used, they still have significant flaws and can generate misinformation.