AI researcher Simon Willison and Dr. Margaret Mitchell, chief ethicist scientist at Hugging Face, both express concerns about the use of AI in this context. They argue that ChatGPT's training data is unlikely to include the full text of each book in question, and even if it did, the AI's results should not be trusted without human verification. Furthermore, they highlight the irony of education officials outsourcing critical decisions about curriculum to a system that lacks understanding and critical thinking abilities.
Key takeaways:
- Administrators in Iowa are using AI, specifically ChatGPT, to help them remove books from school libraries that contain descriptions or depictions of sex acts, in response to a new state law.
- The AI is asked if a book contains a description or depiction of a sex act, and if the answer is yes, the book is removed from circulation.
- Experts warn that AI models like ChatGPT are not equipped to make these kinds of decisions and can produce unreliable results, as they may not have seen the full text of the book in question.
- There is concern about automation bias, where humans place undue trust in machine decision-making, especially in matters of law, safety, or public health.