The researchers also tested AMIE's ability to assist physicians interactively. When doctors had access to AMIE, they were able to include the correct diagnosis in their lists 52% of the time, compared to only 36% when using online search tools. Despite these promising results, the authors note that rigorous real-world testing is still needed before the system can be used in healthcare settings.
Key takeaways:
- Researchers at Google have developed an AI system based on conversational large language models (LLMs) that can aid physicians in generating differential diagnoses, a task that can be challenging even for experienced doctors.
- The AI system, called Articulate Medical Intelligence Explorer (AMIE), was tested on complex real-world case reports and outperformed experienced board-certified physicians, including the correct diagnosis in the top 10 possibilities 59% of the time compared to the doctors' 34%.
- When used as an interactive assistant to physicians, AMIE improved the doctors' diagnostic accuracy from 36% to 52% compared to using online search tools and medical resources.
- Despite the promising results, the authors note that rigorous real-world testing is still needed before the system can be used in healthcare settings, with considerations around system safety, fairness, and transparency being paramount.