Sign up to save tools and stay up to date with the latest in AI
bg
bg
1

Google's medical AI destroys GPT's benchmark and outperforms doctors

May 06, 2024 - newatlas.com
Google's Med-Gemini, a new generation of multimodal AI models, has shown promising results in the medical field. The AI model can process information from different modalities, including text, images, videos, and audio, and is adept at language and conversation, understanding diverse information, and reasoning from large amounts of data. Med-Gemini was tested on 14 medical benchmarks and established a new state-of-the-art performance on 10, surpassing the GPT-4 model family on every benchmark where a comparison could be made. It also demonstrated the ability to retrieve specific information from lengthy electronic health records and engage in diagnostic dialogues.

However, the researchers acknowledge that there is still much work to be done, particularly in terms of privacy and fairness considerations. They envision a future where AI systems like Med-Gemini can accelerate biomedical discoveries and assist in healthcare delivery, but stress the importance of ensuring the reliability and safety of these systems.

Key takeaways:

  • Google's Med-Gemini is a new generation of multimodal AI models that can process information from different modalities, including text, images, videos, and audio, and has been fine-tuned for medical applications.
  • Med-Gemini was trained on MedQA, multiple-choice questions representative of US Medical License Exam (USMLE) questions, and was also developed with two novel datasets, MedQA-R (Reasoning) and MedQA-RS (Reasoning and Search).
  • The AI model performed well in tests, including a 'needle-in-a-haystack' task using a large, publicly available database, the Medical Information Mart for Intensive Care or MIMIC-III, and was able to retrieve the relevant mention of a rare and subtle medical condition, symptom, or procedure.
  • While the initial capabilities of Med-Gemini are promising, the researchers acknowledge that more work is needed, particularly in the areas of privacy and fairness, to ensure the AI system does not unintentionally reflect or amplify historical biases and inequities.
View Full Article

Comments (0)

Be the first to comment!