Google is Using Anthropic's Claude To Improve Its Gemini AI - Slashdot
Dec 24, 2024 - slashdot.org
Contractors working to enhance Google's Gemini AI are evaluating its responses by comparing them with outputs from Anthropic's competitor model, Claude, according to a report by TechCrunch. The report highlights that Google did not confirm whether it had obtained permission to use Claude for these comparisons. In the competitive landscape of AI development, companies often assess their models against industry benchmarks rather than directly comparing them with competitors' models.
The contractors involved in this evaluation process are responsible for rating the accuracy of Gemini's outputs based on various criteria, such as truthfulness and verbosity. They are allotted up to 30 minutes per prompt to decide which model, Gemini or Claude, provides a better response. This meticulous evaluation method underscores the effort to improve AI models by directly comparing them with rival technologies.
Key takeaways:
Contractors are comparing Google's Gemini AI outputs against Anthropic's Claude model.
Google did not confirm if it obtained permission to use Claude for testing against Gemini.
AI models are typically evaluated against competitors using industry benchmarks.
Contractors have up to 30 minutes per prompt to assess the accuracy and quality of responses from Gemini and Claude.
