Sign up to save tools and stay up to date with the latest in AI
bg
bg
1

Exclusive: Google's Gemini is forcing contractors to rate AI responses outside their expertise

Dec 19, 2024 - techcrunch.com
Google has updated its guidelines for contractors working on its AI project, Gemini, requiring them to evaluate AI-generated responses even if they lack expertise in the subject matter. Previously, contractors could skip prompts outside their domain knowledge, but now they must rate the parts they understand and note their lack of expertise. This change has raised concerns about the accuracy of Gemini's responses on sensitive topics, such as healthcare, as contractors may not have the necessary background to assess technical AI outputs accurately.

The new guidelines allow skipping prompts only if they are missing information or contain harmful content. Google stated that while raters provide feedback on various aspects, their ratings do not directly impact algorithms but serve as a data point to measure system performance. This shift in policy has led to worries about the potential for inaccurate information being disseminated by Gemini, especially on complex issues.

Key takeaways:

```html
  • Google has updated its guidelines for contractors working on the Gemini AI project, requiring them to evaluate prompts even if they lack domain expertise.
  • Contractors are now instructed to rate the parts of the prompt they understand and note their lack of expertise, raising concerns about the accuracy of AI responses on specialized topics.
  • Previously, contractors could skip prompts outside their expertise, but now they can only skip if information is missing or if the content is harmful.
  • Google claims that while raters' feedback does not directly impact algorithms, it is a valuable data point for measuring system performance.
```
View Full Article

Comments (0)

Be the first to comment!