OpenAI, the creator of ChatGPT, has also launched a web crawler with instructions on how to block it. Several major publications, including Medium, the New York Times, CNN, and Reuters, have chosen to block it. Google's VP of Trust asserts that this move aligns with their commitment to responsible AI development and consumer privacy.
Key takeaways:
- Google is introducing a new tool called Google-Extended that allows websites to be indexed by crawlers without their data being used to train future AI models.
- Website administrators can control this feature through robots.txt, a text file that allows web crawlers to access sites.
- OpenAI, the maker of ChatGPT, has also launched a web crawler with instructions on how to block it, with notable publications such as Medium, the New York Times, CNN and Reuters choosing to do so.
- Google's VP of Trust emphasizes the company's commitment to developing AI responsibly, guided by AI principles and consumer privacy commitments.