The article also highlights differences in blocking behavior among different types of publishers. Legacy print outlets and outlets with a larger reach were more likely to block AI crawlers. The article concludes by noting that this is a rapidly evolving area, and the situation could change in the short term, especially as some publishers look to strike deals with AI companies and new products are being developed all the time.
Key takeaways:
- By the end of 2023, 48% of the most widely used news websites across ten countries were blocking OpenAI’s crawlers, and 24% were blocking Google’s AI crawler.
- Almost every website (97%) that decided to block Google’s AI crawler was also blocking OpenAI’s crawlers.
- The proportion of news websites that blocked OpenAI varied considerably by country, ranging from 79% in the USA to just 20% in Mexico and Poland.
- All types of news outlet were blocking, but the websites of legacy print publications were more likely to be blocking than those of either broadcasters or digital-born outlets.