Sign up to save tools and stay up to date with the latest in AI
bg
bg
1

Half of Top News Sites Blocked OpenAI’s Crawlers in 2023

Feb 26, 2024 - adweek.com
The ADWEEK Sports Marketing Summit, Brand Play, is inviting registrations for its event in NYC or virtually on May 9. A study by Reuters Institute reveals that by the end of 2023, 48% of top news websites across 10 countries, including Germany, India, Spain, the U.K. and the U.S., blocked OpenAI's crawlers, while 24% blocked Google's AI crawler. The study analyzed the robots.txt of 15 online news sources with the widest reach, such as The New York Times, BuzzFeed News, The Wall Street Journal, The Washington Post, CNN and NPR.

The study found that 57% of legacy print publications, 48% of television and radio broadcasters, and 31% of digital-born outlets blocked OpenAI's crawlers. Similarly, 32% of print outlets, 19% of broadcasters, and 17% of digital-born outlets blocked Google's crawlers. The study also found that news outlets in the Global North were more likely to block AI crawlers than those in the Global South. Once a decision to block was made, no website reversed its stance.

Key takeaways:

  • By the end of 2023, 48% of the top news websites across 10 countries blocked OpenAI's crawlers, and 24% blocked Google's AI crawler, according to a study by Reuters Institute.
  • Legacy print publications were more likely to block AI crawlers, with 57% blocking OpenAI's crawlers and 32% blocking Google's.
  • News outlets in the Global North were more inclined to block AI crawlers compared to those in the Global South. In the U.S., 79% of top online news websites blocked OpenAI, while only 20% did so in Mexico and Poland.
  • Once a decision to block was made, no website reversed its stance by unblocking either an OpenAI or Google AI crawler.
View Full Article

Comments (0)

Be the first to comment!