The new tool has successfully detected a bot that collects content for Perplexity AI Inc., a search engine startup. Cloudflare assigns a score of 1 to 99 to every website visit processed by its platform, with a lower score indicating a higher likelihood of the request being generated by a bot. The company will update the feature over time to address changes in AI scraping bots' technical fingerprints and the emergence of new crawlers. It is also rolling out a tool that allows website operators to report any new bots they encounter.
Key takeaways:
- Cloudflare has introduced a no-code feature to prevent AI developers from scraping website content, available in both free and paid tiers of its CDN.
- The feature uses AI to detect automated content extraction attempts, even when they attempt to avoid detection.
- Cloudflare managed to detect a bot that collects content for Perplexity AI Inc., a search engine startup, which had been making its requests appear as regular user traffic.
- Cloudflare will continue to update the feature to address changes in AI scraping bots’ technical fingerprints and the emergence of new crawlers, and is also rolling out a tool for website operators to report any new bots they encounter.