Sign up to save tools and stay up to date with the latest in AI
bg
bg
1

Amazon Is Investigating Perplexity Over Claims of Scraping Abuse

Jun 27, 2024 - wired.com
Amazon's cloud division, Amazon Web Services (AWS), is investigating Perplexity AI, a startup backed by the Jeff Bezos family fund and Nvidia, over allegations of violating AWS rules by scraping websites that tried to prevent it. The startup, valued at $3 billion, is suspected of relying on content from websites that had forbidden access through the Robots Exclusion Protocol, a common web standard. AWS customers are required to adhere to this protocol, and the company's terms of service prohibit any illegal activity.

The investigation into Perplexity's practices follows a report accusing the startup of stealing an article and further evidence of scraping abuse and plagiarism. Perplexity's AI-powered search chatbot has been linked to these activities. The company has been found to access servers using an unpublished IP address, which has been detected on the servers of the Guardian, Forbes, and The New York Times. Perplexity CEO Aravind Srinivas claims the IP address is operated by a third-party company, which he refused to name. Despite the investigation, Perplexity has made no changes to its operation, claiming that its bot respects the robots.txt protocol.

Key takeaways:

  • Amazon's cloud division has launched an investigation into Perplexity AI, a startup backed by the Jeff Bezos family fund and Nvidia, over allegations of violating Amazon Web Services rules by scraping websites that attempted to prevent it.
  • Perplexity AI is accused of ignoring the Robots Exclusion Protocol, a web standard that indicates which pages should not be accessed by automated bots and crawlers, and of plagiarizing content from other websites.
  • Perplexity's CEO, Aravind Srinivas, claims the IP address involved in the alleged scraping is operated by a third-party company, which he refused to name due to a nondisclosure agreement.
  • Despite the investigation, Perplexity has made no changes to its operations and maintains that its services do not violate AWS Terms of Service. However, it admits that its bot will ignore robots.txt when a user enters a specific URL.
View Full Article

Comments (0)

Be the first to comment!