Despite Perplexity bots, Google AI threat looms

The Guardian, while declining to comment specifically for this story, pointed instead to what it told ADWEEK in June, that it blocked Perplexity bots and requested to secure a commercial license for using its intellectual property. The New York Times continues to block Perplexity bots. Forbes did not respond to a media request.

Perplexity responded by referring to its blog stating that it “respects robots.txt.” The blog explained, “Perplexity will not crawl full or partial text content of a news publisher that has disallowed PerplexityBot via robots.txt. Some news web pages may still be indexed even if a page is blocked via robots.txt. In this instance, only the website domain, headline, and a factual summary of the page are added to our search index.”

“Traditional SEO and understanding how Google is incorporating AI is still a much bigger concern [for publishers] than the relatively small volume of traffic being driven by Perplexity,” said David Carr David Carr, editor, insights news and research, Similarweb.

Data from Semrush found a 490.5% year-over-year growth in referral traffic from Perplexity to 7,023 U.S. websites in August, with a peak in June. It’s worth noting that this is from a relatively small base.

The Semrush sites monitored include government and educational websites like the National Institute of Health (nih.gov) and Wikipedia receiving 1.93% and 1.25% of their traffic from Perplexity, respectively. Meanwhile, news sites command only 0.56% of the referral traffic, according to Semrush.

Chartbeat’s analysis of 5,013 sites across 71 countries shows that education-related sites—those focused on teaching, universities, and teacher training—are receiving the highest referral traffic from Perplexity. These sites account for 3,935 of the 860,257 total page views analyzed, representing 0.00457% of total referral traffic in the U.S.

As this last stat shows, Perplexity referral traffic is still modest, for now.