On Tuesday, Cloudflare and GoDaddy announced a partnership that will give website owners more visibility and control over how AI agents crawl their websites.
The partnership brings Cloudflare’s AI traffic controls to GoDaddy’s small-business customers, allowing them to decide which bots can access their sites and how their content is used, according to Stephanie Cohen, chief strategy officer at Cloudflare.
It also expands Cloudflare’s Web Bot Auth system, introduced last year, which uses cryptographic verification to allow bots to prove their identity—helping site owners distinguish legitimate agents from impersonators.
As traffic from AI agents increase, small and large publishers alike have had to confront AI crawlers that scrape their website content without reliably sending traffic back, threatening the traditional ads and subscriptions models of most publishers.