“We are conducting this work under the purview of the U.K.’s Competition and Markets Authority and privacy regulators, so there is independent regulatory oversight,” the spokesperson added. The spokesperson added that Google’s aim with Privacy Sandbox is to be transparent and collaborative, and it will continue to do more tests.
What we know: performance is only slightly worse
Google tested the efficacy of three types of interest-based audience solutions (dubbed IBA); contextual signals, Privacy Sandbox protocol Topics API and Publisher Provided IDs, which lets brands transact on first-party data within a particular publisher.
The study did not include publisher-provided signals, a tool Google introduced last fall that lets publishers transact contextual data across the open web according to seller-defined audiences protocol.
Google compared status quo traffic, which includes some IBA signals and third-party cookies, with traffic where third-party cookies had been removed.
Google looked at whether advertisers spent more money bidding on the IBA traffic, compared to traffic with third-party cookies, to understand whether campaigns using privacy-preserving signals would allow marketers to run the same size campaigns. It found advertiser spend on privacy-preserving IBA was 2%-7% less than on third-party cookies.
Google also measured the average number of conversions on an ad compared to the dollars spent on that ad, as a measure of how well the ad worked for marketers. Google found that on the IBA inventory, conversions per dollar were 1% to 3% lower than the conversions generated for status quo traffic powered by third-party cookies.
Finally, click-through rates on IBA traffic were within 90% of the status quo.
Google said click-through rates were a proxy for ad relevance, though Webster questioned the reliability of this metric.
“CTR is a fairly meaningless metric,” Webster said, noting that the IAB U.K. has called on brands to stop using the measure to judge ad effectiveness. “Most clicks on mobile (ie, most clicks) are accidents due to annoying placements.”
What we don’t know: efficacy of each protocol
Google’s test grouped together three different types of signals, showing all their results together, making it hard to isolate the efficacy of any one in the post-cookie landscape.
Of note, observers can’t pull out the results on Topics API, which was rejected by trade body W3C earlier this year and has faced criticism from the wider industry.
