TJ Robertson’s analysis of 193,507 AI answers from ChatGPT, Google AI Mode, and AI Overviews includes two rounds of blind page ratings. In both rounds, cited pages opened with a direct answer to the question more often than the pages they were compared with.
When the same AI answers pulled up pages, there was a difference in how often these pages began with a direct answer when they were cited (23%) versus those that were not cited (14%). There also was a difference when comparing cited pages with Google Top-10 pages the AI never pulled up (20% vs. 5%).
The results are based on data collected by Robertson’s company TJ Digital, which sells both Search Engine Optimization (SEO) services and Generative Engine Optimization services. The data is gathered through purchasing questions tracked by Robertson’s company for its clients. Robertson states he views his findings as showing associations rather than any specific page traits causing a citation.
How The Analysis Was Built
The total number of responses (answers) within the study’s analysis was 193,507. Those responses addressed 4,167 questions that did not reference a specific company or product name. The questions covered 78 different client companies representing nine different business models. Responses were captured over the course of eight “snapshots” on an individual week basis during the time frame of August 27 through October 21, 2026, plus repeat runs using the same questions. The agency captures the questions used in those “snapshots” with Peec AI which is an application program that presents a pre-determined set of prompts to AI tools and then collects both the AI tools’ answers and the source(s) of their answers.
In order to collect comparative data regarding the characteristics of pages, the agency extracted and compared the content of 21,874 pages that were cited in AI answers with 4,507 comparative pages provided for 434 questions.
The agency referred to the comparative pages as “passed-over pages”. All such comparative pages were identified as having been among the top ten search results in Google for a specific question; however, they “never came up in the AI’s searches.” The agency relied upon keyword data from Ahrefs for those rankings as opposed to the actual questions being asked. According to the analysis, the agency categorized 174 of those 371 keywords as being similar to the question posed while categorizing the remaining 197 of the keywords as being more general or broader.
Cited Pages Answered Up Front More Often
Robertson had AI raters blind-score 195 pairs of pages so the raters did not know whether the first or second page in each pair was cited. The 20% to 5% split of opening answers based upon direct-answers was derived from this round, which paired cited pages with passed over pages.
Also in this round, half of the passed over pages were missing an answer to the question anywhere within the portion of the page that the raters read; while 23% of the cited pages missed an answer. Also in this round, cited pages received a higher rating for completeness than passed over pages. The rating for completeness was 2.3 (on a 1-to-5 scale) for cited pages, while it was 1.7 for passed over pages.
There was little difference in terms of “promotional” tone between cited and passed over pages. Approximately three fourths of all the pages in both categories were business pages trying to promote their own product(s)/service(s), and both categories rated very similarly (i.e., 3.8 and 3.7) in terms of “tone.”
The 23% to 14% split on opening answers was determined by another round prior to this one where cited pages were being compared to pages that the exact same answer brought up via search, however, did not cite those pages. At present (as of Oct. 28), Robertson’s page has yet to identify how many pages he analyzed in this round, nor has it identified which specific AI model he used to evaluate the pages.
Other Page Traits Linked To Citations
The analysis compares traits on cited and passed-over pages, accounting for each site’s Ahrefs Domain Rating and page type, and reports the results as odds of being cited. A year in the title came with more than twice the odds. A publish or update date in the past 12 months came with 88% higher odds, tables with 69%, first-hand experience with 45%, and prices with 28%.
Pages with an author byline had 28% lower odds, and pages with a call to action had 12% lower odds. Robertson says he can’t explain those two results and wouldn’t remove either element based on them alone.
He says there was an association of 13% higher odds when using schema markup. I covered a report from Ahrefs back in May showing that pages cited by AI were approximately three times as likely to contain JSON-LD. The addition of JSON-LD to pages that were previously cited by the AI Overview did result in no statistically significant citation increase based upon Ahrefs’ testing.
He also states the content traits identified in this analysis were duplicated in a second test (limited to Google Top-10 Pages). The page has references to a per-engine table and a graph regarding those trait odds; however, as of Oct. 28, both the table and the graph are not visible.
Why This Matters
On your own product/service pages, you can find out whether the page provides an answer to the buyer’s question early in the page. In this study the difference in direct-answers to a question provided up front by cited vs. uncited pages was smaller when the uncited pages were also being retrieved by the AI.
Blind rater data shows that citations do not prove that a page has answered the question. A little under a quarter of the cited pages did not provide an answer to the buyer’s question within the portion of the page that raters reviewed.
Page trait odds are based on questions asked by one agency for their clients and describe how cited and passed over pages differed; therefore they are patterns that you may want to test on your own monitored questions.
Looking Ahead
The agency did an analysis comparing pages as they were and cannot show if putting in a direct answer for a question will change the number of times a web-page is cited. To be able to see if the addition of a direct answer changes citations, a comparison study such as was conducted by Ahrefs on schema will need to be done where a test group adds it and a control group does not, measured before and after the addition. Until then, the opening-answer pattern is a hypothesis to compare against what questions you currently track.
Featured Image: Accogliente Design/Shutterstock
https://www.searchenginejournal.com/ai-cited-pages-more-often-got-straight-to-the-answer/594781/
