ChatGPT checks brand sites mostly after it builds a shortlist
Summary
ChatGPT builds its brand shortlist from the pages its first searches retrieve and checks a brand's own site mostly afterwards, so fewer than 1 in 10 brands were named before any search in Seer Interactive's study of the model's reasoning traces.
Read the reasoning for your own priority prompts and state every must-have it names, such as licenses and integrations, in plain text on the page it will check.
ChatGPT picks the brands it recommends from the pages it retrieves during a run, and it checks a brand’s own site mostly after that shortlist exists, Seer Interactive found by reading the model’s reasoning traces. The digital marketing agency ran 20 prompts through OpenAI’s API and read every reasoning summary the model wrote between its searches.
Most brands arrived with the search results. Fewer than 1 in 10 were named before the model ran a single search. Most of them, 59%, appeared in the reasoning step straight after the first round of broad “best of” and review queries, and the rest were added in later rounds. The model named about eight candidates per prompt and mentioned about six in the answer, so the pool a brand has to enter is small, and it opens only after retrieval has happened. Which pages get retrieved is a separate question from which pages rank: only a quarter of ChatGPT’s cited sites appeared on Bing’s first page in an earlier analysis.
The second wave of searching points at the shortlist itself. Only 13% of queries in the first search round used the site: operator to search a single website, while 75% did so in later rounds. The model finds a brand in general results, then goes to that brand’s own pages to check it against the conditions it set earlier, a pattern an earlier look at ChatGPT’s fan-out queries also showed. About half of the shortlist reasoning notes added at least one such condition, such as routing and CRM integrations on a B2B software prompt.
Seer’s home insurance prompt ran that sequence end to end. Asked which home insurance companies are most recommended for homeowners in Georgia, ChatGPT planned to “check credible sources, including the state insurance regulator, a ratings agency, and an insurance association,” then searched for phrases such as “best homeowners insurance companies Georgia 2026 recommendations.” After reading a personal-finance site’s Georgia guide it wrote, “I’m considering a list of five insurance companies,” and its targeted follow-up went to a second personal-finance guide to verify those five.
The last reasoning step before the answer removes brands, and 38% of those final notes say why. One insurer “could be optional, but we might trim that,” the model wrote, and that insurer did not appear in the response. Of the ten brands flagged for trimming across the runs, nine were missing from the final answer, so a hedge in a trimming note is close to a verdict.
Seer groups the exclusions into five reasons and rates the effort to fix each:
| Reason | Example from a trace | Effort to fix, per Seer |
|---|---|---|
| Brand does not fit the prompt’s need, location or use case | A price-comparison site “isn’t available in the US” | High, often not fixable |
| Brand’s own site lacks the signal the model looked for | A business phone provider had “limited information” on its pages | Low |
| Known weakness on what the user asked for | A dry shampoo brand is “often powdery” on a prompt that asked to avoid powder | High |
| Thin third-party validation | A practice management vendor “isn’t as often featured in PT-specific lists” | Medium |
| Bottom of the shortlist, sixth when the model planned five | Depends on the other four |
Only the second of those is work a site controls on its own, so the useful part of a trace is the list of claims it expects your pages to confirm. In one run the reasoning said Maryland requires an HVACR license and an MHIC license and that it was crucial to verify them. A brand that holds the license but does not state it in plain text on the page being checked loses the same way a brand without the license does. The trust reason is harder to buy your way out of: in another run the reasoning noted that “SEO-driven listicles and self-published content dominate the search results,” so it would assess official agency sites and customer reviews instead.
The caveat is the method. The 20 prompts ran on 4 November 2026 through OpenAI’s API on GPT-6 Luna at medium reasoning, the setting Seer found closest to the app, and Seer describes the API as directional rather than a 1:1 match for what users see. The sequence is the useful part, not the exact percentages.
What to do
- Run your own priority prompts and read the reasoning summaries, through the API or Seer’s trace tool, and write down every must-have the model names: licenses, certifications, integrations, service areas.
- Put the ones you hold in plain text on the page a
site:search would land on, not in a PDF, a badge image, or a sentence three clicks deep. - Note which sources the first reasoning step says it trusts, which for Seer’s insurance run were a state regulator, a ratings agency and an industry association, and check whether you appear in them.
- Count the list size the model plans. If it wants five and you are sixth, the work sits with the four above you rather than with your page.