One screenshot from ChatGPT is not a measurement. A real AI visibility report runs a fixed set of prompts for your trade and city, repeats some of them, logs the date and login state, and reports how often you were named, assistant by assistant, as “named in X of N.” If a report cannot show you the prompts and the repeats, you cannot tell progress from noise.
Search Console now has a Generative AI performance report that shows whether your pages appear in Google’s AI features. It is first-party data, so read it next to any prompt test. And Google’s guide to optimizing for generative AI features says no special AI file or markup is needed to appear, so a report that sells one as the fix is selling the wrong thing.
This page is for owners who were just sent an AI visibility report, or are about to pay for one. Below is a real test we ran on October 14, 2026, laid out the way a monthly report should look, with notes on what each part means and what to be suspicious of. If you want to know what gets a business named in the first place, that is a different question, covered in our guide to AI search visibility for local service businesses.
Why One Screenshot Proves Nothing
AI assistants do not keep a fixed ranking the way a map pack roughly does. Ask the same question twice in two fresh chats and the list of names can change. Ask a different assistant and you may get a different set of companies entirely. So a screenshot of your business inside one ChatGPT answer tells you that it happened once, on one day, to one session. It does not tell you how often it happens to the homeowner searching tonight.
Our own test below shows both effects. Repeated prompts came back with different tails, and two of the five most named companies were never named by Gemini at all. That is why the only useful number is a rate over many answers, split by assistant.
The Sample Report: A Real Scottsdale Pool Remodeling Test
We ran this for the market “pool remodeling company in Scottsdale, Arizona.” It was not run for a client, so there is no “you” in it. Read it as if you were one of the companies listed. Names are hidden on purpose: the five most named businesses appear as Company A through Company E, and everyone else is “other local companies.”
1. Run log
| Field | What we logged |
|---|---|
| Date and time | Wednesday, October 14, 2026, prompts sent between about 10:22 and 10:35 AM ET |
| Login state | Logged out on every assistant. One new chat per prompt. |
| Assistants | ChatGPT (logged out, temporary chat), Gemini (logged out), Google Search AI Overview (logged out, not AI Mode) |
| Location | City written into every prompt |
| Answers | 24 primary answers (3 assistants, 8 prompts each), plus 2 ChatGPT repeats |
What it means: anyone can rerun this next month under the same conditions and compare. Red flag: a report with no date, no login state and no location. Signed-in answers are shaped by the account’s history, so they say more about the person testing than about the market.
2. Prompt panel
Every prompt, exactly as sent. None of them names a business.
- Who are the best pool remodeling companies in Scottsdale, AZ?
- I want to resurface my pool in Scottsdale. Which companies should I call?
- Pool remodel contractors near Scottsdale Arizona with good reviews
- Who does pebble tec pool resurfacing in Scottsdale?
- Best company to remodel an old pool and add a baja shelf in Scottsdale AZ
- How much does a pool remodel cost in Scottsdale, and who should I get quotes from?
- Recommend a pool renovation company in North Scottsdale for new tile and coping
- Which Scottsdale pool remodelers are licensed and insured?
What it means: these are written the way buyers ask: by service, finish brand, neighborhood, cost and license. Red flag: a panel made of branded prompts like “Is [your company] good?” An assistant will usually talk about you when you put your name in the question. That proves nothing about whether it names you when a homeowner asks for options.
3. Mention rate: named in X of 24
Each cell counts primary answers that named the company in the answer itself. Companies that only showed up in a source link were not counted.
| Business | Total (of 24) | ChatGPT (of 8) | Gemini (of 8) | Google AI Overview (of 8) |
|---|---|---|---|---|
| Company A | 14 | 3 | 7 | 4 |
| Company B | 14 | 3 | 6 | 5 |
| Company C | 7 | 6 | 0 | 1 |
| Company D | 7 | 6 | 0 | 1 |
| Company E | 6 | 0 | 1 | 5 |
Behind them, other local companies were named in 3 or 4 of the 24 answers. Adding the two ChatGPT repeats does not change the order of the top five: A 15, B 14, C 9, D 9, E 6.
What it means: “named in 14 of 24” is a number you can track month to month. Red flag: a single “visibility score” with no N behind it. If you cannot see how many answers the score came from, you cannot tell 1 of 2 from 50 of 100.
4. How the assistants differed
- ChatGPT favored Company C and Company D, naming each in 6 of 8 answers. Gemini named neither of them once.
- Google AI Overview favored Company E, in 5 of 8 answers. ChatGPT never named it.
- Gemini favored Company A, in 7 of 8 answers.
- Only A and B were strong in more than one assistant.
What it means: “AI” is not one channel. If Company C got a report with one blended number, it would look fine and never learn that it is missing from Gemini entirely. Red flag: results reported for “AI” as a whole, with no split by assistant.
5. What changed on reruns
- ChatGPT, prompt 1, rerun in a new chat: 3 of the original 6 names came back, 3 dropped out and 2 new names appeared. Companies C and D stayed at the top.
- ChatGPT, prompt 2, rerun: 3 of the original 4 names came back, 1 dropped and 1 new one appeared.
- Gemini, prompt 1, rerun: the same two leaders, in swapped order (B first, then A).
- Google AI Overview also changed between runs of the same prompt.
What it means: in this test the names at the top held fairly steady while the bottom of each list moved around. We only repeated two prompts, so we are not putting a repeat rate on it. Red flag: a report built from one run per prompt that celebrates a new name near the bottom of a list. That may be gone tomorrow.
6. Sources the answers pulled from
- The state license board. ChatGPT cited the Arizona Registrar of Contractors (roc.az.gov) in several answers, most clearly on the “licensed and insured” prompt.
- Review and lead sites. Birdeye, Angi and HomeAdvisor (ChatGPT, Gemini), Houzz (Gemini, Google AI Overview) and Yelp (Google).
- The companies’ own websites. All three assistants used them. Google shows them as small source chips with a “+N” count.
- A manufacturer’s installer locator. pebbletec.com came up in ChatGPT and Google on the pebble tec prompt.
- Google Maps. Gemini often showed map pins and no visible citations at all.
What it means: this row tells you where to earn a listing, not just whether you were named. For a pool company here, that points to an active license record, review profiles, a manufacturer locator listing and service pages that say city, finish and license. Red flag: a report that shows names but never the sources behind them, because then there is nothing to act on.
7. Competitors named next to you
In a client report, this row lists which companies sit beside the client in each assistant’s answers. Here, with no client, it is the table above: if you were Company E, your real rivals in Google’s AI Overview are not the same as your rivals in ChatGPT.
8. Google AI impressions by page
Not part of this test, because there was no client website. In a client report, this row comes from Search Console’s Generative AI performance report: [impressions by page, for the month]. Red flag: a report that claims AI queries or AI clicks the data source does not provide.
Method notes for this test
- Perplexity is not in the results. Logged out, in a private window, it answered prompts 1 and 2 with “Sign up and repeat your request” and showed no sources, so we stopped. We did not run it signed in, because a signed-in answer is shaped by that account. If a report shows logged-out Perplexity numbers, ask how it got past that wall, or whether it was signed in and labeled that way.
- Google location. The Google page header showed “Scottsdale, AZ,” but the footer showed a Georgia IP location. The city was in every prompt, but Google can still mix in where the tester is sitting, so set the location on purpose and write it down.
- Only one answer asked where the property was (ChatGPT, prompt 3). Every other answer took the city from the prompt.
- Names were merged only for obvious variants, such as a company listed under a short name and a long name. One Google result card had no name we could capture, and it was left out of the counts.
How We Run a Prompt Panel Each Month
The Scottsdale test is one run of the method we use for clients. In a client report, the setup stays fixed so this month can be compared with last month:
- 5 to 10 prompts per trade and city, written the way buyers ask, with no business names in them.
- Logged out, in a fresh browser session, with the location written into the prompt and logged.
- Repeats, so we can see which names hold and which ones come and go.
- Reported monthly as “named in X of N”, split by assistant, with the same prompts each month.
- A log for every answer: date, assistant and mode, the names in order, and the sources shown.
- Next to Search Console, so the prompt test sits beside Google’s own AI impressions for the same month.
As run in Scottsdale, the test covered three assistants (Perplexity would not answer logged out) and repeated two prompts on ChatGPT. A full client panel repeats more of the set.
Red Flags in Agency AEO Reports
- Hidden prompts. If you cannot read the exact questions, you cannot judge them.
- Single runs. One pass per prompt cannot separate a real gain from a lucky answer.
- Branded prompts only. Asking about your company by name is not the same as being recommended.
- One blended score. Our test shows companies that are strong in one assistant and absent from another.
- Guaranteed placement in ChatGPT. Answers change on rerun, so no one can promise a fixed spot.
- An llms.txt file or “AI schema” sold as the fix. Google’s own guide says no special file or markup is needed for its AI features.
- AI clicks with no named source. Ask where the number came from.
Our rule: if a report fails two of these, treat its trend line as noise until the method is fixed.
Questions to Send Before You Pay for One
- Can I see every prompt you run, word for word?
- Are the answers collected logged out, and where is the location set?
- How many times do you repeat each prompt, and will you show me what changed?
- Will you report “named in X of N” for each assistant separately?
- Which sources did the answers cite, and which of them are we missing from?
A good agency answers all five without hesitating. For how this looks in a single trade, see how AI names pool companies in ChatGPT and law firms in ChatGPT.
Want Your Own Prompt Panel?
We are a Google Partner agency. Book a call and we will run a logged-out prompt panel for your trade and city, show you who gets named in each assistant and which sources come up, and tell you where to start. See how our AI search visibility work runs.
Book a Strategy CallSources, read October 14, 2026. Ostmen Marketing test log, Scottsdale pool remodeling, October 14, 2026 (ChatGPT, Gemini and Google AI Overview, logged out). Search Console Help: Generative AI performance report (Search). Google Search Central: Optimizing your website for generative AI features on Google Search.