One question, 3 AI assistants, 3 different answers

A screenshot would prove nothing

Two reasons.

The first is that a screenshot of an AI assistant’s answer is a photo of something that has already changed. Different account, different day, different phrasing of the question, different names. Ours would be a snapshot of a suburb that is not yours, taken on a date that has passed.

The second is that you would be taking our word for both halves of it. The question we typed and the answer we got. There is no way for you to audit either. Since the test takes about 20 minutes and works better on your own trade in your own patch, the useful thing to publish is the method.

Write the question the way a customer would say it

The most common mistake is typing your own business name. That puts the answer inside the question. Write 3 questions instead, and keep your name out of the first 2.

1. The panic question. Something has broken and it is happening now. “Hot water’s gone at my place in Coorparoo, Brisbane, who can come out today.”
2. The comparison question. Nothing is broken yet and they are shopping. “Who are the good electricians in Coorparoo, Brisbane.”
3. The named question, last and least useful. Your business by name, to see what it says about you when it has been handed the answer. Treat what comes back as a fact check, not as evidence you are being recommended.

4 rules that decide whether the result means anything:

• Start a fresh chat every time. Anything earlier in the conversation feeds the next answer.
• Turn off memory or personalisation where the setting exists. Otherwise you are testing what it knows about you, not what it knows about your trade.
• Name the city as well as the suburb. There is more than one Springfield in this country. It does not necessarily know where your phone is standing, and it may not tell you it is guessing.
• Do not lead it. No “who is the best emergency electrician in Coorparoo with 5 star reviews”. You will get the answer you asked for.

Keep the log, or do not bother running it

3 questions, 3 AI assistants, run twice at least a day apart, is 18 rows. Without the log you have an impression, and an impression a fortnight later is worthless.

Column	What goes in it
Date and time	Both. The same question can come back different at 9am and 9pm.
Assistant	Which one, and note if the app updated between runs.
Question	Word for word, exactly as typed. Not a summary of it.
Names returned	In order. The order is not a ranking, but a change in it is worth seeing.
Reason given	The sentence attached to each name. This is the important column.
Sources shown	Whether it linked anything, and to what.

4 things to read in what comes back

Are the names businesses at all?

Frequently they are not. You get directories, a “best of” listicle, a council page, a marketplace. If that is what fills the answer for your question, then the thing standing between you and that customer is a directory page, not the bloke 2 suburbs over. That changes what you would do about it.

What reason is attached to each name?

This is the column that teaches you the most. That sentence came from somewhere. Go and find the somewhere. Usually it is a line lifted off the business’s own site, a supplier’s installer page, or an accreditation register. When you can trace the sentence, you are looking at the actual mechanism instead of theorising about it.

Where the 3 disagree

They do not share an index and they are not reading the same set of sources. That is why the lists differ, and it is the single most useful thing the exercise demonstrates. There is no shared scoreboard behind these answers. Nobody can sell you a position in a list that 3 companies compute differently and none of them publish.

Where the 3 agree

More interesting than the disagreement. If the same 2 names turn up in all 3 answers, that business is corroborated across multiple independent sources, which is the closest thing to a signal the whole exercise produces. Look those businesses up properly. Not to copy their website. To find out where else on the internet they exist.

What this test cannot tell you

Worth being blunt about, because the test is easy to run and easy to over-read.

• It is a sample of 1 person on 1 device. It is not a measurement of anything.
• You cannot see what a stranger 5km away with a different account sees.
• It gives you no cause. Something changed between Tuesday and Thursday, and you have no way to find out what.
• It produces no score, no position and no trend. If someone shows you a monthly graph of these answers, ask them what it measures.
• Nobody outside those companies has published how the names get chosen. Not us, and not anyone selling you a report. That is the honest state of it in August 2026.

What it is good for is narrower and real: catching wrong facts about your business, finding out which sentences are being quoted and from where, and seeing whether directories own the questions your customers ask. Those 3 findings all lead to work you can actually do.

The free version of this test

The AI answer test does the copy-and-paste part for you: 1 question, put to 3 AI assistants, all 3 answers shown unedited with nothing rewritten and nothing graded.

It saves you 20 minutes. It does not change any of the limits above. The notebook version in this post is the same test, and running it by hand has one advantage the tool will never have, which is that you were there when the answer came back.

The other half of this is the part you control. What an AI assistant can quote about you depends on what is on your pages and whether it is allowed to fetch them at all, which is a separate and much more fixable problem.