The post
# A lot of AI Visibility reporting I see is based on one answer or one screenshot. But how much can you really conclude from a single response?
We wanted to explore how consistent AI recommendations actually are.
# Method
* 1 buying prompt
* ChatGPT, Perplexity and Gemini: 10 fresh sessions each on Monday
* ChatGPT: 10 additional fresh sessions on Wednesday
* 40 answers in total
* Logged every brand, position, reasoning, and cited source
# Findings (illustrative example)
1. The #1 recommendation changed in 6/10 runs on ChatGPT, 7/10 on Perplexity, and 5/10 on Gemini.
2. Positions 3–5 had at least one different name in 8 out of 10 runs.
3. Between Monday and Wednesday, Otterly AI dropped out of ChatGPT's top 5 and another competitor entered.
4. Only 3 sources were cited in more than half of the runs.
# Takeaways
* Repeat each prompt 5–10 times per engine before drawing conclusions.
* Report "recommended in 7 out of 10 runs" rather than simply saying "we're #1."
* A movement of 1–2 runs shouldn't automatically be treated as a meaningful visibility improvement.
* Track recommendation consistency separately from mentions and citations.
# Limitations
40 runs, one buying prompt, and only three AI platforms. This is an illustrative example, not a representative study of AI search behaviour.
**Has anyone tested recommendation consistency for their own brand? Do you see similar fluctuations, or are your results more stable?**
View the original thread on Reddit →