Articles / Measurement
How to compare your AI visibility with competitors fairly
Measure competitors on the same answers as you, compute the gap answer by answer, and call it only when its 95% range excludes zero. Worked through.
- Published
- Reading time
- 6 min read
- By
- AmpliRank
- Tags
- AI Visibility · Competitors · Paired Comparison
Compare your AI visibility with a competitor’s on the same answers: the same questions, asked to the same engines in the same run, with both brands counted in every answer. Then compute the gap answer by answer and call it only when its 95% range excludes zero; otherwise report “no clear gap”, with the range.
In short
- A fair gap is measured on one set of answers, so the questions, engines and dates are identical for both brands.
- Pairing the comparison answer by answer removes everything the two brands share and leaves only the answers that named one and not the other.
- Two brands can have overlapping ranges and still a clear gap, or a large headline gap and no clear one.
- Measuring a competitor on its own questions or on another day compares two different things.
- Share of voice changes when you add or drop a competitor from the list, so report each brand’s rate on the same answers instead.
Why does a fair comparison need the same answers?
A fair comparison needs the same answers because everything else that differs between two measurements ends up in the gap. AI engines answer every question afresh, so a rate is a sampled estimate, and why one AI answer is not a measurement shows how wide its range is at small samples.
When your rate and a competitor’s come from different questions, different engines or different days, the gap mixes three things: the real difference between the brands, the difference between the samples, and whatever changed in the engines in between. Counting both brands in the same answers removes the last two. The pillar guide on measuring AI visibility sets this as a rule of the method, and the glossary defines a paired comparison.
How do you compare on the same answers?
You compare on the same answers by sorting every answer into four groups: it named both brands, only you, only the competitor, or neither. The gap is the difference between “only you” and “only them”, divided by all answers, and its range comes from how those flips spread across the answers. This is the logic of the classic test for paired proportions (McNemar, 1947): answers that treat both brands the same carry no information about the gap.
Here is a worked example on sample data. Taskmoor, a fictional brand, and its rival Workloft are counted in the same 200 answers to unbranded questions. Taskmoor is named in 50 (25.0%), Workloft in 66 (33.0%).
| Answers that named… | Case A | Case B |
|---|---|---|
| both brands | 30 | 14 |
| only Taskmoor | 20 | 36 |
| only Workloft | 36 | 52 |
| neither | 114 | 98 |
| Workloft’s lead | 8.0 points | 8.0 points |
| 95% range of the lead | 0.6 to 15.4 points | −1.2 to 17.2 points |
| Verdict | clear gap | no clear gap |
Both cases show the same two headline rates. In case A, the brands are often named together and the answers that split them lean toward Workloft, so the range of the lead sits above zero. In case B, the brands rarely appear together and many answers split each way, so the same 8-point lead is within the noise.
The free AI visibility calculator has a gap tab that runs this comparison on your own four counts; the link opens it with case A filled in.
Why do separate samples mislead?
Separate samples mislead because they add noise and differences to the gap that have nothing to do with the brands. Three versions of the mistake are common.
- Comparing two ranges side by side. In the example, Taskmoor’s 50 of 200 has a range of 19.5% to 31.4%, and Workloft’s 66 of 200 has 26.9% to 39.8%. The ranges overlap by 4.5 points, which looks like “no difference”. Yet in case A the paired range of the lead, 0.6 to 15.4 points, excludes zero. Overlap ignores that both brands faced the same answers.
- Measuring each brand on its own questions. A competitor tracked on questions written around its own strengths, or questions that name it, will look stronger for reasons that belong to the questions. Branded vs unbranded questions shows how far a question mix alone can move a rate.
- Measuring on different days. Engines update their models and search indexes, and answers move between runs even when nothing changed. A competitor measured last week and you measured today carries a week of that movement in the gap.
The fix is the same for all three: one run, one set of unbranded questions, every brand counted in every answer.
What about share of voice?
Share of voice is your mentions as a share of all mentions of the brands you track, and it changes when the list of brands changes. If Taskmoor is named in 50 answers and Workloft in 66, Taskmoor’s share of voice is 43.1%. Add Planhaven, named in 40 of the same answers, and Taskmoor’s share falls to 32.1% without a single answer changing.
Share of voice can still be useful inside one fixed list of competitors. Report it next to each brand’s own rate, with its range and the number of answers, so a reader can see what moved: you, a rival, or the list.
Which competitors should you compare against?
Compare against the brands the engines name instead of you, not only the ones your sales team names. The answers already list them: count which brands appear in the answers that leave you out, and add the most frequent ones to your comparison.
Then keep the list fixed for a quarter. Each competitor needs nothing extra to measure, because it is counted in the answers you already collect, but every change to the list breaks share-of-voice trends and makes old reports harder to read.
What can a clear gap not tell you?
A clear gap says the engines name one brand more often on these questions; it doesn’t say why. The reasons sit in the evidence behind the answers: the searches the engines ran and the sources they cited. A rival that is named because a review site the engines cite lists it first needs a different response from a rival whose own pages answer the questions better.
A gap also changes over time like any other figure. To check whether a gap has narrowed, compare it on the questions and engines both runs asked; is that change real? works through the same paired logic for changes.
Where to start
Take your last run, keep only the unbranded questions, and count for your closest competitor the answers that named both of you, only you, and only them. Put the counts into the calculator’s gap tab and report the gap with its range. The method page shows how AmpliRank measures competitors on the same answers as you.
Sources
- Quinn McNemar, Note on the sampling error of the difference between correlated proportions or percentages, Psychometrika 12 (2), 153–157, 1947, accessed 14 November 2026.
- Edwin B. Wilson, Probable Inference, the Law of Succession, and Statistical Inference, Journal of the American Statistical Association 22 (158), 209–212, 1927, accessed 14 November 2026.