Skip to content

Articles / Choosing a tool

How to choose an AI visibility tool: 8 questions to ask any vendor

Eight questions that separate a measurement from a guess: answers behind each number, repeats, ranges, same-answer comparisons, engines, sources and units.

Published
Reading time
8 min read
By
AmpliRank
Tags
AI Visibility · Buying Guide · Checklist

Choose an AI visibility tool by how it measures, not by how its dashboard looks. Ask any vendor eight questions: how many answers sit behind each number, how often each question is repeated, whether a range is shown, whether competitors are compared on the same answers, whether branded questions are reported apart, which engines are asked and how often, whether the searches and sources are shown, and what a plan’s unit buys each month.

In short

  • An AI visibility figure is a sampled estimate. A tool should show the sample behind it and the range around it.
  • “40%” from 20 answers could be anything from 21.9% to 61.3%; from 400 answers, 35.3% to 44.9%. Ask which one you’re buying.
  • Competitors should be measured on the same answers as you, and changes called only when their range excludes zero.
  • Branded questions inflate a headline; ask for the unbranded rate on its own.
  • Plans use different units. Convert each to answers a month before comparing prices.

Why do these questions matter?

These questions matter because two tools can report different numbers for the same brand and both be honest about their arithmetic. AI engines answer each question afresh, so every visibility figure is an estimate from a sample of answers. The pillar guide on measuring AI visibility sets out the method; the questions below check whether a tool follows it.

Ask them on a demo, in a trial or in writing. A vendor that answers all eight with specifics is selling a measurement. A vendor that answers with a score and a chart is selling a picture. The AI visibility glossary defines the terms the questions use.

1. How many answers are behind each number?

Every figure should come with the number of answers it was computed from. Without it, you can’t tell a firm number from a lucky one.

Reported rate Answers behind it 95% range
40% 20 21.9% to 61.3%
40% 40 26.3% to 55.4%
40% 200 33.5% to 46.9%
40% 400 35.3% to 44.9%

The four rows show the same headline and very different certainty. How many answers do you need? gives the counts for other rates and margins.

Good answer: “Each rate shows its answer count, by engine and in total.”

2. How many times is each question asked?

Each question should be asked more than once over time, and you should be able to see how many times. AI engines give different answers to the same question, so one asking per question per month is a snapshot of a moving target.

Repeats are not the whole answer: answers to one question tend to agree, so more distinct questions add more information than more repeats of a few. Ask how many distinct questions and how many repeats make up the sample.

Good answer: “You choose the repeats per question, and the report shows questions, engines and repeats separately.”

3. Is a range shown with every rate?

A rate should never appear without its range. The range says where the true rate very likely sits, given the answers collected; the standard interval for a proportion is the Wilson score interval (Wilson, 1927).

Ask what level the range is at (95% is usual), and whether it appears in exports and shared reports, not only in the app. A chart line with no band around it invites decisions on noise.

Good answer: “Every rate carries a 95% range and its answer count, everywhere it appears.”

4. Are competitors compared on the same answers?

Competitors should be measured on exactly the answers you’re measured on: same questions, same engines, same day. Then a gap is a like-for-like difference, and its range can be computed answer by answer; how to compare your AI visibility with competitors fairly works through one.

The same rule applies to change over time: two runs are comparable only on the questions and engines both asked. Is that change real? shows how a six-point rise can be a clear change or noise depending on how answers flipped.

Good answer: “Gaps and changes are compared on the same answers, and called only when their range excludes zero.”

5. Are branded and unbranded questions reported apart?

Branded questions, the ones that name you, should be reported apart from unbranded ones. Answers to “is your brand good?” name you almost by construction, so mixing them into a headline lifts it without saying anything about discovery. Branded vs unbranded questions shows a blended rate moving from 39% to 58% with no change in either underlying rate.

Good answer: “The headline counts only questions that don’t name you; branded questions are a separate view.”

6. Which engines are asked, and how often?

The vendor should name the engines each plan asks and how often runs happen. Buyers use different engines, and engines differ in what they search and cite (how AI answer engines choose which brands to name), so a figure for one engine says little about another.

Ask whether every engine on the plan is asked every run, whether runs happen on a schedule, and whether figures can be split by engine. A split needs enough answers per engine to have its own useful range.

Good answer: “These named engines, on this schedule, with a rate and range for each.”

7. Are the searches and sources behind each answer shown?

The tool should show what the engines searched and which sources they cited, answer by answer. A rate tells you how often you’re named; the searches and sources tell you why, and what to work on.

With web search on, engines run their own searches before answering, a step Google calls query fan-out for its own AI features, and cite the pages they used. Ask whether you can see both, and whether the cited pages are grouped so you can find the third-party sites that decide your category. Those sites are the starting point of how to get your brand named in AI answers.

Good answer: “Each answer keeps the searches the engine reported and the sources it cited.”

8. What does a plan’s unit mean, and what does it cost a month?

A plan’s unit should convert cleanly into answers a month. Vendors price in prompts, questions, credits or answers, and the same word can mean different things. One useful definition: one answer is one question asked to one engine once.

Convert every plan with the same formula: questions × engines × repeats × runs a month. A plan with 50 questions, 4 engines, 2 repeats and weekly runs (4 a month) produces 1,600 answers a month. Then compare price per month with the answers, engines and features you actually need.

Good answer: “A plan includes this many answers a month; one answer is one question to one engine once; the price is published.”

The checklist

Print this page to take the checklist into a demo.

How does AmpliRank answer the eight?

AmpliRank answers each question from its published method and plans:

  1. Answers behind each number: every rate shows the number of answers it comes from.
  2. Repeats: you choose how many times each question is asked in a run; more samples narrow the range.
  3. Ranges: every rate comes with a 95% Wilson interval and the number of answers.
  4. Same answers: competitors are measured on exactly the same answers, and a change or gap is called only when its 95% range, paired over the same questions and engines, excludes zero.
  5. Branded apart: the headline is the share of answers, to questions that don’t mention the brand, that name the brand. Branded questions are shown separately.
  6. Engines and schedule: AmpliRank supports ChatGPT, Claude, Gemini, Perplexity and Google AI Mode. The method page names the engines switched on today, and pricing shows which engines each plan includes. Runs can be scheduled weekly or monthly.
  7. Searches and sources: each answer keeps the searches the engine reported and the sources it cited. Google AI Mode shows its cited sources but not its searches.
  8. Unit and price: plans are sold in answers a month, where one answer is one question asked to one engine once, at prices published on the pricing page.

Where to start

Send the eight questions to every vendor on your shortlist before the demo, and ask for the answers in writing. Score each answer as specific, vague or missing. A tool with a missing answer to questions 1, 3 or 4 can’t tell you whether a change is real. The free AI visibility calculator checks any figure a vendor shows you, if they give you the answer count.

Sources

■ Ask · Measure · Compare · Trace · Fix · Share

Measure what AI says about you.

Cookies