Which questions should you use to monitor visibility in AI?

Measuring visibility in AI system answers means, first of all, deciding what to ask. The answer changes with the question: if the sample is made up of questions no customer would ever ask, the result will be precise and useless. If it changes every month, there will be nothing to compare against anything.

The sample is therefore the measurement tool, and it needs to be built with the same care as a questionnaire: representative of the needs of the customers you want, balanced, and stable enough to allow comparisons over time. This guide helps you put it together and decide the rules for changing it.

Three families of questions, not one

A sample made up of only one type of question measures only one thing. The three families answer to different moments and need to be kept distinct in reading the results too.

• Informational. Whoever asks these is still figuring out the problem: what it is, when it's needed, how complicated it is. Appearing here doesn't bring customers tomorrow, but it's the moment when opinions are formed.
• Comparative. Whoever asks these is weighing two paths: do it or not, one solution or another, in-house or outsourced. These are the questions where an honest answer about limits is worth more than an enthusiastic description.
• Vendor-choice. Whoever asks these is looking for who to turn to, often with a constraint of area or industry. These are the closest to the decision and usually the least numerous.

The proportion between the three isn't the same for everyone: it depends on how long your sales cycle is. A service that's bought after months of evaluation needs more comparative questions; a job that's searched for only when it's needed requires fewer.

Branded and unbranded are two different measures

Questions that contain your company's name measure what the system knows about you: whether it knows you, whether it describes you well, whether it confuses your name with someone else's. Unbranded questions measure something entirely different: whether you get named when someone is looking for a solution without having you in mind.

• Keep them in two separate groups and never average them together. An improvement on the first says nothing about the second.
• Branded questions also serve as a quality check: they verify that what's said about you is correct, not just that it's present.
• Unbranded questions are the measure that matters for growth, and they're also the ones where results move more slowly.
• Watch out for ambiguous names: if your company shares its name with a common word or with another company, you need questions phrased to tell them apart, or you'll just be measuring noise.

A sample set of questions

Here's what a small excerpt looks like, for a company that does equipment maintenance. The questions are illustrative and meant to show the criterion, not to be copied.

Next to each question, write down why it was included. It's the column that, six months from now, lets you understand whether it still makes sense to keep it, and it stops the sample from filling up with questions whose origin nobody remembers.

Balancing services, territory, and the moment of decision

The most common risk is a sample that's unbalanced without anyone noticing: twelve questions about the service the owner cares about, and none about the other two, which bring in the same amount of work.

• A minimum number of questions for every service you want to be found for, even the ones that are less interesting to talk about.
• Territory only where you genuinely operate. Adding provinces you don't cover inflates the sample and produces negative results that don't mean anything.
• Questions should be written the way a customer would say them, not the way you would write them. Internal technical vocabulary rarely matches that of whoever is searching.
• If a service is new and nobody is searching for it yet, keep it in a separate group: measuring visibility on a question nobody asks isn't a goal.

How big it should be, and when the results say nothing

The right size isn't a fixed number: it's whatever you can actually reread. A sample needs to be looked at one answer at a time, at least now and then, because the count of citations doesn't tell you whether you were named well or badly. If nobody has time to reread it, the sample is too big.

Version the sample, and give a reason for every change

A sample that changes silently destroys the ability to compare: it looks like an improvement, but really the questions have changed. The discipline costs little and comes down to a few rules.

1. The sample has a version and a date. Each measurement notes which version it was taken on.
2. Every change has a written reason: a new service, an area you've stopped operating in, a question that turned out to be ambiguous.
3. Removed questions aren't deleted, they're archived with the date. This lets you reread the historical record without getting confused.
4. Changes are grouped into a few moments a year, instead of adding a question every time someone has an idea for one.
5. When the sample changes significantly, comparisons with the past are made only on the questions that remained identical.

What this guide doesn't cover

This guide covers building the measurement tool: which questions to ask and how to keep them stable. How answers are collected and evaluated — how often to measure, what to count, how to distinguish a favorable citation from any old mention, how to read a trend — is later, separate work, covered elsewhere. The choice of topics to write about, which often springs from these same questions, also follows its own criteria.