How to Measure AI Visibility

Measure AI visibility by defining the questions and surfaces you will observe, preserving the answers, and labeling mentions, recommendations, citations and factual accuracy separately. Report the denominator for every rate and keep collection failures out of the category of valid answers that omit your brand.

A useful baseline answers a bounded question: how did these systems describe and recommend this product for this question set during this period? It does not directly measure all buyer conversations or establish market share. Your report should preserve that boundary even when the headline result is attractive.

The fictional dataset below makes the calculations inspectable. Its numbers illustrate the method; they are not Jam customer results or industry benchmarks.

Download the illustrative observation data (CSV) and calculation script (Python). The values are fictional; the calculations can be checked locally.

Define the decision the measurement will support

Begin with the work someone will do differently after reading the report. A founder choosing which product page to improve needs different evidence from a communications lead investigating wrong claims or a docs team testing implementation success.

Avoid using one score to stand in for all five. A citation-rich answer can contain an incorrect recommendation. A working integration can occur without a favorable buyer-facing comparison. Measurement becomes more useful when those differences are visible.

Keep the question set explicit

Record the actual questions rather than only topic labels. “Integration infrastructure” is a topic; “Which integration platform supports a small team shipping customer-facing OAuth connectors?” is a question with a task and constraints.

Separate unbranded discovery questions from diagnostics that name your product. The latter are valuable for checking identity and factual accuracy, but they make mentioning the brand much more likely. Combining the groups can make discoverability appear stronger without any change in independent recommendations.

Keep versions of the question set. If you add twenty favorable questions midway through a comparison, the total rate changes partly because the sample changed. Report the original fixed set and the expanded set separately. The buyer-question selection guide explains how to map personas, tasks and constraints before running measurements.

Question weights also require a declared rationale. Equal weighting answers one question about the selected inventory. Weighting by business priority answers a different one. Neither becomes real search demand merely because the result is a percentage.

Define the unit of observation

One observation is one attempt to obtain an answer for one exact question on one recorded surface at one time. Preserve the question, raw answer, attached citations and collection metadata. A repeat run creates another observation; it should not overwrite the first.

Name the surface precisely enough for comparison. A consumer chat interface, a search-enabled API response and a coding-agent session are different experimental settings. Record model information when it is available, but do not invent a version the interface does not expose. Keep relevant country, language and search settings with the observation.

Anthropic's evaluation guidance distinguishes tasks, trials, graders and outcomes. That vocabulary helps avoid conflating an attempted evaluation with a successful result. It does not prescribe a universal number of questions or repetitions for GEO.

Decide in advance how to classify refusals, empty answers and collection errors. A completed answer that says the product cannot be recommended is evidence. A transport timeout is missing evidence. An answer that cannot address the question may need a separate not-assessable status rather than being silently discarded.

Use separate metrics with published definitions

The following definitions are an operational rubric for this guide. Your platform may use different names or denominators, so map them explicitly before comparing numbers.

Count a brand once per answer for answer-level mention rate, even if the answer names it five times. Count an owned citation once per answer for this rate, even if three owned pages appear. Page-level citation frequency is a separate view that can identify which resources contribute evidence.

Peec's documentation describes visibility around mentions, position around rank when appearing, and sentiment around the description's tone. This is a useful example of why metric definitions matter. It does not make mention frequency identical to preference or recommendation.

Work through a small fictional dataset

The ten rows below were created to illustrate difficult labels. A and B are fictional observation surfaces. There are eight unbranded attempts and two questions that explicitly name the product.

Row 2 is a negative comparison: the product is named, but not recommended. Row 3 cites an owned tutorial without naming its publisher in the answer. Row 5 recommends the product using a third-party source. Row 7 describes the product without suggesting it for the task. These are different outcomes, not annotation mistakes.

For unbranded observations, six of eight attempts returned valid answers. Collection completion is therefore 6/8 = 75%. Two failed attempts remain visible; they do not count as answers where the brand was absent.

Among the six valid unbranded answers:

• Mention rate is 4/6 = 66.7%.
• Recommendation rate is 2/6 = 33.3%.
• Owned-source citation rate is 3/6 = 50%.

The two seeded diagnostics are reported separately. If they were mixed into the valid-answer pool, mention rate would become 6/8 = 75%. That is a different measurement, produced partly by questions that already named the company.

The arithmetic and label counts in this example were checked locally. The tiny dataset is not sufficient to infer broad differences between surfaces or estimate population behavior.

Do not hide missingness in a headline percentage

Suppose a second collection returns only the two favorable answers from the fictional unbranded set because all other attempts fail. Its mention rate among valid answers would be 100%, while completion would be 25%. Reporting only the first number would invite the wrong conclusion.

Show valid-answer rates alongside completion counts and failure reasons. If failure is concentrated in a particular surface, region or question type, inspect that pattern before aggregating. Do not assume the missing answers would resemble the observed ones.

If you also choose to report “mentions per scheduled attempt,” label it explicitly. In the original example that value is 4/8 = 50%, but it combines answer behavior with collection availability. It is not interchangeable with 4/6 valid-answer mention rate.

For time comparisons, state whether retries replace failed attempts, supplement them or form a separate attempt history. A successful retry can restore evidence for a scheduled cell, but preserve the earlier failure in the operational record.

Review difficult recommendation and accuracy labels

“You could use Product A, but Product B is better for this requirement” contains both a conditional candidate and a preferred choice. Decide whether your rubric records any recommendation, preferred recommendation, or both. Keep the text that justified the label so another reviewer can challenge it.

Entity matching deserves the same care. A common product name can refer to another company. Domain, product category and context help disambiguate the mention. Mark uncertain matches for review rather than counting every matching string.

Factual accuracy should be evaluated at claim level. An answer can correctly describe deployment and incorrectly describe pricing. Split those claims, record primary evidence, and retain the relevant plan, version and checked date. Do not label every negative description inaccurate.

For automation, calibrate a grader against a human-reviewed sample and examine disagreements. A numerical output from another model is not self-validating. The sentiment and accuracy guide provides a more detailed rubric for those judgments.

Repeat observations without pretending they are independent buyers

Repeated runs can reveal instability: the same question may include a product in one answer and omit it in another. Preserve repetitions and report the distribution for important questions rather than only the most favorable response.

Multiple trials of the same prompt do not become multiple independent buyer journeys. They share the wording, setup and collection period. Avoid a population confidence claim that ignores this clustering or a statistical significance claim based only on raw answer count.

Choose the number of repeats according to the decision and resources available. A first diagnostic may be small; a costly content or positioning decision may deserve broader coverage and more review. There is no universal sample size established by this guide.

When a provider changes or the interface behaves differently, annotate the comparison. Keep the fixed question set where possible, but do not hide a changed measurement surface behind an uninterrupted trend line. Use the cross-platform comparison guide to make those boundaries explicit.

Connect visibility to business outcomes carefully

Answer visibility, referral traffic, qualified leads and successful integrations belong in the same decision process, but they are not the same metric. A cited answer may produce no click. A referral may be poorly qualified. A buyer may encounter the product through several channels before converting.

Use site analytics and customer research to add context without assigning every conversion to the last observed citation. If a user reports finding the product in an AI answer, preserve that evidence and its limits. A survey response and a measured referrer each illuminate different parts of the journey.

DX's measurement framework separates utilization, impact and cost in engineering productivity. The useful transferable idea is to keep activity, outcome and expenditure distinct. DX's engineering metrics are not evidence for GEO effectiveness or a source of expected citation uplift.

After a page change, compare equivalent observations and record the intervention date. Other content, retrieval behavior and system changes may also influence the result. The content-change measurement guide explains how to report a useful before/after result without overstating causality.

Publish a baseline someone else can inspect

A compact baseline report should include the decision it serves, question-set version, collection dates, surfaces, attempts and valid answers, failure breakdown, labeling definitions, rates with counts, and representative evidence. Add an explicit list of limitations and unresolved labels.

End with the actual next work: inspect a recurring wrong claim, improve a missing explanation, investigate a highly cited comparison, or collect more evidence before changing anything. A dashboard that cannot connect a headline score to an inspectable answer is less useful than a modest report that can.

Is a higher mention rate always better?

No. Mentions can be negative, inaccurate or irrelevant to the buyer's task. Read the framing and recommendation labels, then inspect claim accuracy. A single volume metric can conceal a worsening description.

Can we compare two vendors' visibility scores directly?

Only after understanding their question sets, surfaces, collection methods, definitions and denominators. Matching labels do not establish matching measurements. Use the same evidence sample to discover where their interpretations differ.

What should we track first?

Begin with a manageable set of important unbranded buyer questions, separate named-brand diagnostics, and enough preserved evidence to review every surprising result. Expand coverage when you can explain and act on what the current baseline shows.

Source references

The following sources were checked September 17, 2026. Illustrative examples and proposed procedures are identified separately in the article.

• Anthropic's evaluation guidance. Checked September 17, 2026.
• Peec's documentation. Checked September 17, 2026.
• DX's measurement framework. Checked September 17, 2026.

Continue reading

• Choose Buyer Questions for Your GEO Program
• Compare Visibility Across AI Answer Platforms
• Measure Whether a GEO Content Change Helped

Explore GEO with Jam

See how Jam approaches AI visibility research and content improvements for developer-tool teams.

Explore Jam for GEO