Illustration by Raffaele Mainella for <em>Nos Invisibles</em> (1907). Public domain, via The Public Domain Review.
View contents
I keep getting the same underlying question from clients: “Do I show up when someone asks ChatGPT or Perplexity about what I offer?” Why now? If you work in tech, you probably already sense the answer, but it’s worth looking at the numbers. According to OpenAI’s report on ChatGPT usage in the first quarter of 2026, several Latin American countries (among them the Dominican Republic, Mexico, Brazil and Costa Rica) are among the ten that climbed the most in messages per capita, with the Dominican Republic moving up nine spots in the global ranking. And a January 2026 report from McKinsey and the World Economic Forum estimates that AI could add between 1.9% and 2.3% in annual productivity to the region. These numbers don’t say your brand needs to appear in AI answers (those are separate things), but they do describe the ground: more and more people ask an assistant before they ask a search engine, and that’s where the question of whether you’re in that conversation comes from.
The problem shows up when you try to measure it. The natural reflex is to open analytics and look for the number, and that’s a good instinct. But two problems open up here. The first is more basic than it seems: some companies don’t even have Search Console or any analytics tool set up, so there’s nowhere to look. And the second, for those who do have it: there’s no “Search Console for AI.” Often the metric isn’t there, the traffic source arrives unidentified, or the assistant simply answers without anyone clicking through to your site. In my experience, it’s one of the first frustrations anyone hits when they want to review their users’ data and understand where they come from.
In fact, at UXR we’ve already added an AI visibility section to our client dashboard, to track this specifically, and it seemed useful to organize what the literature and other colleagues around the world have been saying about how to do it well. This is in no way meant to be exhaustive, not at all. It’s a topic I find genuinely interesting and I hope to keep learning and writing much more about it, but I’m sharing one of my most recent takeaways.
Why a single response doesn’t work
The temptation is to open ChatGPT, type the question, check whether your brand appears, and draw a conclusion. The problem is that this single response means almost nothing.
Language models generate text by sampling from a probability distribution: at each step they don’t pick “the” right word, they draw among the possible ones according to their probability. That’s why the same question, asked twice, can return your brand once and omit it the next time. It’s not that the model contradicts itself: it’s how it works. One detail I found surprising: this variation doesn’t fully disappear even when you set the model’s “temperature” to zero. Research on LLM reproducibility (several papers available on arXiv, listed in the references) shows that floating-point arithmetic and the order of operations on the hardware can alter the result even in theoretically deterministic setups. Anthropic itself recommends sampling multiple times to check output consistency.
The practical consequence is very clear: drawing a conclusion from a single response is like estimating a survey by asking one person.
What to do, based on what we have so far
I have good news: you don’t need a huge sample or expensive tools. With a bit of method and basic statistics, a small number of runs is enough for a reasonable estimate. This is what I’ve been applying, leaning mainly on Graphite’s work (“Demystifying Randomness in AI,” 2026) and on classic proportion statistics.
Run each query at least 10 times. That’s the threshold where the error becomes manageable. According to Graphite’s data, with 10 responses per query the mean absolute error in the visibility estimate sits around 5.6%, and more than 98% of queries land with an error of 10 points or less. You can go higher, but there are diminishing returns: estimates from 10 responses aren’t very different from those you get with 200. One operational detail that helps: use a fresh chat for each run and turn off memory, so one response doesn’t carry over into the next.
Measure visibility first, position second. It helps to separate two things. The first is binary: does your brand appear or not? The second, finer one, is where it appears relative to others. Position only becomes meaningful once visibility is already high: if your brand appears in 2 out of 30 responses, its “average position” is computed over so few cases that it’s mostly noise. If you’re starting out and don’t yet appear consistently, visibility is the metric that matters.
Use the Wilson interval for the margin of error. Since visibility is a proportion (appearances over total runs), it carries an associated uncertainty worth stating. The Wilson confidence interval (proposed by Edwin B. Wilson in 1927 and still recommended as a default for proportions) works well precisely when the sample is small or the proportion is close to 0% or 100%, which is where the classic normal approximation fails. To make it concrete: if your brand appears in 5 of 10 responses, the 95% Wilson interval runs roughly from 24% to 76%. In other words, with 10 runs you can claim “appears / barely appears,” but you can’t confidently tell a 50% from a 60%. That takes a larger sample.
Measure on a cadence, weekly or biweekly. Because there’s baseline noise, looking at a single measurement or measuring too often leads you to mistake normal variation for a trend. A regular cadence, read alongside the confidence intervals, lets you separate signal from noise and see whether a content change actually moved the needle.
A case: visibility can be zero
A result that comes up more often than you’d expect is the absolute zero. In a recent diagnostic for a project in the United States, we measured the brand’s presence across three language models (LLMs), ten queries each, derived from an analysis of its competitors: zero mentions out of thirty. The rest of the direct competitors were equally absent, too: a few large platforms showed up, but no player comparable to the project.
A clean zero like that is more informative than it looks. There’s a bound (the “rule of three”) stating that, when you observe zero appearances in n attempts, the upper limit of the 95% confidence interval is around 3/n. With 30 runs, that puts real visibility below roughly 10%. Put another way: if the brand had meaningful presence, we’d very likely have seen at least one appearance. We saw none. And there’s a hidden advantage: a zero is an excellent baseline. Any future appearance, after a content intervention, becomes a clear and easy-to-defend signal.
I plan to write separately about this case and the full diagnostic it came from, because AI visibility was just one of several levers there, and not even the main one.
There’s much more to this, and on a good part of it we’re still learning in practice and reading others. Open questions remain: how to choose the query set well so it reflects how people actually search, how to handle responses with web search enabled, how to compare across models. Each is worth its own post. If you measure AI visibility a different way, or if you see something I’m missing here, I’d love to hear from you.
References
- Druck, G. & Smith, E. (2026). Demystifying Randomness in AI. Graphite.io. [Agency report; primary methodological basis for sample size and visibility tracking]. https://graphite.io/five-percent/demystifying-randomness-in-ai
- Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22(158), 209-212. [Primary source for the Wilson confidence interval].
- OpenAI (2026). Report on ChatGPT usage, first quarter of 2026. [Latin America adoption data; cited via El Cronista, July 1, 2026].
- McKinsey & World Economic Forum (2026). From Potential to Productivity: Latin America in the Intelligent Age. [Estimate of AI’s impact on regional productivity].
- On LLM non-determinism (reproducibility, sampling and temperature): a set of works available on arXiv, including “Non-Determinism of Deterministic LLM Settings” (arXiv:2408.04667) and “Defeating Nondeterminism in LLM Inference” (Thinking Machines Lab, 2025). [Technical basis for why responses vary].
A note on sources: Graphite’s work is an agency report, not a peer-reviewed paper; I use it for its transparent methodology and authors with a research background. The Wilson statistics and the non-determinism works are primary and academic sources.
Paulina Contreras · UXR SpA
