How to Run Prompt-Level SEO Experiments to Diagnose Your AI Search Visibility in 2026

What Are Prompt-Level SEO Experiments?

Prompt-level SEO experiments are structured tests where you treat individual AI prompts the way an SEO would treat individual keyword queries. Instead of asking “are we more visible in AI search this month?” you ask “did this specific change to this specific page move our inclusion rate on these specific prompts?” That shift from broad question to narrow, testable claim is most of what makes the methodology work.

Jason Tabeling's Search Engine Land piece frames this with an “if, then, because” structure that is genuinely useful for keeping experiments honest. If we add FAQ schema to pages that already have Q&A sections in their HTML, then we should see those sections more often included in ChatGPT responses, because explicit schema markup makes the structure easier for the LLM to ingest. Each clause does work: “if” forces you to define the change, “then” forces a measurable prediction, and “because” forces you to articulate why it should work — which is what stops you from running tests with no theory and finding random results.

The practical scope is narrower than enterprise GEO tools. You are not trying to measure your share of voice across 500 prompts in real time; you are trying to answer a single question about a single change to a single page or content block. That is the right altitude for a small business. We've made the broader case for narrow measurement in why prompt volume is the wrong GEO metric in 2026 — tracking 500 prompts where your brand “appears” is a vanity metric. Running 10 prompts where you can prove a change moved the needle is a real signal.

The metric layer matters too. Search Engine Land's GEO metrics roundup lists eight things worth tracking — citation frequency, share of model voice, answer inclusion rate, entity recognition, sentiment, prompt coverage, retrieval success, and downstream conversion influence — and prompt-level experiments are how you actually move the first three. The diagnostic value comes from pairing this with a structural understanding of where AI search visibility breaks down. Our 10-gate AI search pipeline diagnostic maps the failure points — crawl, parse, ingestion, retrieval, ranking, citation. Prompt-level experiments are how you actually test which gate a specific page is failing at. The pipeline tells you where to look; the experiment tells you whether your fix worked.

Why Run Experiments Instead of Just Watching Visibility Tools?

Three reasons we keep coming back to. The first is that LLM outputs are noisy in ways that watching dashboards hides. Tabeling's piece names this directly: “prompt drift” means the same prompt yields different results across days, even within hours. A daily monitoring tool that screenshots one response per prompt per day is averaging out variance you should know about. Run the same prompt three times in a row and you may get three meaningfully different answers — a brand cited once, position three the next time, and not mentioned at all on the third. If your visibility tool only saw the first one, you would draw a wrong conclusion.

Otterly.ai's published methodology tracks citations and brand mentions across ChatGPT, Perplexity, Google AI Overviews, Google AI Mode, Gemini, and Copilot, and one of their large-scale studies analyzed “1+ million AI citations across ChatGPT, Perplexity, and Google AI Overviews from January-February 2026.” That kind of sample size beats noise by sheer volume. Most small businesses cannot run that volume, so they have to beat noise the other way — by repetition on a small set of prompts.

The second reason is causal isolation. Visibility tools tell you that your AI inclusion rate went up or down, but they cannot tell you why because too many things change between snapshots. The website was updated. The model version changed. Your competitors published new content. A news event shifted the answer space. Prompt-level experiments hold most of those variables constant by design — same prompts, same model, same week — so a change in inclusion rate has a much shorter list of plausible causes.

The third reason is cost. Otterly.ai, Profound, and Peec are real products that real teams use; we have used them. They start at hundreds of dollars per month and scale with prompt volume. For a Fort Wayne dentist or a regional manufacturing company, that is a hard line item. A prompt-level experiment notebook and an hour a week is a $0 start, and it gets you the answers that matter most for content decisions. We make the case for cost-aware AI tooling in our piece on AEO tools for Fort Wayne small businesses, and the experiment methodology is the free baseline that decides whether you ever need the paid tools at all.

How Do You Design a Prompt-Level Experiment That Actually Works?

The standard structure that holds up across the methodologies I have read is straightforward and worth running through every time:

1. Pick a single hypothesis. Use the if/then/because frame. Not “improve our AI visibility” — that's not testable. “If we add a 60-word direct-answer paragraph at the top of /service/hvac-tune-up, then our inclusion rate on the prompt ‘best HVAC tune-up service Fort Wayne’ will rise from baseline, because LLMs preferentially extract content from the top of pages.”
2. Choose 5–10 target prompts. Tabeling's guidance: “Execute a set of 5-10 target prompts daily for seven consecutive days.” Pick prompts that a real customer would actually use, not the ones you wish they would. Mix exact-intent prompts (“HVAC tune-up Fort Wayne”) with broader research prompts (“how often should I service my furnace in Indiana”) and at least one comparison prompt (“HVAC companies in Fort Wayne”).
3. Lock the model and version. Tabeling is explicit: record the “specific model and version used for testing.” Models are updated constantly and a version change between baseline and measurement is a fatal flaw. If you are testing on ChatGPT, write down whether it was GPT-5.1, the date, the time, and whether you were logged in. The same applies to Claude, Perplexity, or Google AI Mode.
4. Define the testing environment. Cleared browser cache, no login state where possible, same device, same IP location range. Log everything. The whole methodology is built on holding things constant; one variable change invalidates the comparison.
5. Run baseline for seven consecutive days. 5 prompts × 7 days = 35 observations. 10 prompts × 7 days = 70 observations. Each observation is one run of one prompt. Capture the entire response, the citation list if visible, and a position-in-response number for any mention of your brand.
6. Implement one change. One. The “single-paragraph swap” methodology Tabeling describes — modifying only one targeted text element — is what makes the result interpretable. If you simultaneously add schema, rewrite an H2, and update internal links, you cannot tell which change moved the needle.
7. Wait for re-indexing. This is the step most small businesses skip. AI search engines do not re-ingest your page the moment you publish; depending on the engine, the lag can be days to weeks. Tabeling's protocol assumes a meaningful gap between change and re-measurement. We typically wait 7–14 days.
8. Run the measurement phase. Same 5–10 prompts, same model and version (or as close as you can get), same environment, seven consecutive days.
9. Compare averages, not single observations. Average inclusion rate baseline vs. measurement. Average position-in-response baseline vs. measurement. Sentiment shifts if you tracked them. Single-day swings tell you nothing.

Two specific traps to avoid. First, don't use prompts you control the answer to (your brand name alone). Those are always going to cite you and don't measure anything useful. Second, don't pick prompts where you already rank #1 on Google — the AirOps study covered by Search Engine Land found pages in the top organic position were cited 58.4% of the time versus 14.2% for position ten, so you have very little headroom on prompts where you are already top-ranked. Pick prompts where you are mid-pack and citations are genuinely at stake.

What Should You Capture and How Long Should You Run It?

A small spreadsheet is enough infrastructure. The columns we use:

That is roughly 9 columns. A spreadsheet of 70 rows (10 prompts × 7 days) takes about 60–90 minutes per phase to fill out by hand and is genuinely usable. Tabeling's guidance also recommends maintaining “an organized, time-stamped repository of the exact prompt queries used for baseline and measurement phases” with inclusion rate, position-in-response, and sentiment/framing data — that is what the spreadsheet is.

For statistical caution, two honest notes. With 35–70 observations per condition, you can reliably detect large effects (inclusion rate moving from 20% to 50%) but not small ones (20% to 28%). If your measurement-phase inclusion rate is just a few percentage points above baseline, the change might be drift, not a real effect. The fix is repetition — run the same experiment again the following month. If it replicates, the effect is real; if it doesn't, it was noise.

You also cannot directly compare experiments run on different models. ChatGPT inclusion rates and Perplexity inclusion rates measure different things on different retrieval pipelines; treating them as the same metric is a common rookie mistake. If you want a multi-engine view, run parallel experiments — same change, separate baseline and measurement on each engine — and report each separately.

How Do You Read the Results Without Fooling Yourself?

Three honest interpretation rules. First, look at the change in inclusion rate, not the absolute level. If your baseline was 18% inclusion and your measurement was 27%, that's a real signal. If your baseline was 18% inclusion and your measurement was 19%, that's noise.

Second, separate inclusion from position. A prompt where you used to be cited in position 4 and are now cited in position 1 is a meaningful win even if your raw inclusion rate didn't move. Conversely, an inclusion-rate gain on prompts where you are buried at position 8 is worth less than it looks.

Third, look for consistency across the 5–10 prompts in your set. A change that moves three out of ten prompts and barely touches the others is real but narrow — your improvement applies to the kind of query that resembles those three prompts, not to the whole topic. A change that moves seven out of ten in the same direction is broad.

There is a useful structural reminder from Search Engine Land's recent piece on intent alignment vs. technical SEO: once a site has reached technical parity with competitors, the next gain comes from intent alignment, not more technical work. The author quotes the threshold directly — “once a site reaches technical parity with its competitors — the point at which a proper infrastructure no longer gives you an advantage — Google shifts its ranking criteria toward relevance.” The same logic applies inside AI search. Most prompt-level experiments where the result is “no change” are really telling you the issue is intent — your content is technically fine, but it is answering a slightly different question than the prompt. The fix in those cases is rarely more schema; it is rewriting the answer to match the prompt's actual ask. We covered the broader Google Search Console version of this in intent gap analysis with Google Search Console.

For prompts where you do see a clear lift, the next move is to figure out why so you can replicate it. A “changed nothing structural, just rewrote the first paragraph as a direct answer, gained 14 points of inclusion rate” result is actionable across your whole site. A “added FAQ schema and the result is unclear” might warrant another month of testing on a different page.

What Does This Look Like for a Fort Wayne Small Business?

Concretely, here's how a typical Fort Wayne service business — say, a residential plumber serving Allen and DeKalb counties — would run their first experiment. This is illustrative, not a case study from a specific client.

The hypothesis. Adding a 70-word direct-answer paragraph at the top of /services/water-heater-repair will increase inclusion rate on prompts about water heater repair in Northeast Indiana, because LLMs disproportionately extract content from the first 200 words of a service page.

The 7 prompts.

1. Water heater repair Fort Wayne
2. How much does water heater repair cost in Fort Wayne
3. Best plumber for water heater repair in Allen County
4. How long does water heater replacement take
5. Plumbing companies near Auburn Indiana water heater
6. When should I replace vs repair my water heater
7. Same-day water heater repair Northeast Indiana

The model. ChatGPT GPT-5.1, logged out, Chrome incognito, run between 10am–11am EDT each day, cleared cache between runs.

The baseline. 7 days × 7 prompts = 49 observations. The plumber records: cited or not, citation position, which competitors were named.

The change. A single 70-word direct-answer paragraph at the top of the service page that names a price range, the typical timeline, and the service area in plain prose. No schema added, no other content changed.

The wait. 10 days for re-indexing.

The measurement. Same 7 prompts × 7 days = another 49 observations.

The honest read. Suppose baseline inclusion was 12 of 49 (24%) and measurement was 19 of 49 (39%). That's a 15-point gain across 49 observations — a meaningful signal, especially if the gain is concentrated on prompts 1, 3, and 7 (the explicit-intent prompts). If the same change shows no movement on the research prompts (4 and 6), that tells the plumber what kind of content to add next: research-stage content with explanatory paragraphs, not just service-page direct answers.

This is roughly the workflow we walk Fort Wayne and Auburn clients through, paired with the broader competitive context of a Fort Wayne AI competitor analysis so the client knows whether their competitors have been moving on the same prompts. The hyper-local angle matters because the competitive landscape on a Fort Wayne service prompt is much narrower than the landscape on a national prompt — meaningful gains are easier to detect because there are fewer brands fighting for citation slots.

Need Help Designing Your First Prompt-Level Experiment?

The methodology in this guide is intentionally do-it-yourself. A small business can absolutely run their first three experiments without paid tools or outside help. We've laid out the structure that way on purpose, because we'd rather see ten Fort Wayne businesses run honest small experiments than two of them paying for enterprise dashboards they don't have time to read.

Where we do help is when the experiments stop telling a clear story — when results are mixed, when sample sizes need to grow, or when an experiment surfaces a deeper content or positioning issue that needs structural work. Our AEO services cover that next layer: experiment design support, results interpretation, and the content rewrites that follow when an experiment reveals an intent gap. If you want to start with a free 30-minute review of your current AI visibility, contact us — we'll send back a one-page recommendation on the single experiment we'd run first.

For broader context, our answer engine optimization guide covers the full AEO stack, and our 10-gate AI search pipeline diagnostic is the structural complement to the experiment methodology in this guide — together they cover where to look and how to test.

Sources & Further Reading

1. Search Engine Land: searchengineland.com/prompt-level-seo-experiments-ai-search-476813 — Prompt-level SEO: How to run experiments that move AI search visibility.
2. Search Engine Land: searchengineland.com/geo-metrics-to-track-476642 — 8 GEO metrics to track in 2026.
3. Search Engine Land: searchengineland.com/chatgpt-citations-ranking-precision-length-study-474538 — ChatGPT citations favor ranking and precision over length.
4. Search Engine Land: searchengineland.com/intent-alignment-technical-seo-476823 — Intent alignment beats technical SEO once parity is reached.
5. Otterly.ai: otterly.ai/blog/llm-monitoring-methodology — Otterly.ai LLM visibility monitoring methodology.