How we
measure.

1What we check in your store's data

The things any good checker reads, with every product counted for Shopify Catalog, not a sample.

• Every product, against Shopify Catalog's requirements. Shopify Catalog leaves out a product that has no category, no image or a variant priced at 0, or that isn't published and searchable. We count those across the whole catalog. For a larger store, Shopify runs one bulk query over every product, at most once a day, and we keep only the counts. A store with up to 50 products is checked directly.
• What helps an agent choose. On your 50 most recently updated products: a GTIN or barcode, a description of 40 words or more, a product type, image alt text, an SEO description and a SKU.
• Your store's facts. Your return window, the order amount for free shipping, and the other delivery and return facts you confirm in the app. We judge answers against what you confirmed. We don't guess.
• Your policies. Whether your refund and shipping policies are there in words an agent can read: Shopify draws on them when it answers agents' questions. And whether you have terms of service and a privacy policy: Shopify's AI channels take a store only with those two and a refund policy.
• Agent data on product pages. Whether Emtor's theme embed is on, with its product data not set to Off, and whether what it adds reaches your products. By default it adds the shipping and returns you confirm, and each product's rating, to the product data your theme prints as structured data, as Shopify's own themes like Dawn and Horizon do, and your store's name, logo and policy links to every page. Readers that join that data into one product get the additions; one that reads each block on its own doesn't. Many themes print their product data without the @id the additions go under, so the agent test reads one of your product pages to see whether they attach. Until a test has, the line gets no points. Where they attach to nothing it gets none either, and the app asks you to choose Full product data, which prints the whole product instead.
• AI crawlers. Whether your robots.txt lets 12 AI crawlers read your product pages: 9 that answer shoppers (Google-Extended among them: it decides whether Gemini may answer from your pages) and 3 that collect pages for model training. We read it by the rules of the robots.txt standard, RFC 9309. Keeping training crawlers out is your choice, not a fault. A firewall can still turn a crawler away, and the file doesn't show that: when you ask for a live check, we ask for your home page and one published product's page from our server, once with each crawler's name and once with ours, and show what came back. A crawler turned away there is a sign, not proof, since real crawlers come from their own networks.

2What the agent tries

It asks your store what a shopper would ask:

What is your return policy? How many days do I have to return an item, and who pays for return shipping?

How much does shipping cost to [your country], is there free shipping over a certain amount, and how long does delivery take?

It searches by the kind of product first, then by the name of a product your admin says is in stock, and opens up to 6 results until one has a variant it can add. It also asks for that product's page on your site and records whether a bot check or a block turned it away, and when the page opens, reads its structured data: how many Product entries it has for the product, the price, currency, availability and rating each states, whether they agree with each other, and whether their prices and currency agree with Shopify's own data for the product. The checkout gets a made-up shopper: an example.com email, a made-up street in a real city so your store can quote delivery, and a phone number from a range set aside for fiction, where the country has one. Nothing ships. If your store hands the checkout over to the shopper with a link, the agent opens that link and records what it shows. After the steps, it reads your store's profile for agents, agents.md and llms.txt at your web address, once each and without following redirects, and checks the profile against what UCP requires of one. The receipt shows these, and they add nothing to the score.

The first test starts by itself the first time you open Emtor. After that it runs when you ask, and every 24 hours once you switch on the daily test. Every run is recorded, even one that failed or didn't finish, so a missed day never looks like a quiet one. When a scheduled run finds a new problem, it runs again 2 minutes later, and the newer run decides.

3The agent is signed, and it never pays

• Signed, not anonymous. The agent signs its calls to your store's agent interface with HTTP Message Signatures, RFC 9421, using an ECDSA P-256 key. Its public key is in the agent's published profile, so your store can tell a request came from Emtor and wasn't changed on the way. Shopify checks these signatures: on our test store, a request changed after signing was refused. If a store refuses our signature, the agent asks again unsigned, records why, and doesn't count it against the store. Every run records whether it went signed.
• It never pays. The agent never calls the step that completes a purchase. It creates a checkout, picks a delivery option and cancels the checkout, and the cancel counts only when your store answers “canceled”. So the last question, whether an agent can pay you, stays “not tested”. We never report it as passed.
• Only your own store. The test runs only for a store whose owner installed Emtor, against that store's own myshopify.com address. It talks to your store and to Shopify's checkout, and nowhere else. Installing the app is the permission.

4How we ask AI assistants

We ask ChatGPT, Perplexity and Claude two kinds of question, set out in the next section, and keep every answer.

• More than one answer. Each question goes to each assistant 3 times, because one answer proves little. A call that brings no answer isn't counted, so a failed call can't pass for “not mentioned”.
• As a shopper in your country. The assistants search as if from the country in your confirmed facts. Without one, the fact questions use the US and the shopper questions a country you ship to. Claude is told the country only for the US: Anthropic's search refuses codes it doesn't take, and on 6 November 2026 it refused Lithuania's, so for any other country Claude searches without one.
• When you ask, within a cap. You start a check with a button, at most once a day. Every answer is paid for, so each store and the whole app have a spending cap over the last 30 days, and a check stops before a call that wouldn't fit, so it can end with fewer answers. There's no automatic schedule yet, only the one question asked again after a fix, below.
• Once more after a fix. About 24 hours after a policy fix for a fact an assistant got wrong or didn't give, the question about that fact is asked again by itself, once, within the same cap, and the answers are shown beside the ones from before the fix. A day, because assistants answer from their own search indexes, which take a while to read a change. It is a check after a fix: it doesn't hold your own next check back.
• Named and dated. Every result names the assistant, the model, the country and the date. We keep each answer's text and the links it cited.
• Collection method and terms. We ask through the assistants' developer APIs and don't scrape their apps. Gemini isn't asked: its API terms don't allow storing and analysing answers grounded in search, and keeping the answers is the point.

5Questions with your name, and without it

A question that names your store puts you in the answer before the assistant looks anything up. Ask “What is [your store]'s return window?” and the store gets mentioned almost every time, by construction. A visibility number built from questions like that measures the question, not you.

So we ask two kinds of question, keep them apart, and never blend them into one number:

Both kinds are built into Emtor. Both are on the app's AI answers page, each with its own button.

6How answers are judged

Fixed rules read each answer first, for every figure it gives for the fact: a number of days or an amount. They understand numbers written as words, like “thirty days” or “seventy-five dollars”, durations in weeks or months, currencies, and thresholds like “free over $75”. When an answer gives exactly one figure, plainly, the rules decide. When it gives several, a range, “up to”, a condition, a hedge or a figure it denies, Claude (anthropic/claude-haiku-5.5) judges it through Anthropic's API, via OpenRouter and without web search, and has to quote the answer's own words that state the figure. Its verdict counts only if that quote is in the answer word for word and its figure fits the verdict. Figures are read in English, German, French, Spanish, Italian, Dutch, Portuguese and Lithuanian; outside English only Claude can call one right. Each answer gets one of five verdicts:

Right

It gives your fact as what applies.

Wrong

It gives a different value, or denies yours.

Doesn't know

It says it can't tell.

Not mentioned

It says nothing about it.

Unclear

Several values, a range or a condition that can't be settled, a judgement its quote didn't bear out, or an answer that couldn't be judged. It never counts as right.

The rules are tested against a set of phrasings, including ones that must not match, such as “$175” when your fact is $75, and the whole judge against a set of tricky answers, such as a current figure next to an older one. A phrasing or an answer it gets wrong on a real store goes into those sets. The app shows each verdict with the answer's words it rests on.

Answers to shopper questions are read by fixed rules too: for your store's name or domain, in the text or in a cited link, and for your place in the list the answer gives. A mention in running text has no place, and we don't guess one. Your store's own answers to the agent are read by the fixed rules alone. The purchase test then sums up in four answers: finds your products, gets answers from your store, prepares a checkout, and pays. The last one is always “not tested”. When Shopify itself doesn't respond, with a server error, a rate limit or a timeout, the verdict is “couldn't check”, not a failure of your store.

7What “2 of 3” and the interval mean

We report counts, not percentages. “Right in 2 of 3” means 3 answers came back and 2 stated your fact. “Wrong in 1 of 3” is the factual inaccuracy rate for that fact and that assistant: answers that state a different value, out of all answers.

With so few answers, chance plays a big part. So for every count we work out a 95% interval (Wilson) for the share of right answers: the range the assistant's true rate plausibly lies in.

For shopper questions the count is the answers that name you, with the same kind of interval. So 3 of 3 is a good sign, not a guarantee: the true rate could still be under half. Three answers make a check an alarm for big changes, not proof of small ones. When a call brings no answer, the count is out of fewer and the range is wider. Answers to the same question asked at the same time aren't fully independent either, so the real range is wider still. More answers, on more days, make it narrower.

8What the score is made of

The Agent-ready score is out of 100. Most of it comes from what the agent found when it tried to buy.

A line of the agent test that partly passes gets half its points, and a line the test couldn't check gets none until a later run checks it. There's no score before the first agent test, because its biggest part would be a guess. Payment isn't in the score, because it's never tested. AI answers aren't in it either: they're reported on their own, as counts. Every line says whether it comes from the agent or from your data, so a data check never passes for a live result.

9How a fix is proven

• Before and after, with the same test. After a policy fix, the agent test runs again by itself, 30 minutes later and again 24 hours later, so you see before and after without asking. How long Shopify takes to update what it tells agents after a policy edit isn't measured yet, which is why there are two re-checks.
• Read back, not assumed. After every change, the app reads the value back from Shopify and calls the fix done only if Shopify saved it.
• Unfixed facts are the control. Compare the fact you fixed with the one you didn't. If only the fixed one changed, that points to the fix. If both moved, something else changed, such as the assistant itself. With 2 facts and 3 answers each, that's a signal, not proof.
• Undo is the on/off check. Every fix can be undone for 30 days. Undo it, test again, and the old answer should come back. A result that turns off and on with the fix is the clearest proof a single store can give. If what a fix wrote has changed since, whether you changed it or Shopify did, the fix shows as changed, and undo isn't offered.

10What we don't claim

• No revenue multipliers. We don't promise a multiple of your AI revenue, and we won't show one store's before and after as proof that we caused it.
• API answers stand in for the apps, and they aren't the same. A shopper in the ChatGPT app may be logged in, have memory on and be somewhere else, and can get a different answer than our API call. The apps can also answer with models other than the ones named above.
• One store can't prove revenue impact. It has too few orders, and too much else changes at the same time: prices, ads, the season. Showing that agent-readiness raises revenue takes many stores and a control group. Until we have that, we don't claim it.
• A pass isn't a promise. Shopify decides what Shopify Catalog shows. A category helps, but it doesn't guarantee that a product gets in.
• Outside the US, the test address isn't proven yet. The agent has a test address for 13 countries. So far only the US one has been tested against a store.

11Data we read, and what we never read

• The app may read your products (read_products), your store's policies (read_legal_policies) and your theme's settings, to see whether our embed is on (read_themes).
• It writes to your store in two ways. The facts you save go into the app's own metafield in your store, which our embed shows to agents on your product pages. And a fix, once you press it and approve Shopify's request the first time, changes your policies (write_legal_policies) or your products (write_products).
• It never reads your customer records. It doesn't ask Shopify for access to customers, so it can't see their names, emails, phone numbers or addresses. It reads your orders only if you allow it, when you open its AI sales page and approve Shopify's request (read_orders). Emtor reads order totals, dates and where the shopper came from, never names, emails or addresses, and keeps none of it.
• What leaves your store. For the fact questions, your store's name and domain go inside the questions through OpenRouter: the ChatGPT questions only to OpenAI, the Perplexity questions only to Perplexity and the Claude questions only to Anthropic. When the rules can't settle an answer to a fact question, that answer, its question, the fact you confirmed and the shopper's country go through OpenRouter only to Anthropic, for Claude to judge without web search. The shopper questions carry only a kind of product, a price, a country and, if you offer it, free shipping, without your name, and go the same three ways. When you ask for a product description draft, that product's title, vendor, type, category, tags, options, variant names and description go through OpenRouter only to Anthropic, with a short excerpt from one of your active products' descriptions so the draft is written in your store's language. OpenRouter is told to use only these companies, never another one that hosts the same model. Nothing is saved to your store until you press Save. The agent test talks only to your store and to Shopify's checkout, and the live crawler check asks your store for its own pages, following redirects only to your store's domains and Shopify's.
• What we keep. The agent's last 60 runs, each test's score with what it came from for 40 days, the AI answers with the links they cited, the log of fixes and, from the whole-catalog check, only the counts, in the app's database at our hosting provider. All of it is deleted when Shopify tells us to, 48 hours after you uninstall, except what each AI answers check and description draft cost, and when: that stays without your store's name, the questions or the answers, so the app's 30-day spending limit still counts it.

12The IAB's decision-grade bar

In August 2026 the IAB published guidance on measuring visibility in AI answers, and set out what makes such a measurement “decision-grade”. Emtor doesn't meet that bar yet. Here is where the AI answers check stands, point by point.

• Enough answers per question, with the count disclosed

  Partly

  Disclosed: up to 3 per assistant in a check, shown as a count. That's too few for a stable result, which is why each count comes with its interval.

• Measured every week or more often

  Not yet

  A check runs when you ask, at most once a day. There's no schedule yet.

• Variation over 7 days, confidence levels and a way to verify

  Partly

  A 95% interval for each question's count, worked out in every check; the app shows it beside each shopper question's count, but a fact question only as a count, such as “Right in k of n”, and this page lists the interval for the fact questions. No range over 7 days yet. The rules that read answers are tested against fixed sets of answers and phrasings.

• All four kinds of question, kept apart

  Partly

  Fact questions and shopper questions are kept apart and never pooled. The shopper questions cover two of the four kinds, recommendation (the best under a price) and transactional (where to buy); comparison and informational ones aren't asked yet.

• Assistants that cover most of the AI use

  Partly

  ChatGPT, Perplexity and Claude. Gemini is left out because of its API terms, and Copilot isn't checked.

• The collection method, terms and live web search, disclosed

  Met

  Developer APIs through OpenRouter, web search on, the shopper's country set (for Claude, only the US), and Gemini left out because of its terms. All of it is on this page.

The IAB also names what to measure. One measure is the Factual Inaccuracy Rate, and we report it as “Wrong in k of n”, for each fact and each assistant. And it calls a programme with fewer than 50 questions exploratory. A store gets 2 fact questions and two or three shopper questions today, so read the results as an alarm for big changes, not as a visibility study.

Source: IAB, Measuring Visibility in the AI Era, 3 August 2026, as summarised by PPC Land's report on the IAB framework.

13Changelog

• 7 November 2026 Also: the line Agent data on product pages gets its points only when your product pages get the embed's data. The embed has to be on, with its product data not set to Off, and where it adds to your theme's product data, an agent test has to have seen the additions attach on one of your product pages. Until a test has read one, the line gets none, and where they attach to nothing the app asks you to choose Full product data. Until now, the embed being switched on was enough.
• 7 November 2026 After a policy fix for a fact an assistant got wrong or didn't give, the question about that fact is asked again by itself, once, about a day later, within the same cap, and its answers are shown beside the ones from before the fix.
• 6 November 2026 Also: Google-Extended is now counted with the crawlers that answer shoppers, not with the training ones. Google uses it for Gemini's answers in its apps as well as for training, so keeping it out can keep your pages out of those answers. It doesn't change Google Search.
• 6 November 2026 Also: AI crawlers can be checked live. We ask for your home page and one product page from our server with each crawler's name, and say what came back as a sign, not proof.
• 6 November 2026 Also: the agent reads your store's profile for agents, agents.md and llms.txt at your web address, and the structured data on the product page it opens. They're on the receipt, not in the score.
• 6 November 2026 Also: an answer the fixed rules can't settle is judged by Claude through Anthropic and counts only if the answer's own words bear the verdict out, a fifth verdict, Unclear, never counts as right, and what leaves your store says so. Until now an answer counted as right if your figure appeared anywhere in it, even as an older figure next to the current one.
• 6 November 2026 Also: the app can read orders, only if you allow it on its AI sales page: their totals, dates and where the shopper came from.
• 6 November 2026 Also: Claude is asked too, with Anthropic's own web search, and its questions go only to Anthropic. It is told your country only if that's the US. Gemini and Copilot still aren't checked.
• 6 November 2026 Also: what leaves your store now says that a shopper question can mention free shipping, when you offer it.
• 6 November 2026 Also: after you uninstall, what the AI answers checks and description drafts cost now stays, without your store's name, so the app's 30-day spending limit still counts it. What we keep says so.
• 6 November 2026 Later still: each question and draft goes only to the company named under what leaves your store, never to another one that hosts the same model, and the short language excerpt a draft carries is now named there and taken only from active products.
• 6 November 2026 Later the same day: the shopper questions are on the app's AI answers page, and product description drafts are listed under what leaves your store.
• 6 November 2026 First version. It describes the agent test, the AI answers check, the data checks and the score as built into Emtor on this date.