How does the agent test work, step by step?

Emtor sends a shopping agent into your store through the Universal Commerce Protocol (UCP), the interface Shopify gives AI shopping agents. The agent does what one does when a shopper asks it to buy, in 8 steps:

1. Finds your store's checkout service
2. Asks your store about returns
3. Asks your store about shipping
4. Searches your catalog
5. Opens a product and picks a variant
6. Creates a checkout (no payment)
7. Chooses a delivery option
8. Cancels the test checkout

What it asks your store

Two questions a shopper would ask, word for word:

What is your return policy? How many days do I have to return an item, and who pays for return shipping?

How much does shipping cost to [your country], is there free shipping over a certain amount, and how long does delivery take?

The result quotes your store's answers. If you confirmed your store facts, the answers about your return window and free shipping are checked against them.

How it picks a product

It starts from your 25 most recently updated active products: it searches your catalog for the commonest kind among them, then for the name of one your admin says is in stock. It opens up to 6 results until one has a variant it can add to a cart. It passes over those it knows agents can't buy: gift cards, pre-orders, products sold only by subscription, bundles, and products whose title says they're personalized. It also opens that product's page on your site, the way a browser agent would, and records whether a bot check or a block turned it away.

What it reads at your address

On the product page it opened, it reads the structured data (JSON-LD): how many Product entries the page has for the product, the price, currency, availability and rating each one states, whether they agree with each other, and whether their prices and currency agree with Shopify's own data for the product. After the steps, it reads what agents find at your store's own web address, once each and without following redirects: your store's profile for agents (/.well-known/ucp), checked against what the Universal Commerce Protocol requires of one, and agents.md and llms.txt, which Shopify serves for your store by default and which point agents to your checkout service and your policies.

The receipt shows all of it under What agents read at your address, and none of it changes your Agent-ready score. When something isn't there, the receipt says how your address answered: for example no profile where your domain isn't served by Shopify, neither file on a Hydrogen storefront, an llms.txt that no longer points to your checkout, or your password page while your store is password-protected.

The checkout

It creates a checkout for one item with a made-up shopper: the email address agent-test@example.com, a made-up street in a real city so your store can quote delivery, and a phone number from a range set aside for fiction, where the country has one. If your store doesn't deliver to that address, it tries another city in the same country, where it has one.

It picks a delivery option, switching to another one when your store offers more than one, and checks that your store charges for the one it picked. If your store hands the checkout over with a link to finish paying, the agent opens that link and records what it shows. Then it cancels the checkout. Why it never pays

The shopper is in the country you set in Store facts. Without one, it's the US if you ship there, or else the first country you ship to. The test has an address for 13 countries, and so far only the US one has been tested against a store. In a country without one, the checkout stops before delivery, and the result says that's a limit of the test, not of your store. Until the test has one, that line earns no points in the Agent-ready score.

What you get

The agent's way through your store, station by station: Agent connection, Shipping & returns, Search, Product, Checkout, Delivery and Pay, each with what the agent did there. Pay is never attempted, because the agent never pays. When Shopify itself doesn't answer in time, a station says Couldn't check: it isn't counted as your store's problem, though it earns no points until a later test checks it.

Every test also has a receipt: each step, how long it took and what your store said. Open it from the test's result, then use Print or save as PDF to keep it.

When it runs

The first test starts by itself the first time you open Emtor. After that, it runs when you start it, and every 24 hours once you turn on the daily test. After a policy fix, it also runs by itself 30 minutes and 24 hours later.