Shopify's agent tools answer returns, not arrival dates or pickup

Shopify’s built-in policy tool answers an agent’s question about returns and little else. Bill Hunt, president of the agency Bisan Digital, put buyer questions to the policy tool on 88 of the top Shopify storefronts and got a usable return window from 70 of them, in an audit published by Search Engine Journal. Asked what shipping to Boston costs and when it would arrive, only one store replied, and none gave an answer an agent could act on.

Only the return window came back reliably.

Question put to the policy tool	Stores that replied	Usable answers
Return window	88 of 88	70
Can used or open-box items be returned	88 of 88	5
Can an order be cancelled, and by when	8 of 88	2
Shipping cost to Boston	1 of 88	0
Arrival date	0 of 88	0

Two stores answered the cancellation question with their shipping FAQ. One store publishes “free standard ground shipping on U.S. Orders over $75” on its own site, and that line never came back when the tool was asked what shipping costs. Hunt calls a confident wrong answer more dangerous for an agent than no answer, because the agent has no reliable way to tell that the text does not address the question it asked.

The checkout route itself works. Hunt opened 74 live checkouts through the Universal Commerce Protocol (UCP) over the Model Context Protocol (MCP), Shopify’s server route that lets an agent create and read a checkout with no browser involved. Every one created the checkout, read it back, confirmed a cancel, and refused a second cancel with a coded error instead of repeating it silently. Median response was about 1.1 seconds. A live checkout also priced shipping to Boston at 36 of the 74 stores, so an agent can get the cost at half the sample, but only by starting a purchase.

Shopify’s second route, WebMCP, registers callable tools inside the shopper’s own browser tab and reached checkout on September 28. It is the same approach behind Cloudflare’s agent browser, which calls site tools instead of clicking. Hunt’s September 30 test of that route read a full checkout back with line items and totals.

What those checkouts never returned was a date. None of the 74 gave an arrival estimate or a carrier, though the protocol has fields for both. The fulfillment declaration, the file at /.well-known/ucp.json that tells an agent how a store delivers, is emptier still: all 93 stores that publish one list shipping as the only method, and not one declares pickup. Hunt found a store that documents Pickup in Store on its own site while its agent endpoint answers “No pickup locations available.”

Almost none of these stores wrote their own agent surfaces. 89 of the 122 serve an /agents.md file, the plain-text brief that tells an agent how a site works. Of those, 83 are near-identical copies of Shopify’s generated template, and five were written by the merchant. Shopify switched the tools on for every Liquid storefront on August 5, so counting agents.md files measures the rollout and not merchant readiness.

Half the sample also blocks its own policy pages from crawlers. 51 of the 122 stores disallow /policies/ in robots.txt. That is not Shopify’s current default: Shopify’s documentation says the rule persists only on themes with an old robots.txt.liquid template, and deleting the template restores the default. The disallow does not stop the policy tool, which an agent calls server-side, but it does keep crawlers and live fetchers from reading the page where the return rule is written.

On the product page, where the buying action lives, two-thirds of the stores that use dropdowns ship a size or option selector with no label. An agent then has to guess which control sets the size and which sets the color.

What to do

Ask the store’s own policy tool the questions buyers ask. This one call shows what an agent gets back:

curl -s https://YOURSTORE.com/api/mcp \
-H 'Content-Type: application/json' \
-d '{"jsonrpc": "2.0" ,"id":1, "method": "tools/call",
"params":{"name":"search_shop_policies_and_faqs",
"arguments":{"query": "Can I cancel an order after placing it,
and by when?"}}}'

Then fill the fields the answers came back empty on:

• Set return and cancellation rules in Shopify admin, and write the same values as plain numbers in the refund and shipping policy pages and the FAQ. A rule that exists only as prose is the case that failed across Hunt’s sample.
• Turn on delivery dates if the store can meet them, so an arrival estimate reaches the protocol instead of leaving the field empty.
• Check that /.well-known/ucp.json declares pickup if the store offers it, instead of defaulting to shipping only.
• Remove Disallow: /policies/ from robots.txt by deleting the old robots.txt.liquid template, which restores Shopify’s default.
• Replace the default agents.md with the store’s actual return window, cancellation rule, shipping threshold and pickup options.
• Add MerchantReturnPolicy and OfferShippingDetails to product structured data, with itemCondition for used or refurbished stock, so the same rules are readable without a tool call.
• Label every size and option selector, which also fixes it for screen reader and keyboard users.