A customer mentions, almost in passing, that they found your store by asking ChatGPT for a recommendation. You are pleased, and then a little unsettled. Is your store regularly named in those answers, or was this a fluke? How would you even find out?

You can find out with nothing more than a notebook and an hour. This guide shows you how to test whether your Shopify store appears in AI answers, how to read the results honestly, and what to fix first if the answer is “not really”.

Step 1: Run a proper buyer test

The mistake most owners make is to type their own brand name and feel reassured when the assistant knows who they are. That only proves the assistant has heard of you. It says nothing about whether a stranger who has never heard of you would be pointed your way.

So test the way a real buyer asks. Use questions that do not include your brand name. Here are five shapes that work for almost any store:

  • Best for a use case: “What is the best [product] for [use case]?”
  • With a limit: “What is a good [product] under [price]?”
  • Where to buy: “Where can I buy [product type] for [kind of buyer]?”
  • Alternatives: “What are good alternatives to [well-known competitor]?”
  • Gifts: “What are good [product] gift ideas for [type of person]?”

Make each one specific. “Best running shoes” is easy to answer with big names. “Best zero-drop running shoes for wide feet on long runs” is where smaller stores can actually win, because the answer depends on details that only some stores publish.

Write your questions down, pick five to ten, and try them in more than one assistant, for example ChatGPT, Google’s AI answers, Perplexity, Claude and Copilot. Record the results in a simple table like this:

Question Assistant Did we appear? Who did appear? What do they have in common?
Best [product] for [use case] ChatGPT No Store A, Store B Comparison guide, many reviews
Where to buy [product type] Perplexity Yes Us, Store C Clear shipping and returns info

The last column is the valuable one. Look at who keeps appearing and ask what those stores have that you do not.

Step 2: Do not trust a single result

Here is the catch. Ask the same question twice and you can get two different answers. Results shift by session, by location and by model version. One appearance proves very little, and one absence proves little too.

What matters is a pattern:

  • Do you show up repeatedly, not just once?
  • Do you show up in several assistants, not just one? A store can be named by most assistants and be completely missing from another, which a single check would hide.
  • Do you show up for different wordings of the same need?

Repeat the test every week or two, and keep your notes. After a month you will have something much more honest than a screenshot.

Step 3: Work out what kind of problem you have

If your results are poor, the cause is usually one of two things, and the fixes are completely different.

  • Technical visibility. The assistants cannot read your store properly, because of a blocked bot, a price that only appears through scripts or broken structured data.
  • Recommendation visibility. They can read you fine, but they choose other stores, because those stores explain themselves better or are talked about in more places.

Start with the first, because it is cheaper to fix and blocks everything else. Our guide to auditing your store for AI visibility walks through each technical check.

If you can see bot activity through a security or CDN tool, there is a useful clue. Assistants use bots that fetch a page live when a real person asks a question. If those never touch your product pages, it looks like a discovery or access problem. If they visit regularly and you are still not named, they are reading you and choosing someone else, which points to content and trust.

Step 4: Find out which route the customer came through

When your customer said they found you through ChatGPT, one of two different things could have happened. They are worth telling apart, because they are fixed differently.

  1. They were shown your website as a source. The assistant read pages on the web and mentioned your site. That depends on crawl access, the content on your pages and what others say about you.
  2. They were shown your product as a product card, with a price and a link. That runs through product catalogs, not through blog posts. On Shopify, it depends on whether your store is connected to the relevant sales channels, and on how complete your product information is.

Ask the customer which it was, if you can. Then, for the second route, open your Shopify admin and check the channel and catalog settings, and look at the fields Shopify uses to understand a product. Make sure the brand field is filled in properly (it is not always the same as the vendor field), that the product type is specific and that variant details are entered as structured data and not just written into the description. Shopify has also been adding reports on AI channels, so it is worth looking at your analytics for those.

Step 5: Fix the things that most often move the needle

Merchants who have gone through this process tend to land on the same short list:

  • Clean, detailed product data. Specifications, materials, sizes and use cases written out in plain language, not just adjectives.
  • Real content around the product. Comparison pages, “how to choose” guides and genuine reviews give an assistant something to quote.
  • Mentions elsewhere. Stores that get named often have coverage on independent sites: reviews, roundups and forum discussions that describe them consistently.
  • Fast, readable pages. If a bot cannot read a page easily, it cannot quote it.
  • Being findable in Bing. Some AI search features draw on Bing’s index alongside other sources, so it is worth adding your site to Bing Webmaster Tools, and to Google Search Console, and checking that your pages are indexed.

Step 6: Measure sales honestly

This is the hard part, and it is fine to admit it. Visits from AI assistants often arrive with no clear referrer and are counted as “direct” traffic. Some people also discover a brand in an assistant and then search for its name in Google, which makes it look like a branded search.

A few habits help:

  • Add “ChatGPT or another AI assistant” as an option in a post-purchase question such as “How did you hear about us?”
  • Set up a custom channel in your analytics for the AI referrers you can identify.
  • Watch for growth in direct visits to your product and guide pages, and in searches for your brand name.
  • Keep crawler visits separate. A bot visiting is a sign of discovery, not a customer.

None of this is exact. Treat it as direction, not accounting.

What not to do

  • Do not judge from one test. Look for patterns over weeks.
  • Do not buy a promise. Nobody can guarantee you a place in an AI answer. Be careful with anyone who does.
  • Do not stuff pages with keywords or generic text. Assistants need specific, honest information.
  • Do not expect one small file to change everything. Things like llms.txt are cheap to add but unproven.

A simple 30-day plan

  1. Days 1 to 3: run your buyer test and record it.
  2. Days 4 to 10: fix technical access and readability, following the audit guide.
  3. Days 11 to 20: improve your product data and write one honest comparison or “how to choose” page for your best product.
  4. Days 21 to 30: ask customers for reviews on independent sites, set up your survey question and repeat the buyer test to compare.

Final thoughts

The customer who found you through an AI assistant was a small signal that the door is already open a little. The stores that widen it are not doing anything magical. They make their products easy to understand, they say clearly who each one is for, and they give assistants no reason to hesitate.

If you would like help turning that into a plan for your own store, book a free call and we will go through your results with you. You can also start with our free Store Health Score, or read how a technical SEO review works.