Claude and Copilot as search engines: a beginner's guide

Claude SEO / Copilot SEO means making your website easy for Claude, the assistant built by Anthropic, and Microsoft Copilot to find, read, trust and cite when they search the web to answer a question. Both assistants can answer from training alone, but when they look something up they show the pages they used, and those cited links are where your visibility and your visits come from. This guide explains how each assistant searches, which crawlers and indexes sit behind it, and the first steps a business can take.

What Claude SEO and Copilot SEO mean

Neither term describes a separate discipline. It is ordinary SEO applied to two assistants that have their own plumbing. Microsoft's guidance on AI search, published in October 2025, says crawlability, metadata, internal linking and backlinks still matter, and calls them the starting point rather than the whole job.

What changes is how a page gets chosen. Microsoft describes assistants like Copilot breaking a page into smaller pieces, a process it calls parsing, judging those pieces for authority and relevance, and assembling an answer from several sources at once. Ranking still happens, but it is less about ordering whole pages and more about which passages earn a place in the final answer.

The visible result is a citation. When Claude uses the web it gives direct citations, source links and relevant quotes. In Microsoft 365 Copilot Chat, a Sources button appears under a web-grounded answer, and selecting it shows the exact query Copilot sent to Bing and the pages it used. A page that is never cited is invisible in this channel.

How Claude searches the web and picks its sources

Claude answers many questions from training alone. When a question benefits from current information it invokes a search tool, reads several sources and writes a reply with citations and source links. Anthropic's help centre says every response that uses web search includes citations.

Web search began as a feature preview for paid users in the United States in March 2025 and, according to Anthropic's update of 27 May 2025, became available globally on all Claude plans. In the newer Claude experience there is no web search toggle: Claude searches when it decides that searching helps. On Team and Enterprise plans an owner has to enable web search for the workspace first.

Anthropic's developer documentation lists what triggers a search: recent events, current prices or statistics, explicit requests to look something up, and information about specific organisations, people or products that might have changed. That last category covers most questions a buyer asks about a business.

With web search on, Claude can also fetch a specific page when someone pastes its URL, a feature Anthropic calls web fetch, and it may use a location inferred from the person's IP address for local results. Anthropic does not publish a ranking formula, and its web search help page names Bing only for image results, so this guide does not guess at the rest.

Anthropic's three crawlers and what blocking each one does

Anthropic runs three robots, each with a different job. Its help centre describes them like this.

• ClaudeBot collects web content that may contribute to training Anthropic's models. Blocking it signals that your site's future material should be excluded from training datasets.
• Claude-SearchBot analyses content to improve the relevance and accuracy of search responses. Blocking it stops Anthropic indexing your content for search, which Anthropic says may reduce your visibility and accuracy in search results.
• Claude-User fetches a page when a person asks Claude something that needs it. Blocking it stops Claude retrieving your content for that answer, which may reduce your visibility in user-directed web search.

Training and answering are separate decisions. A business that wants to stay out of training data can block ClaudeBot and still allow the two bots that put it in front of buyers. Anthropic's own example for blocking one bot looks like this.

User-agent: ClaudeBot
Disallow: /

Anthropic says its bots honour robots.txt, support the non-standard Crawl-delay directive, and will not try to get around CAPTCHAs. The rules must be set on every subdomain you want covered, so a shop or blog on its own host needs its own file, and blocking Anthropic's IP addresses instead may not work correctly or persistently, because it stops the bots reading the file.

The robots.txt format is an IETF standard, RFC 9309. A crawler follows the group that names it, and falls back to the asterisk group only when no group carries its name. A compliant crawler should not rely on a cached copy for more than 24 hours, so a change should take effect within about a day. And the standard says the rules are not a form of access authorisation: robots.txt is a request.

Our SEO for Claude service sets this policy for clients bot by bot and subdomain by subdomain.

How Copilot finds its answers through Bing

Copilot is Microsoft's assistant. In Microsoft 365 Copilot Chat, when web search is on, Copilot generates a short query from a few words of your prompt, or from a file you have uploaded, and sends that query to Bing. The full prompt is not used as the query unless it is already very short. Bing's results are then used to shape the answer.

Microsoft calls connecting an answer to real information grounding. Copilot can ground a response in your organisation's work data, in the public web, and in content attached to the prompt. Grounding produces the citations under an answer, and Microsoft advises checking them. It also warns that Copilot can give outdated answers if it is pointed at an old website or an earlier version of a file.

Availability varies by licence, subscription and admin settings, and on a work or school account the organisation may have turned web search off entirely. If Copilot cannot find relevant, accessible sources, Microsoft says it may give a more general answer.

Microsoft also states that experiences like Copilot are powered by Bing's search index. If Bing has not crawled a page, or holds an old copy of it, Copilot has little to cite and may quote details you changed months ago. Bing is where Copilot visibility starts, which is why our SEO for Microsoft Copilot service is built on Bing Webmaster Tools and IndexNow rather than on the assistant itself.

How this differs from Google SEO and from ChatGPT

Most of the writing work is shared between assistants, but the technical access is different for each one. Copilot's web results come from Bing, so the tools that matter are Bing Webmaster Tools and IndexNow. Google Search Console has no say in what Copilot cites, and a page that ranks well on Google but is missing or stale in Bing's index gives Copilot nothing to work with. For Claude, the access levers Anthropic documents are its own crawlers, so the first question is whether Claude-SearchBot and Claude-User can read your pages.

The same logic separates Claude from ChatGPT and every other assistant. Under the robots.txt standard a group of rules speaks only to the bot it names, so a rule written for ClaudeBot says nothing to anyone else's crawler, and access has to be decided crawler by crawler. If you are new to the whole area, start with what AI SEO means and come back here for the Claude and Copilot specifics.

What makes a page easy for Claude and Copilot to quote

Microsoft published detailed guidance on this in October 2025. Anthropic's public documentation focuses on crawler access and search behaviour rather than page structure, so Microsoft's list is the most detailed first-party guidance we found, and it applies to any assistant that lifts passages from a page.

• Title, description and H1 that agree. Microsoft calls these the signals an assistant uses to interpret purpose and scope, and says consistent alignment between them improves confidence.
• Descriptive headings. H2 and H3 tags divide a page into clear sections, so an assistant can tell where one topic stops and the next starts. Microsoft compares them to chapter titles. Microsoft's example swaps “Learn More” for “What Makes This Dishwasher Quieter Than Most Models?”
• Question and answer pairs. Plain questions with direct answers match how people actually search, and Microsoft says assistants can often lift the pair word for word into a response.
• Concise, self-contained answers. One or two sentences that address a question directly, phrased so they still make sense when pulled out of the page.
• Measurable facts instead of adjectives. Microsoft's example is “42 dB dishwasher designed for open-concept kitchens” rather than “quiet dishwasher”. Words like innovative mean little without specifics, and claims like cutting-edge leave an assistant unsure how to classify the page.

Schema markup helps as well. Microsoft describes it as code, usually JSON-LD added through your CMS or by a developer, that labels content as a product, review, FAQ or event so machines can interpret it with confidence. Microsoft lists schema among its core practices for AI search visibility, while stressing that nothing guarantees selection in AI answers.

A worked example

Take a hypothetical commercial cleaner in Geelong whose services page opens with “We are a passionate, innovative team delivering next-level cleaning solutions”. An assistant finds nothing there it can quote or check. Rewritten answer-first, the page opens with “We clean offices of up to 5,000 square metres in Geelong and the Bellarine every weeknight, with police-checked staff and a 24-hour re-clean guarantee.” Every claim is specific and the sentence stands on its own.

How to get started

1. Ask the assistants your buyers' questions. Write down the ten questions a customer asks before choosing a supplier like you, put each one to Claude and to Copilot, and record which businesses and pages are cited. In Microsoft 365 Copilot Chat, open the Sources button and note the query it sent to Bing; in other versions, record the cited links.
2. Check robots.txt on every subdomain. Look for groups naming ClaudeBot, Claude-SearchBot and Claude-User, and for an asterisk group that blocks everything. Decide training and answering separately, and if you want out of training, block ClaudeBot only.
3. Verify your site in Bing Webmaster Tools and submit your sitemaps. Bing says it respects preferences expressed through robots.txt and other supported controls, so check that nothing is accidentally keeping Bing out.
4. Set up IndexNow. It is a free, open-source protocol that tells participating search engines the moment a page is added, updated or deleted, and many content management systems and CDNs already support it. One request can carry up to 10,000 URLs, and a 200 response only means the URLs were received. Bing recommends it so that AI answers use the current version of a page.
5. Keep local details current in Bing Places for Business. Bing says accurate address, hours and contact details help keep a business eligible for inclusion in AI answers to location-based questions.
6. Rewrite your most important pages answer-first. Start with the pages closest to a sale: services, pricing, comparisons and locations. Put the direct answer in the first sentence, follow it with evidence, and keep the text in plain HTML rather than in images, PDFs or tabs.

Common mistakes that keep sites out of AI answers

• Blocking every AI bot with one rule. A blanket block written to keep content out of training also stops Claude-User and Claude-SearchBot, the two bots that serve answers.
• Rules on the main domain only. Anthropic says opt-outs must be added for every subdomain, so a shop or blog on its own host is a separate decision.
• Blocking by IP address. Anthropic says this may not work correctly or persistently, because the bots can no longer read your robots.txt.
• A robots.txt that returns a server error. Under RFC 9309, if the file is unreachable because of a server error, a crawler must assume the whole site is disallowed.
• Answers hidden in tabs, accordions, images or PDFs. Microsoft says assistants may not render hidden content, that PDFs often lack the headings and metadata HTML provides, and that text inside images is read less reliably. Long walls of text, decorative symbols and unanchored claims are also on Microsoft's lists of mistakes.
• Bing never set up. Without Bing Webmaster Tools and IndexNow nobody is checking what Bing holds, and Copilot can end up quoting prices or services from an old copy of a page.

How to measure visibility in Claude and Copilot

Copilot is easier to measure because Microsoft publishes first-party data. In February 2026, Bing Webmaster Tools launched a public preview called AI Performance, which counts how often your pages are cited in Microsoft Copilot, in Bing's AI-generated summaries and in some partner products. It reports total citations, the average number of unique pages cited per day, a sample of grounding queries, page-level citation counts and a trend over time. Microsoft notes the counts show frequency of citation, not placement or ranking.

In June 2026, Microsoft began rolling out four preview features globally. Intents and Topics group grounding queries by purpose and by theme. Citation Share shows the percentage of all citations for a grounding query that went to your site, and Compare overlays a previous period on the current view. Microsoft describes Citation Share as observational: it is not a ranking, does not expose competitor domains and does not represent traffic share.

Because Bing Webmaster Tools and its AI reporting are available globally, Copilot optimisation for New Zealand businesses follows the same steps as in Australia.

Claude has no equivalent publisher report that we could find documented, so measurement is manual and rests on three sources.

• Tracked questions. Run the same fixed set of buyer questions in Claude each month and record whether you are cited, how you are described and which page was used.
• Server logs. Your logs show when ClaudeBot, Claude-SearchBot and Claude-User visit and which pages they read. On Mojo Dojo's own site, AI crawlers made up 35% of 240,060 bot visits over 30 days, level with search engine crawlers, so the logs are worth reading.
• Referral traffic. Separate visits from claude.ai in your analytics so you can see what Claude sends and whether those visitors enquire.

Third-party citation trackers can show direction of travel, but they are built on their own query sets rather than official data. Treat Microsoft's report and your own logs as the record.

Frequently asked questions

Can I stop Claude training on my site but still appear in its answers?

Yes. Block ClaudeBot in robots.txt and leave Claude-SearchBot and Claude-User allowed. Anthropic says blocking ClaudeBot signals that future content should be excluded from training, while the other two bots handle search and user requests.

Does ranking on Google help in Copilot?

Not directly. Copilot sends its query to Bing, and Microsoft says Copilot experiences are powered by Bing's index. Good content helps everywhere, but Bing indexing is its own job.

Will Copilot always search the web?

No. Web search availability varies by account type, licence and organisation settings, administrators can turn it off for work and school accounts, and if Copilot cannot find relevant sources it may answer in general terms.

Does Claude always search the web?

No. Claude searches when it judges that current information helps, or when asked to, and in the newer Claude experience there is no toggle.

Where to go from here

Start with the two checks that cost nothing: read your robots.txt for Anthropic's three bots, and verify your site in Bing Webmaster Tools so you can open the AI Performance report. Then run your buyer questions in both assistants and write down who is cited. That baseline tells you whether this channel matters before you spend anything on it.

For the fuller picture, our free guide, The GEO Playbook, covers how a generative engine picks its sources, the crawler and rendering decisions, and how to measure a channel with no rankings, across ChatGPT, Gemini, Perplexity and Claude.