What is AI search, and how do AI search engines work?

AI search is a kind of search where an AI model reads a set of web pages for you and writes a direct answer in plain language, usually with links to the pages it used, instead of only handing you a ranked list of links. Google AI Overviews and AI Mode, ChatGPT search, Perplexity, Copilot Search in Bing and the web search built into Claude and the Gemini app all work this way. Each answer is built from a small number of sources, so for a business the practical question is whether your pages are among them. This guide explains where those answers come from, how the main tools differ, why they sometimes get things wrong, and what you can do about it.

What is AI search?

That is the whole idea, and it goes by several names: AI-powered search, generative search and answer engines. They all describe a search experience where a model reads sources and writes a direct answer with citations, rather than only returning a ranked list of links.

A classic results page gives you ten or so links and leaves the reading to you. An AI search engine does the first pass of reading for you: it runs searches, pulls in a handful of pages, writes a summary and shows which pages it used. The links are still there, but they now sit under the answer as its evidence.

The main AI search engines and how they differ

Google AI Overviews. Launched at Google's I/O conference in 2024, AI Overviews give people the gist of a complicated question with links to explore further. Google says one is only shown when its systems decide it adds something beyond the classic results, so they often do not trigger.

Google AI Mode. A separate, end-to-end AI search tab for questions that need reasoning, exploration or comparison. Google began rolling it out to everyone in the US on 20 May 2025 with no Labs sign-up, and says AI Mode and AI Overviews may use different models and techniques, so they can cite different pages.

ChatGPT search. Introduced on 31 October 2024 and opened to everyone in supported regions, with no sign-up, on 5 February 2025. ChatGPT decides when a question needs current information, and as of October 2026 OpenAI says web search is available on every ChatGPT plan from Free to Enterprise and Edu, and to people who are not signed in.

Perplexity. Its help centre, last updated in August 2026, describes four steps: interpret the question, search the internet in real time, compile the most relevant information into a summary, and attach numbered citations linking to the original sources.

Copilot Search in Bing. Launched by Microsoft on 4 April 2025, it blends a traditional results page with a generated answer. It cites sources prominently, links whole sentences inline to where they came from, and with one click shows every link used to generate the answer.

Claude. Anthropic publishes three crawlers for Claude: one for model training, one that fetches a page when a user asks about it, and one that indexes content to improve search results. So Claude does retrieve live pages when asked.

How AI search engines work, step by step

Most of these tools follow broadly the same five steps.

1. Understand the question. The system interprets a conversational question, often long and multi-part. Google says people in its AI experiences ask longer and more specific questions, then follow-ups.
2. Rewrite and fan out. One question becomes several searches. Google calls this query fan-out: breaking a question into subtopics and issuing many related searches at once. OpenAI says ChatGPT typically rewrites a question into one or more targeted queries, reviews the first results, and may send more specific follow-up queries.
3. Retrieve. The system pulls candidate pages from an index. Google's AI features sit inside Google Search and draw on its index. OpenAI says ChatGPT search uses third-party search providers as well as content supplied directly by partners. Perplexity says it searches the web in real time for each question.
4. Write the answer. A language model reads the retrieved passages and writes one response. Google says its models keep finding more supporting pages while a response is generated, so an AI result can link to a wider set of sites than a classic results page.
5. Cite. The answer links back to its sources: links in Google's AI results, a Sources panel in ChatGPT, numbered citations in Perplexity, and in Copilot Search a list of every link used.

On top of this sit conversation memory, since ChatGPT, Perplexity and Copilot Search all keep the context of earlier questions, and depth: Google's Deep Search, announced for AI Mode in May 2025, can issue hundreds of searches for one fully cited report, and Perplexity's Research feature runs dozens of searches and reads hundreds of sources.

Training data versus live retrieval: where the answers come from

An AI model learns language and a great deal of general knowledge by reading an enormous body of text during training. That knowledge is frozen when training stops, and the model cannot say exactly where a fact came from.

Retrieval is the fix. In a 2020 paper accepted at NeurIPS, Patrick Lewis and colleagues described retrieval-augmented generation, or RAG: a model that pairs what it learned in training with an external index it can look things up in before it writes. They reported more specific and more factual language than a model relying on memory alone.

Picture two assistants asked about your opening hours. One answers from what it read months ago; the other checks your website first and tells you where it looked. AI search engines work like the second. Google calls this grounding. Its crawler documentation describes it as feeding the model content from the Google Search index when the question is asked, so the answer is more accurate and more relevant.

For a business the split matters because the two layers respond to different things. Retrieval reads the most recently crawled version of your pages, so changes to crawlable, indexed pages can be reflected once they are recrawled. Training is shaped over a longer period by everything written about you across the web. Both rely on automated crawlers, and the major companies give you separate controls for each job.

Why AI answers can be wrong

AI answers sound confident, which makes their errors easy to miss. OpenAI defines hallucinations as plausible but false statements generated by language models and said in September 2025 that they remain a fundamental challenge for all large language models, partly because standard training and evaluation reward guessing over admitting uncertainty, much as a test rewards a lucky guess over a blank answer.

Search does not remove the problem, because retrieval can fetch the wrong page or an outdated one. OpenAI's help page warns that search results and citations in ChatGPT can be incomplete, outdated or incorrect, and advises opening the cited source to check that it supports the answer.

For a business the lesson is practical: an AI answer about your company is only as good as the pages it retrieved. If your prices, services, locations or hours are out of date on your website, your Google Business Profile or a directory, an assistant may repeat the stale version. Correcting a wrong answer usually means finding the page the model read and fixing it there.

AI search versus traditional search: what changes for your business

Three things change for a business: which questions get an AI answer, what happens to clicks, and what counts as being visible.

Longer questions get AI answers. In Pew Research Center's analysis of March 2025 browsing data from 900 US adults, covering 68,879 Google searches, 8 percent of one- or two-word searches produced an AI summary, rising to 53 percent for searches of ten words or more and 60 percent for searches beginning with who, what, when or why. Overall, 18 percent of searches produced one.

Clicks change. In the same study, people who saw an AI summary clicked a traditional result in 8 percent of visits, against 15 percent with no summary, and clicked a link inside the summary in 1 percent of visits. Those are US, Google-only figures from one month. Google says clicks from results pages with AI Overviews are higher quality, meaning visitors are more likely to spend more time on the site, and advises judging visits by conversions rather than clicks alone.

Visibility changes. Search indexes still feed the retrieval step, but being visible in AI search means being cited, mentioned or recommended inside the answer. There is more room than it sounds: Pew found 88 percent of the AI summaries it examined cited three or more sources and only 1 percent cited a single source.

Is optimising for AI search just SEO? AI SEO, GEO and AEO

Mostly, yes. Google says there are no additional technical requirements to appear as a supporting link in AI Overviews or AI Mode: a page needs to be indexed and eligible to show in Google Search with a snippet, and the existing SEO fundamentals continue to apply, from allowing crawling in robots.txt and at your CDN to internal links, page experience, content in text form, structured data that matches the visible text, and current Merchant Center and Business Profile details.

The labels for this work are AI SEO, generative engine optimisation (GEO) and answer engine optimisation (AEO). They describe the same job: making your business easy for AI systems to find, understand, trust and cite. Our guide to AI SEO unpicks the overlap, and our GEO guide and our AEO guide cover each label. In practice it is the SEO you already know, with more weight on answer-first pages, consistent facts about the business and new measurements, which is how our AI SEO services are put together.

One caveat applies to every label. No one can guarantee a citation, because each tool chooses its own sources. OpenAI says this plainly: ChatGPT ranks search results using multiple factors, and placement is not guaranteed.

AI crawlers: who is reading your site, and should you block them?

Every AI search engine reads your site through crawlers, and the major companies now give you separate controls for training and for search, so you can allow one without the other.

OpenAI. OAI-SearchBot surfaces sites in ChatGPT search; sites that block it are not shown in ChatGPT search answers, though they can still appear as navigational links. GPTBot collects content that may be used for training. The settings are independent, so you can allow OAI-SearchBot and disallow GPTBot in the same file, and OpenAI says a change takes about 24 hours to apply to search.

Anthropic. ClaudeBot collects content that could contribute to model training. Claude-SearchBot indexes content to improve Claude's search results, and Claude-User fetches pages when someone asks Claude a question. Anthropic says disabling either of the last two may reduce your visibility in Claude's search results, and that IP blocking may not work as an opt-out because it gets in the way of Anthropic reading your robots.txt.

Google. Googlebot governs crawling for all of Google Search, and because AI is built into Search that includes AI Overviews and AI Mode. Google-Extended is a separate token covering training of future Gemini models and grounding in Gemini Apps; Google says it does not affect inclusion in Google Search and is not a ranking signal. To limit what Google shows from your pages, including in AI features, use nosnippet, data-nosnippet, max-snippet or noindex. Since August 2026 Search Console has also had a Search generative AI setting, under Settings, that lets you exclude your site from AI Overviews and AI Mode without dropping out of the regular results. It is on by default, Google says it is not a ranking signal and does not affect training, and a change takes a few days to apply.

Robots.txt itself is a standard, RFC 9309, published by the IETF in September 2022, and the standard says its rules are not a form of access authorisation, so anything that must stay private needs real access controls. A sensible default is to allow the search crawlers from every major company, decide separately about the training crawlers, and check that your CDN or firewall is not silently blocking the ones you allowed.

How to start getting found in AI search

1. Check access. Confirm Googlebot, OAI-SearchBot and Claude-SearchBot are allowed in robots.txt and that your CDN or firewall admits the IP ranges OpenAI and Anthropic publish. Check that key pages return HTTP 200, are indexed, and keep important content in text rather than only behind heavy JavaScript. In Search Console, check that the Search generative AI setting is left on.
2. Lead with the answer. On each important page, put a direct one- or two-sentence answer at the top of each section, then evidence, definitions and a short FAQ. This is our own approach rather than a published rule: a system building an answer from passages needs one it can lift cleanly.
3. Write for the longer question and its sub-questions. Google's May 2025 guidance asks for unique, non-commodity content, and notes that people in its AI experiences ask longer, more specific questions and follow-ups. Because one question fans out into several searches, a site that answers the whole set of related questions on a topic gives the retrieval step more to find. List the questions a buyer asks before choosing a business like yours and answer each one on your site.
4. Earn mentions elsewhere. In our experience, consistent mentions and links from reputable third-party sites make a business a safer source to name, though no engine publishes a formula for this.

For a fuller checklist on shaping pages so answer engines can quote them, our AEO playbook walks through each step.

How to measure your visibility in AI search

Search Console. Google counts AI Overviews and AI Mode traffic in the main Performance report under the Web search type. Since August 2026 Search Console has also had a separate Generative AI performance report showing how often your pages appeared in AI Overviews and AI Mode, broken down by page, country and device. It reports impressions only, so you still cannot separate AI clicks from classic clicks.

AI referrals in analytics. Set up a segment for visits referred from chatgpt.com, perplexity.ai and the other assistants. Many businesses have not: in our AI Visibility Audit benchmark of more than 700 B2B companies, 42 percent said they could see AI traffic in their analytics but could not segment it. In October 2026 an apparent drop in ChatGPT referrals in GA4 was most likely a tracking gap: for about a week many inline links in ChatGPT answers stopped carrying their utm_source tag, so a large share of those visits were filed as Direct.

A regular prompt check. List the questions a real buyer would ask before choosing a business like yours, ask them in Google AI Mode, ChatGPT, Perplexity, Copilot and Claude on a fixed schedule, and record whether you are cited, mentioned or recommended, and who else is. Answers vary by day and by tool, so watch the trend over months rather than one result.

Frequently asked questions

Can I opt out of AI training but stay in AI search?

Yes, with the major companies. Block GPTBot and allow OAI-SearchBot for OpenAI, block ClaudeBot and allow Claude-SearchBot and Claude-User for Anthropic, and use Google-Extended for Google, which covers Gemini training and grounding and has no effect on Google Search. One catch: blocking Google-Extended also keeps your content out of grounded answers in the Gemini app, so for Google the split holds for Search, including AI Overviews and AI Mode, but not for Gemini.

Can I pay to appear in AI answers?

Ads and organic answers are separate. Google can show labelled ads above, below or inside AI Overviews, including in Australia, and is testing ads in AI Mode, most recently for people who connect Gmail and Google Photos. OpenAI's OAI-AdsBot only visits landing pages submitted as ChatGPT ads, and OpenAI says organic placement is not guaranteed. Neither company documents a way to buy a place in the organic answer.

How long does it take?

Google says recrawling a changed page can take from several days to several months, and you can request a recrawl in Search Console. Google also says a change to the Search generative AI setting takes a few days to apply. OpenAI says a robots.txt change takes about 24 hours to apply to ChatGPT search. Being cited has no fixed timeline.

The practical next step is to run the access check above, then ask three or four real buyer questions in the main tools, note who gets named, and read the pages they cite. That half hour teaches more about AI search than any definition.