Does Google penalise AI-written content?

Not for being AI-written. Google says it rewards helpful content however it is produced. Its spam policies target content mass-produced mainly to rank, by any method (scaled content abuse). A page that says nothing new is weak whoever wrote it.

Should I block AI crawlers in robots.txt?

Decide per crawler. Some collect training data (such as GPTBot) and some fetch pages to answer searches (such as OAI-SearchBot); blocking a search crawler keeps you out of that assistant’s answers. Google-Extended controls model training and does not affect Google Search or AI Overviews.

Will AI Overviews kill my traffic?

It depends. A page whose value is one short fact loses clicks when the results page states it. Pages people use, compare, buy from or trust keep them, and a cited page can gain visits. Search Console counts AI-feature traffic inside its ordinary web figures, so watch clicks per query over time.

Is it still worth learning SEO?

Yes. The skills that matter most are the old ones: a site that can be crawled and indexed, pages that say plainly what they are, and being worth citing, which is what an AI system needs before it can use a page as a source.

Short answer

Is SEO dead now with AI?

No. AI changed where people read the answer, not how an answer gets found. Google's AI Overviews, ChatGPT and Perplexity all build their answers from pages they can discover, fetch, understand and trust, and they cite the pages they used. Getting a page into that set is what SEO has always been.

What has changed is the reward for some kinds of page. Quick factual answers now get read on the results page instead of on your site. Pages that people need to use, compare, buy from or trust are worth as much as ever.

What actually changed

Since 2024, Google has shown AI-written summaries above many results, and a full conversational AI Mode beside them. ChatGPT and Perplexity search the web and answer with citations. For a question like “how many ounces in a cup”, the reader gets the answer without clicking anything.

That is not new in kind. Featured snippets and knowledge panels have answered simple questions on the results page for a decade. AI answers extend it to longer questions, and they make the cited sources more visible than an ordinary blue link ever was.

What did not change

An AI answer is only as good as the pages it draws on, so the systems still have to find pages, fetch them and decide which ones to trust. Google says plainly that to appear as a supporting link in its AI features, a page has to be indexed and eligible to show in Search with a snippet. There is no separate AI index to submit to. The technical basics decide eligibility the same way they always did:

The page answers 200, is not blocked in robots.txt, and does not carry a noindex.
Other pages on the site link to it, so it can be discovered.
Its title and opening say what it is for.
The content is in the HTML the server sends, not assembled later by scripts.
Other sites cite it, which is still how anything decides whom to trust.

Which pages lose clicks, and which do not

Lose: definitions, conversions, quick facts, and generic “what is X” articles that restate what is already everywhere. If an AI summary can replace your page in two sentences, it will.

Keep: anything local (“near me”, opening hours, a service in a town), anything transactional (prices, stock, booking), comparisons with real detail, tools that do a job, original data, and first-hand experience. And searches for your own name.

So the strategic change is to put your effort into the second list, and to make sure each of those pages says the thing only you can say in its first paragraph.

What to do differently

Check robots.txt for AI crawlers, deliberately. Many sites block them by accident, through a template or a security plugin, and many block search crawlers when they meant to block training crawlers. Read the file and decide per crawler. The robots.txt tester shows how a file's groups and rules resolve.
Put the content in the HTML. Many AI crawlers fetch a page and read what the server sent, without running JavaScript. A scan here reads pages the same way, so a page that comes back with no text and no headings in the scan is likely invisible to them as well.
Answer first. State the answer in the first paragraph, plainly, then explain. It is what a reader wants, and it is the sentence an AI system is most likely to quote.
Be specific. Numbers, prices, dates, steps, named places and products. Specifics are what a summary needs a source for; generalities it can produce on its own.
Keep measuring clicks per query. If a query's impressions hold and its clicks fall, the answer is now on the results page. Either the page has more to offer than that answer, or the effort is better spent elsewhere.

Questions people ask

Does Google penalise AI-written content?: Not for being AI-written. Google's published position is that it rewards helpful content however it was produced. What its spam policies do target is content mass-produced mainly to rank, by any method, which it calls scaled content abuse. A page that says nothing new is a weak page whether a person or a model wrote it.
Should I block AI crawlers in robots.txt?: Decide per crawler, because they do different jobs. Some collect training data (OpenAI's GPTBot, for example) and some fetch pages to answer a search right now (OpenAI's OAI-SearchBot). Blocking a search crawler keeps you out of that assistant's answers and citations. Google's Google-Extended token controls use of your content for training its models; it does not affect Google Search, including AI Overviews.
Will AI Overviews kill my traffic?: It depends on what your pages answer. A page whose whole value is one fact that fits in a sentence loses clicks when the results page states the fact. A page people need to use, compare, buy from, book or trust keeps them, and a page cited in an AI answer can win visits it would not have had. Search Console counts traffic from Google's AI features inside its ordinary web figures, so watch the trend of clicks per query rather than looking for a separate line.
Is it still worth learning SEO?: Yes. The skills that matter most now are the old ones: making a site that can be crawled and indexed, saying plainly what each page is, and being worth citing. Those are exactly what an AI system needs before it can use your page as a source.

See your site the way a crawler does

The basics that decide whether search engines and AI assistants can use your pages, checked across the whole site. Free, no account.

Scan a site, free