Skip to main content
Insights

Why Open Web Content is Now Just Free Training Data

The open web is dead.

For about fifteen years, B2B brands had a workable arrangement with search engines. You spent real money producing something useful, and in return you got traffic. Not a great deal, not a bad deal. Just a transaction. That arrangement is over. The terms got rewritten while nobody was looking, and your content budget is now effectively a research grant to companies that sell AI.

Publish a thoughtful piece on the open web today and something like the following is likely to happen. An AI crawler will ingest it in under few second, boils it down to three or four sentences inside a search summary, and the user never leaves the platform they queried on. You paid for the work/research. They kept the audience. You don't even get to know who they were.

Our own estimates with over 500+ google search console properties suggest that less than 25% of searches on desktop now result in a click. This means over 75% searches have zero clicks.

People keep framing this as a temporary glitch in SEO. It isn't. The open web, as a place where B2B companies can reliably acquire customers through long-form thought leadership, has stopped working.

The old content marketing math worked because of distribution at scale. Spend three grand on a serious guide about procurement bottlenecks in mid-market enterprises, and you could reasonably expect thousands of organic visits across a two-year stretch. Some fraction of those readers entered a funnel. The unit economics held up.

What broke is the distribution side. Search engines aren't really search engines anymore. They're answer engines. That single change collapses the model.

SparkToro's click data shows 56% of Google desktop searches now end in zero clicks. Someone types a query, reads the AI summary or the knowledge panel, and bounces. On mobile it's past 70%. AI Overviews are pulling informational click-through rates down by as much as 40%.

There's a cruel little irony in this. The cleaner and better-structured your writing is, the easier it is for a language model to digest, rephrase, and serve to someone who will never know your name. You're paying writers to train your competitor's product.

When more than half of searches generate zero outbound traffic, the content marketing P&L doesn't shrink. For most B2B companies, it goes to zero.

The search results page has stopped being a directory of places to go. It's a destination built to keep you from going anywhere.

This is bigger than Google's layout choices. People are searching differently. Gartner is forecasting a 25% drop in traditional search volume, with users shifting toward conversational interfaces. ChatGPT cracked the top ten desktop search destinations. Gemini and Claude are pulling their own weight.

The query itself has changed. Nobody types "top 10 CRM systems" anymore. They type something like: "I run a 50-person logistics firm on a custom legacy database. What CRM should I pick and how rough is the migration going to be?" That gets one synthesized answer. Ten blue links sitting below are basically furniture at that point. The user isn't scrolling.

The optimistic counter from the search-industry crowd goes like this. Yes, total click volume is down, but the people who do click through citation links in an AI answer are insanely qualified. High intent. Deep in the buying cycle. Search isn't dying, it's trading volume for quality.

I want this to be true. In a few narrow technical niches, it probably is. A developer chasing down a specific API integration pattern might genuinely click through to read your docs.

But run the numbers. If you go from 20,000 monthly organic visits to 1,500, no realistic conversion rate lift will save your pipeline. You cannot run a growth engine on the small slice of people who bother to expand citation dropdowns. And the bigger problem is the trade itself: you're handing over 100% of your intellectual property in exchange for maybe 5% of the visibility. That isn't marketing. That's liquidation.

If the open web doesn't work as a channel anymore, the move isn't to stop writing. It's to put the writing somewhere else.

The best thinking lives behind some kind of wall. Not the old gated PDF whitepaper move, because nobody fills out a form for a PDF anymore. I mean structural walls. Things a crawler cannot ingest. Newsletter platforms, private communities, interactive tools that require a login, proprietary data that lives inside a product rather than a blog post.

The shift is from search-dependent distribution to owned audiences. When somebody reads your analysis inside your newsletter or your community, they're encountering your actual brand. Not an AI's compressed version of it.

The goal isn't getting indexed. The goal is building an audience that knows to come to you directly and skips the search bar entirely.

If you're a CMO or founder watching the organic traffic chart slope downward, here's what an honest audit looks like.

Start with the synthesizable content. Walk through your top organic pages. If a smart 22-year-old could give a passable answer to the question in two paragraphs (think "what is SaaS churn"), that traffic isn't coming back. Stop updating those posts. Don't delete them, just stop pouring money in.

Then get your real assets off the open HTML web. Proprietary benchmarks, original research, frameworks you actually built, none of that belongs on a public blog. It belongs in a tool, a portal, a subscriber-only environment. If a crawler can't read it without authentication, an LLM can't repackage it for your prospects for free.

Last move is harder. Change what you measure. Search rank and sessions are turning into vanity metrics. Start tracking how often your brand surfaces in conversational responses across ChatGPT, Claude, and Gemini. Move your KPIs toward direct subscribers, returning users inside your owned products, reply rates on your owned channels.

The internet is splitting into two places. There's the open web, rapidly turning into a noise floor of AI-generated filler that mostly exists to feed other AI systems. And there's the closed web, where actual practitioners talk to actual humans.

If you're still publishing your best work into the open, hoping Google rewards you for the generosity, you're not playing the game. You're the inventory.

John Rambrandt
Written by

John Rambrandt

I’m a computer engineer who spends my time building for the Web and contributing to open source, but I’ve always been obsessed with how things break. I treat cybersecurity as a puzzle that involves both code and people, blending technical defense with a deep dive into the psychology of social engineering. My goal is simple: build tools that are open, scalable, and actually resilient against real-world threats.