Soft 404s Caused 90% Traffic Loss: 2026 Diagnostic Playbook

What Is a Soft 404 and Why Does Google Deprioritize Your Whole Site Because of Them?

A traditional 404 is honest: the server says “this page doesn't exist,” and Google removes it from the index. A soft 404 is a page that lies. The server says 200 OK and serves you HTML, but the HTML contains nothing useful — an empty product listing, a “no posts found” archive, a thin landing page that looks like a placeholder. Google's Search Central documentation explicitly tells site owners to return a real 404 or 410 status code for pages that no longer have content, because the lie wastes crawl budget and creates indexing ambiguity.

The Search Engine Land case study quantifies how fast this compounds. At peak, the publisher had roughly 120,000 soft 404 pages across thirteen country domains — 90,400 on the main site alone — plus 513,369 pages stuck in the “Crawled — currently not indexed” bucket in Brazil. Google was visiting those URLs and choosing not to add them to the index, which signals that the crawl budget was being spent on URLs the algorithm did not believe were worth ranking. That kind of waste pushes legitimate pages further back in the crawl queue. New articles took 24 hours to get indexed at the worst of it.

For a small business with a few hundred pages, the volumes are smaller — but the mechanics are identical. If your /search/?q=blue-widgets URL renders an empty “no results” page and returns HTTP 200, that's a soft 404. If your /category/winter-coats archive shows the “no posts” template after you cleared seasonal stock, that's a soft 404. If your Shopify collection still exists but every product inside has been unpublished, that's a soft 404. The page works for a human; the page is invisible — or worse, suppressive — for organic search.

Why is this dangerous rather than just inefficient? Because the algorithm has finite trust in your domain. Every empty, thin, or auto-generated page it crawls is a vote that your site doesn't deserve full crawl coverage. That can pull down indexing of pages you care about. The Search Engine Land case study found that fixing the soft 404 pile let indexed pages in Germany roughly double — from ~150,000 to 370,748 — without a single line of new content being added.

How Do You Diagnose Soft 404s in Google Search Console in 60 Minutes?

The diagnostic flow lives almost entirely inside Google Search Console's Page Indexing report. Here is the order we run it for clients, including a small Fort Wayne service-business site we audited last quarter.

Step 1: Open the Pages report and sort by “Why pages aren't indexed.” The report groups every URL Google has discovered into reasons. The buckets you want to focus on are “Soft 404,” “Crawled — currently not indexed,” “Discovered — currently not indexed,” and “Duplicate without user-selected canonical.” Together those four buckets account for most crawl-budget waste in small business sites. Click into each one and look at the URL patterns — soft 404 buckets usually surface obvious culprits like /?s= search results, /tag/ archives, or /collections/all variants.

Step 2: Run the URL Inspection tool on three suspicious URLs from each bucket. The URL Inspection tool is the highest-signal check available in Search Console. Paste in a flagged URL, then click “Test live URL” → “View tested page” → “Screenshot” and “HTML.” If the rendered screenshot is blank, missing the main content, or shows a “no results” template, Google is rendering exactly what we expected — and it's classifying that render as soft 404 for a reason. Sometimes the issue is JavaScript: the page works in a browser but renders empty in the headless Chrome that Googlebot uses.

Step 3: Pull the Crawl Stats report under Settings. The Crawl Stats report shows how many requests Googlebot makes per day and which response codes it gets back. If your “Average response (ms)” is climbing while “Total crawl requests” stays flat, you're often spending budget on slow URL patterns — a common signal of parameterized soft 404 traps. The case study tracked crawl rate drops from 60,000–70,000 daily requests to 20,000–30,000, which was the first measurable signal that Google had lost faith in the domain. See also Google's crawl-budget management guide for larger sites.

Step 4: Search your own site with site: queries. A site:yourdomain.com inurl:? query in Google often surfaces the parameterized garbage your CMS is exposing — search results pages, sort/filter parameters, session IDs in URLs. If you see dozens of variants of the same canonical page, you have a duplicate-content + crawl-budget problem masquerading as a soft 404 problem.

Step 5: Cross-check Bing Webmaster Tools. Bing's Site Explorer tends to surface different indexing anomalies than Google, and it's free. We often find pages indexed in Bing but rejected by Google — the gap usually points to a JavaScript-rendering issue Google catches and Bing doesn't.

For a typical small business site with under 1,000 URLs, this five-step flow takes about an hour. The output is a list of URL patterns that are either soft 404s, crawl-budget waste, or both. That list is the input to the fix step.

What Causes Soft 404s on WordPress, Shopify, and Next.js Sites?

Each major small business platform ships its own flavor of soft 404, and the diagnostic shortcut is knowing which patterns to look for first. We see the same offenders repeatedly across Northeast Indiana clients and out-of-market builds.

The Next.js pattern is the one we see fixed least often. Modern frameworks default to soft 404s because returning a “not found” component with a 200 status code is easier than wiring up notFound: true properly. When we audit Next.js builds for clients we look for two specific signals: (1) the /404 route returning HTTP 200 when accessed directly, and (2) dynamic routes like /products/[slug] rendering a “Product not found” component without setting the right status code. Both are silent and both kill organic traffic over time. Our web performance optimization guide covers the related Core Web Vitals angle that often compounds the problem.

WordPress sites have a different recurring failure: the WPML or Polylang language switch creates duplicate URLs (/en/ versus /) where one variant ends up serving empty templates after a theme upgrade. The case study publisher hit this at scale on a domain migration; small business sites hit a smaller version of it constantly.

How a Fort Wayne Small Business Owner Can Run This Audit Without a Developer

We get asked this constantly: can a non-technical owner actually do a soft 404 audit, or does it require a developer? The honest answer is that the diagnostic phase is fully doable from the Search Console UI in an hour. The fix phase usually needs someone who can edit your CMS, but most small-shop CMSes give you the tools.

Picture a typical Auburn, Indiana small business — say a specialty retailer that runs a Shopify store and a WordPress blog. The owner notices her organic traffic has been drifting down for six months. She logs into Search Console, opens the Page Indexing report, and sees 142 URLs in the “Soft 404” bucket. About 90 of them are /search?q= URLs from her store's internal search. Another 30 are /collections/winter-2024 and similar seasonal collection URLs she emptied months ago. The remaining 22 are blog tag pages with one or zero posts.

That's a clean diagnosis with three concrete fix paths. In Shopify she can add a <meta name="robots" content="noindex"> to the search results template through theme code (or ask her theme support to do it). The seasonal collections she can either redirect (301) to the current season's collection or unpublish and let them return real 404s. The thin tag pages she can either consolidate (merge tag/winter-coats and tag/coats-winter into one canonical tag) or noindex if they're not getting traffic. The whole project is a couple of hours of work, mostly checkbox-style configuration inside the platforms, not custom code.

The local nuance that matters here: small business owners in Fort Wayne, Auburn, and the broader Allen County / DeKalb County area tend to run on Shopify, WordPress, or Squarespace, not headless stacks. That's good news. Those platforms expose noindex controls, sitemap controls, and redirect controls in plain UI. If your site is on one of them, you can ship most of the fix yourself. If you're on a custom or Next.js build, that's when we recommend looping in a developer — and at Button Block, that's where our web development team gets involved.

One honest caveat: the Search Engine Land case study makes clear that recovery is slow even when fixes ship correctly. The publisher needed ~12 weeks to drop soft 404 errors 69% in Brazil. And total traffic only recovered to 25–35% of the original forecast even after the indexing crisis was resolved. The site that loses 90% of its traffic doesn't get all of it back. The bigger lesson: fix this early. Soft 404 prevention is cheaper than soft 404 remediation by an order of magnitude.

How Do You Actually Fix Soft 404s Without Making Things Worse?

The fix flow has three priorities, in order: return correct status codes, reduce crawl-budget waste, and protect legitimate pages from getting reclassified. Skipping any of those in sequence is how teams accidentally deindex their own product catalog while trying to clean up soft 404s.

Priority 1: Return correct HTTP status codes. If a page genuinely has no content and no plan to get content again, return HTTP 410 (Gone) or 404 (Not Found) — both behaviors are documented in Google's HTTP and network errors guide. 410 tells Google “remove this permanently and don't bother re-crawling.” 404 tells Google “this isn't here right now” and the URL stays in the crawl queue longer. Use 410 for retired products and pages you'll never republish; use 404 for genuinely transient missing content.

Priority 2: Remove auto-generated empty pages or noindex them. Internal search results pages, sort/filter parameter URLs, and empty archive pages should not be in the index. Add <meta name="robots" content="noindex"> per Google's noindex documentation or block via robots.txt if you can't change the page template. Don't do both — robots.txt blocking prevents Google from seeing the noindex tag, which means the URL can stay in the index as a URL-only entry.

Priority 3: Reinforce canonicalization for variant pages. If you have duplicate variants of the same content (paginated archives, tracking-parameter URLs, sort variants), make sure each variant points its rel=canonical to the master version. The Search Engine Land case study found 2,532 “Alternate page with canonical tag” entries in Search Console — most of which were resolved correctly by tightening canonical implementations.

Priority 4: Plug the leak. Every fix above is reactive. The proactive piece is ensuring your CMS, theme, or build can't generate the same problem again. For WordPress that means a permanent noindex on search results, tag pages with fewer than X posts, and author pages with no published posts. For Shopify that means a robots.txt rule for filter parameters. For Next.js that means a code review of every dynamic route to confirm it returns notFound: true when the data isn't there. For all platforms it means a quarterly check of the Page Indexing report so the next pile of soft 404s gets caught at 5 pages, not 90,400.

We also recommend pairing this with a content decay audit. Thin and stale pages don't return soft 404s on their own — but they're adjacent to the same algorithmic judgment about whether a page deserves index space. The two clean-up flows reinforce each other.

How Do Soft 404s Affect AI Search and AEO Citations?

This is the question we get most from clients running answer engine optimization programs alongside traditional SEO. Short answer: soft 404s hurt AI citations roughly the same way they hurt Google rankings, but the mechanism is slightly different.

Google's AI Overviews and Search Generative Experience pull from pages that are indexed in Google's main index. If your page is sitting in the “Crawled — currently not indexed” bucket — which is what soft 404s and thin-content pages often look like — it cannot be cited in AI Overviews because it never made it into the candidate set. The case study publisher noted that their Google Discover share grew significantly only after the indexing crisis was resolved, which is the same dependency: AI surfaces sample from the indexed corpus, not from the crawled corpus.

For ChatGPT, Perplexity, and Claude with web search, the dependency is similar but not identical. Those systems use third-party crawlers and indexes (e.g., Bing for Copilot, various retrieval-augmented setups for the others) and the same soft 404 hygiene rules apply. If your page renders empty to the crawler — whether that's Googlebot, Bingbot, or GPTBot — it's not going to be retrieved or cited.

There's a second-order effect we see less discussed: soft 404 pages can poison the rest of your site's authority signals. Internal links pointing into thin/empty pages are essentially wasted authority — you're telling the algorithm to spend trust on a page that, on inspection, contains nothing. Cleaning up soft 404s by either fixing the content or removing the page reroutes that internal-link authority to pages that actually rank and cite. That's the part most “fix your soft 404s” guides miss.

The Small Business Soft 404 Prevention Checklist

The diagnostic-and-fix flow above is the reactive playbook. Once you've shipped the cleanup, the next step is making sure you don't re-accumulate the same pile in six months. Here's the recurring checklist we run with retainer clients:

• Monthly: Open the Page Indexing report in Search Console. Skim the “Why pages aren't indexed” section. Note any new URL patterns in Soft 404 or Crawled–not indexed. Investigate any pattern that appears in more than five URLs.
• Quarterly: Re-run a site:yourdomain.com review of indexed URLs. Look for unfamiliar parameter URLs, search-results URLs, or paginated archives that shouldn't be there.
• Pre-launch (any CMS upgrade, theme change, or migration): Validate that 404 and 410 status codes still return correctly. Confirm sitemap.xml doesn't include URLs that have been removed. Run Screaming Frog or Sitebulb against staging before push.
• Post-launch: Watch the Search Console Pages report for two weeks. Any spike in soft 404s in that window means something in the migration broke status codes for a URL pattern. Fix immediately — recovery from a delayed catch is much slower.
• Annually: Audit redirects. 301 redirect chains accumulate over time and can themselves trigger soft 404 classifications if intermediate URLs serve thin content.

For most small business sites under 5,000 pages, that recurring cadence — about an hour per month plus a longer audit before any major CMS change — keeps the soft 404 pile under control indefinitely.

How Button Block Helps Fort Wayne and Northeast Indiana Sites Avoid Traffic Collapse

We've been running technical SEO audits for small businesses across Auburn, Fort Wayne, and the broader Allen County / DeKalb County corridor for years, and the soft 404 audit is the single highest-ROI check we do. It's free to start, requires no platform changes to diagnose, and the fixes are usually configuration work — not custom development. If your Search Console traffic has been drifting downward without an obvious cause, the Pages report is the first place we look.

If you'd rather not run this audit yourself, our SEO services team handles the full diagnostic flow — Pages report walkthrough, URL Inspection sampling, crawl-budget review, and a written remediation plan tied to your platform. For Next.js and headless builds where the fix involves code changes, we loop in our web development team directly. Most audits ship in under two weeks, and the fixes typically pay for themselves in restored organic traffic within a quarter.

Sources & Further Reading

1. Search Engine Land: Indexing Issues That Caused a 90% Traffic Drop — and How We Fixed Them — The 2026-05-12 case study that motivates this playbook.
2. Google Search Central: Soft 404 errors documentation — Official guidance on why and how to return real 404 or 410 status codes.
3. Google Search Console Help: Page Indexing report — The primary diagnostic surface for soft 404 buckets.
4. Google Search Console Help: URL Inspection tool — Used in Step 2 of the diagnostic flow to confirm Google's render.
5. Google Search Central: Large site owner's guide to managing your crawl budget — Reference for why soft 404s compound the crawl-budget problem.
6. Google Search Console Help: Crawl Stats report — Used in Step 3 of the diagnostic flow.
7. Bing Webmaster Tools: Site Explorer documentation — Free cross-check for indexing anomalies Google may not surface.
8. Google Search Central: Block indexing with noindex — Reference for safe noindex implementation.