Solving “Soft 404” in Google Search Console

A few months ago we decided to upgrade our website’s Nuxt version from 2 to 3, and what seemed like a straightforward task with minimal side effects ended up becoming a problem that we wouldn’t be able to solve for months to come.<br><br>

In the course of a week after we bumped Nuxt to version 3.15.4, the number of indexed pages on Google Search Console dropped to less than half of what it was before – from around 400 pages to less than 200. That wasn’t supposed to happen, not at all.<br><br>

Our first instinct was to focus our attention on the things that are more closely related to GSC, like the sitemap and robots.txt generation. We first thought that maybe the sitemap was not including the pages that were not indexed, but after double-checking it, we realized that the sitemap was fine. After that we decided to look into the robots.txt file because we thought that maybe we were “disallowing” some pages by mistake, but again, to our surprise, the robots.txt file didn’t seem to have any problems.<br><br>

We tried a few other tweaks to deal with the issue, and of all the things we tried, zero had worked, and that’s when it came to us, the realization that this problem would take much longer to solve than any of us had anticipated.

There are several reasons why a webpage may fail to be properly indexed by Google, including:

・Rendering errors
・Duplicate content
・Extremely short or low-quality content

These issues are relatively common and, in most cases, can be resolved without much difficulty.
For example, if a page fails to render due to a server error, identifying and fixing the underlying issue will allow the page to load correctly and be indexed again. If duplicate content is the problem, revising the content to ensure uniqueness or adding a canonical tag can signal to Google which version should be indexed. Likewise, if a page is considered too thin or low-quality, improving its content can increase the likelihood of proper indexing.

However, as we worked through various indexing issues in Google Search Console (GSC), we encountered one problem that was particularly confusing—one that seemed almost impossible to resolve:

the “soft 404” error.

In GSC, a soft 404 refers to a situation where a page does not return an actual 404 Not Found status, yet Google determines that the page has not been properly loaded or does not provide meaningful content.
The challenge is that GSC does not clearly indicate the cause.
As a result, identifying the root issue requires extensive manual investigation.

About a week after upgrading to Nuxt 3, one of our team members raised a critical observation:

Could the GSC issues and search result anomalies be caused by the Nuxt 3 upgrade itself?

Soon after, we discovered another problem—some pages were appearing in search results with error messages. Surprisingly, Google can sometimes index pages even when they are clearly returning errors.

For instance, imagine a case where the homepage fails to load due to a server-side issue and displays a “500 Server Error.” Instead of recognizing this as a failure, Google may index the page as-is.
As a result, users searching for our site would see:
* “500 Server Error” as the page title
* Error details displayed in the meta description
And unfortunately, this issue was affecting our homepage, the most critical page of the site.

Naturally, resolving this became our top priority due to its potential impact on both user perception and SEO performance.

We spent not days or weeks, but several months investigating the issue—testing dozens of potential solutions.

At one point, we came across a Reddit post from an engineer describing a problem identical to ours. Their proposed solution was something we had never considered before.

According to the post, the issue was related to JavaScript chunks generated by Nuxt.

Chunks are small JavaScript files (e.g., 7vEvX3qN.js) stored in the _nuxt directory. They are essential for performance, as they allow pages to load only the code they need.

However, the problem occurs during crawling:

Googlebot attempts to download all required files, but fails when there are too many chunk files (e.g., 30+).

When this happens:
* Some chunks fail to load
* The page attempts to reference missing files
* Errors occur during rendering

This issue only happens when Googlebot crawls the page— not when users access it normally.

Even more confusing, GSC’s Live URL Test showed no issues.
From our perspective, everything looked fine.

But in reality, errors occurred during crawling, and those errors were reflected in search results.

The Reddit post suggested a simple workaround:
Combine all JavaScript chunks into a single file.

build: {
  rollupOptions: {
    output: {
      inlineDynamicImports: true
    }
  }
}

With this approach:
* Googlebot only needs to load one file
* Rendering errors are less likely

And indeed, within days:
* The homepage no longer displayed errors in search results
* Soft 404 pages began to be indexed

Technically, the issue was resolved.

However, this introduced a new problem:
Severely degraded performance.

Loading a single large JavaScript file significantly slowed down page load times across the site.

We continued searching for a better approach—and eventually discovered a key insight:
The issue was not that pages were loading chunks. The issue was that Googlebot was trying to crawl the chunk files themselves.

In other words, the problem was not on the page level—but at the crawler level.

The solution was straightforward:
Prevent Googlebot from accessing the _nuxt directory.

This was achieved by adding the following rule to robots.txt:

robots: {
  groups: [{
    userAgent: ['*'],
    disallow: [
      '/_nuxt/',
    ],
  }]
}

With this configuration:
* Googlebot ignores chunk files * Rendering errors no longer occur * Pages can be crawled correctly

After implementing this fix, we:
* Re-enabled chunk splitting
* Removed inlineDynamicImports: true
* Monitored site performance and indexing

The result:
* No recurrence of soft 404 errors * No issues in search result display * Performance fully restored

It has now been over two months since resolving the issue. * No recurrence has been observed * Remaining GSC issues are manageable and understood

This problem was particularly difficult because: * The cause was not obvious * It only occurred during crawling * Standard debugging tools showed no errors

Even with over a decade of frontend experience, it took us nearly four months to fully resolve.

Final Note

If you are facing similar issues, we hope this experience can help guide your investigation. We ourselves were only able to reach a solution thanks to shared knowledge from the engineering community. By sharing this, we aim to contribute back to that ecosystem—and hopefully save others from the same long debugging process.