[SEO] _next Folder Being Crawled by Search Engines

[abhishekmardiya]

Summary

I’ve noticed that the _next directory is being crawled by search engines. Since this folder contains build assets (JS, CSS, chunks, etc.), I believe it shouldn’t be indexed.

From an SEO perspective, allowing _next to be crawled may:

• Waste crawl budget
• Expose unnecessary internal build files
• Potentially create duplicate or irrelevant indexed URLs

However, I asked AI about this and received the following explanation:

Search engines crawl the /_next/ folder because your HTML pages reference those JavaScript files, and Google needs them to properly render your site. These files are static build assets, not actual web pages, so it is completely normal for them to appear as “Crawled – currently not indexed” in Search Console. They are required for rendering but are not meant to be indexed. You should not block /_next/ in your robots.txt, as doing so could prevent Google from rendering your pages correctly and potentially harm SEO. This behavior is expected and not an issue.

However, I’m not fully confident about this explanation and would like confirmation from the community.

Additional information

No response

Example

No response

[icyJoseph]

I guess, as long as these don't return HTML, then it should be fine. Like, the bot fetches that JS to run the page, but it doesn't go and index it. Reverse the question, can you search a page by using "actual code" included in that page - AFAIK, nope.

Not a bad idea to use a robots.txt though, it has a use case of course. Keep _next/static reachable though.

Some SEO gurú expert might overrule me here though.

[ikytai]

@icyJoseph, in your opinion, Do googlebot these days rely on _next/data files to render pages? I can see Googlebot is crawling VERY heavily this type of content on a next.js website. Blocking them in Robots.txt will break the pages?

[mmehsanali]

I am facing the same issue on my website, https://www.unionlimousine.com/.

We are using Next.js, and Google Search Console is reporting several URLs with /_next/ related paths. I have also noticed unexpected /_next/ requests being crawled and indexed attempts showing up in reports.

Has anyone found a reliable solution for preventing /_next/ assets from causing indexing or crawling issues? We have already reviewed our robots.txt and sitemap configuration, but the problem still persists.

If anyone has successfully resolved this issue in production, could you please share:

• Your Next.js version
• Any robots.txt changes
• Middleware or redirect rules that were updated
• Any Google Search Console fixes that helped

Thanks in advance. Any guidance would be appreciated.

For Next.js websites, /_next/ is an internal framework directory used for JavaScript chunks, CSS files, images, and other build assets. It is normal for crawlers and browsers to request these files. Issues usually come from middleware, redirects, robots.txt rules, or Google indexing reports rather than the /_next/ folder itself.