How to diagnose, repair, and monitor your Google Search Console health — step by step.

How robots.txt Blocking wp-includes Breaks Google Rendering

A robots.txt line like Disallow: /wp-includes/ looks like cleaning up technical clutter. It is actually one of the most damaging lines a WordPress site can carry: Googlebot cannot fetch the CSS and JavaScript it needs to render your pages, so it indexes what a browser with no stylesheet would see.

Quick answer

Fetch /robots.txt and look for Disallow rules covering wp-includes or the admin asset paths inside wp-content. Remove them — blocking these directories was never needed for security and breaks rendering. If your robots.txt is a physical file on the server, edit the file; a WordPress robots filter will not touch it. Then re-validate a key URL in the URL Inspection tool.

Why it happens

The DIY fix

  1. Curl /robots.txt (browser UA on shared hosts).
  2. Remove Disallow lines covering wp-includes and required asset paths. Keep sensible blocks like wp-admin (except admin-ajax.php) and search pages.
  3. Check whether a physical robots.txt file exists on the server; if so, edit that file — WordPress-level robots filters will not override it.
  4. Validate with the URL Inspection tool's rendered view and confirm the CSS/JS now load.

The Brambyl route

The robots asset-block check flags these lines as critical, the repair handles both the filter path and the physical file with a backup of the original, and the diagnosis explains which lines are safe to keep so you do not strip real security blocks by accident.

FAQ

Doesn't blocking wp-includes improve security?

No. wp-includes serves public assets. Blocking it does not protect anything and actively harms crawling. Real hardening happens elsewhere (logins, file permissions, WAF).

Why does the site look fine to me but broken to Google?

Your browser caches assets. Googlebot fetches raw and obeys robots.txt literally. The rendered-source check in Search Console shows you Google's view.

Is this related to my sitemap?

Indirectly: robots problems reduce crawl confidence and contribute to the discovered-not-indexed state.

Related guides