Yesterday, webmasters across the internet began receiving scary-sounding warning notices from Google Search Console informing them that Googlebot was unable to access certain CSS and JavaScript elements due to robots.txt rules.
How To Quickly Fix ‘Googlebot Cannot Access CSS and JS Files’ Warning in Search Console
Google’s own Gary Ilyles posted a quick and simple method of resolving this issue on StackOverflow yesterday, and the fix is surprisingly quick and easy to implement. According to Gary, “simplest form of allow rule to allow crawling javascript and css resources” is to add the following lines to your robots.txt file:
User-Agent: Googlebot
Allow: .js
Allow: .css
These few lines of code will allow Google’s crawling bots explicit access to all CSS and JavaScript elements to ensure that these site assets are fully visible and indexable. NOTE: Implementing these allow rules are superseded by any disallow rules, meaning the above lines will not solve any blocked elements issues, and any directory or structural rules are still applied.
Google Looks at CSS and JS for Homepage and Mobile View
John Mueller of Google stated on Google+ that “…when Google checks for blocked CSS and JavaScript assets, they don’t go too deep into your site. They primarily look at your home page and then the mobile/smartphone view of your web site.”
Google Doesn’t Care About Third Party Embedded CSS/JS Elements
In that same Google+ post, John Mueller also cleared the air surrounding third party/offsite elements, saying that the notification shouldn’t have been sent out to users for blocked CSS/JS elements that are offsite:
We’re looking for local, embedded, blocked JS & CSS. So it would be for URLs that you can “allow” in your robots.txt, not something on other people’s sites (though blocked content on other sites can cause problems too, eg, if you’re using a JS framework that’s hosted on a blocked URL).
How Many Websites Received the Blocked Elements Notification?
John Mueller stated on Twitter that Google only sent out about 18.7%, or roughly 1/5, of the number of alerts they sent out when they began including mobile usability warnings – so if you survived through the mobile usability changes and came out on the other side alive, chances are your website will rebound from this without a scratch.