In addition to guest posting on the UpCity blog, SocialSEO is featured as one of the Top SEO Agencies in the United States. Check out their profile here.

If you’ve worked within web development or digital marketing for a while, odds are you’ve heard horror stories of the noindex tag. Often used when building a new site, a noindex designation keeps your site from being indexed, meaning it will not appear within Google or search results. Forgetting to remove the noindex tag after the site goes live can prevent a massive number of users from ever finding it. For one simple line of code, the noindex tag can do a great deal of damage and cost you serious bucks if improperly used.

Selectively noindexing pages, however, can greatly increase your visibility within search engines.

Sounds counterintuitive, right? How can showing less of my site on Google result in it being shown more within results?

Understanding How Google Reads Your Site

Before that, some background… Google doesn’t see sites the same way you and I do. While the Upcity homepage looks like this to us:

Google sees this:

Now, Google doesn’t have real flesh and blood human beings reading your site; they’re too busy making self-driving cars and speakers that order you pizza. Instead, they send of billions of bots, sometimes called spiders, to crawl your site in their place. Every time these spiders arrive on your site, they read every line of code on the page, following each link they come across. These spiders aren’t patient, however… they only have a limited amount of time and memory allocated for each visit to your site, meaning they can leave your site before getting to the good stuff if crawling takes too long. You can see how long they stay on your site and how much memory they utilize within your Search Console under “Crawl Stats” (don’t have Search Console set up? Follow this guide)

To give your site the best chance to rank, it’s vital that every page on your site has a clear link to it. Simple enough, right? Maybe not… since these Spiders follow every link they see, how they navigate your site differs greatly from how you and I would, and they can swiftly go in directions you weren’t intending. For example, below is a visual representation of Google’s crawl on a site for a business consultant, with dots representing pages and lines representing links.

See that blue rectangle? That represents pages that bring potential customers to this site: services, locations, the homepage… things like that. Everything outside of the rectangle are secondary pages like blogs and news articles. While these pages are no doubt important to the brand, look at how much of the Spiders’ resources we’re taking up on pages that do nothing to make money. By asking Google to index these pages, we are essentially saying that a news article from 2012 is as important as the homepage, diluting the value of the pages that bring business.

Enter noindex.

How Noindexing Impacts Crawling

Clearly, Google was spending too much time crawling relatively unimportant or irrelevant pages. But by placing a noindex tag on the pages outside of the rectangle, we placed more value in those top-level pages that are most likely to result in a conversion. This allows Google more time and resources to read, understand, and, in turn, rank these higher-value pages more visibly within search results. Keep in mind that noindexing pages doesn’t impact how they look or function; users on your site will still be able to read them as normal. The result of noindexing them was dramatic.

So how do you do it? Well…

!WARNING!

Remember that first paragraph? About losing money? Noindexing is like dynamite: it can do a lot of good, but if you use it improperly, you may not be walking away. It’s best to noindex a small number of pages at a time, determining the impact after 30 days, and reverting to a prior state if needed. Keep in mind that this is also not a cure-all: the site used as an example had a full SEO campaign that had been going on for six months, following all best practices. A quick-fix, this is not. This is a tool, not a silver bullet.

How to Noindex Without Breaking Everything

If you’re ready to take the dive, take a look at your Organic traffic within your Analytics over at least a twelve-month period, compared to the same time last year, and filtered by Landing Page. Take note of the following:

• Which pages no longer receive Organic traffic?
• Which pages no longer receive conversions?
• Which pages have abnormally high Bounce Rate?

Once you have a list of underperforming URLs, you’ll need to perform mini-SEO audits on each of them. For every URL on that list, answer these questions:

• Does this URL rank for any Keywords?
• Could this URL benefit from better leveraging of either existing or new backlinks?
• Could I target a more applicable Keyword on this page?
• If not, could the content be reworked to better target the keyword?
• Is this page relevant in the path to conversion for a user (informational regarding product/service, provides info not found elsewhere, etc)?

If any page on your list of URLs doesn’t receive traffic or result in conversion, has a high bounce rate, cannot be rejigged to rank better for existing or new keywords, and has few to no backlinks, the page may be a candidate for noindexing. Again, there’s no one universal checklist for this. Treat noindexing like a tough question on the SATs: we’d rather you do nothing than guess and get it wrong.

Put yourself in a user’s shoes… imagine they clicked onto that page from a search engine result. Would they be happy with their destination? Do they really need this info, or can it be found elsewhere? Combined with the questions above, if the page is not crucially imperative to the structural integrity of the site or if it contains little tangible benefit to a user, we can likely noindex it. To be safe, keep a list of URLs you’ve noindexed. Should things go sideways, you can always revert your site back to a prior state, though negative results may still linger.

There are also a number of pages that should pretty much always be noindexed, such as:

• Logins for your CMS (/wp-admin)
• “Thank You!” pages after filling out a form or making a purchase
• Printer-friendly version of pages containing duplicate content
• Result pages from internal searches
• Multiple paged articles or lists (that’s a job best left to pagination)

Once you have your final list of candidates, add this tag to each specific page you’d like to noindex:

<META NAME=”ROBOTS” CONTENT=”NOINDEX, FOLLOW”>

…and that’s it, really. Not too scary, right? The next time Google crawls the page, they’ll see the tag and remove the page from results.

You may notice there is a “FOLLOW” designation after “NOINDEX”, and that’s by design. A page removed from the index is still crawled and read by Google, meaning they still travel through links as before. By calling out “FOLLOW”, you can ensure this crawling is unimpacted, including benefits from your internal linking.

Let’s be honest… not every page on your site is important. Acting like it is may be costing you money. By only presenting pages useful to your users in the search engine results, you can improve your rankings, increase visibility, and ensure you’re not leaving money on the table.

Just do it safely.

About the author

Grant Effinger

Senior SEO Specialist at SocialSEO

Grant Effinger is a Senior SEO Specialist at SocialSEO, a full-service digital marketing firm headquartered in Colorado Springs and serving the entire country. He is fascinated with SEO (no, really), and enjoys going down the rabbit hole to find unique approaches and bleeding-edge trends for his clients to thrive. His interests include building links while listening to music about being sad, spending his disposable income on outlandish socks, losing at trivia, and writing utterly captivating bios.