How to “Discourage Search Engines from Indexing this Site” in WordPress

Not sure how to discourage search engines from indexing your site? It’s a very common thing that people wanted to do with their WordPress blogs. Sometimes they don’t want their blogs to be indexed by search engines like Google, Bing, Yahoo, etc.

If you’re someone who also doesn’t want their blogs to be indexed by search engines and wanted to discourage search engines from indexing your site, then here in this guide, I’m sharing a few steps through which you can easily make this happens.

But before we further share the steps to discourage the search engines from indexing the site, here is a few things that you need to keep in your mind;

  • The first thing which we need to take care of is, doing this thing on a live site having a lot of traffic from Organic methods is not recommended. If you do so, your site will lose ranking and traffic from search engines.
  • Another thing that you need to take care of is, only apply these settings on sites that are still in the development phase.

But why we discourage search engines from indexing our site?

When we’re working on some project or the development of the website, we don’t want our dummy pages to get indexed by search engines – because in most of the cases, it is hard to deindex those pages from search engines, and it can be harmful for the SEO of our website to index dummy or low-quality pages.

Thus we discourage search engines from indexing our site during this phase and once our website is fully ready to be indexed, we remove this option from our website.

We have learned about when we have to use this option for deindexing our site from the search engines, now we’ll learn…

How to Discourage Search Engines from Indexing this Site?

There are many ways through which you can discourage or stop search engines from indexing your website – but in this guide, we will only share the easiest ways through which you can stop search engines from indexing your site, and these are;

  • Discourage Search Engines Using WordPress Admin Panel
  • Discourage Search Engines Using Robots.txt File

So these are some of the easiest ways to discourage your site from getting indexed in search engines.

Now we’ll share the steps through which you can easily stop your site from getting indexed in Google or any other search engines using the methods which I shared above.

1. Discourage Search Engines Using WordPress Admin Panel

So the first method which we’re going to cover in this article is discouraging search engines using the WordPress admin panel. This step is one of the easiest steps to completely deindex your site in Google or any other search engine.

You have to logged-in to your WordPress admin panel and then click on the “Settings > Reading” option (as shown in the image below).

deindex site from google, discourage search engines from indexing this site, discourage search engines in wordpress, search engine visibility wordpress
WordPress Settings

Once you’re on the “Reading” page of the settings, you’ll see an option of “Search engine visibility” on the last.

deindex site from google, discourage search engines from indexing this site, discourage search engines in wordpress, search engine visibility wordpress
“Search Engine Visibility” Option in WordPress

You just have to check on the box after the name “Search engine visibility” and then click on the “Save Changes” button showing in the blue color.

Once you click on the “Save Changes,” it will apply the changes for your site, and your site will completely get deindexed from Google and other search engines until you again make the changes.

Top

2. Discourage Search Engines Using Robots.txt File

Discouraging search engines using robots.txt is another method that bloggers and developers prefer. This method is simple, but it requires a little bit of technical knowledge, so if you’re comfortable with a few technical things, you can also prefer using this method.

In this method, you need to login to your cPanel (or any other control panel, which you have) and then open the file manager > public_html folder.

deindex site from google, discourage search engines from indexing this site, discourage search engines in wordpress, search engine visibility wordpress
Create Robots.txt File

Once you’re into the public_html folder, create a file named “robots.txt.” It is basically a text file that a webmaster creates to instruct Google and other search engines how to crawl pages on their website.

Once the file is created, add the following lines in the file to discourage search engines from indexing your site.

Pro Tips: Some SEO plugins like Rank Math and Yoast creates a robots.txt file for you. You can easily find them on the “Settings” page of your tool.

For example, in RankMath, you can find the Robots.txt file on the”Rank Math > General Settings > Edit Robots.txt“.

deindex site from google, discourage search engines from indexing this site, discourage search engines in wordpress, search engine visibility wordpress
Robots.txt Option in RankMath

If you want, you can also include the code here, and it will dynamically create the robots.txt file for you with the code we shared above. 🙂

Top

Bonus: How to Discourage Search Engines from Indexing Specific Posts

Till now, we have learned about a few ways to discourage search engines from indexing the whole site – but who can we discourage search engines not to index specific posts?

There are a few ways to discourage search engines from indexing specific posts, and the easiest method is to use SEO plugins like Rank Math or Yoast SEO to disable on specific posts.

You simply have to open the post which you want to noindex in the WordPress editor and then click on the “Advanced Settings” of your SEO tool and then set “Robots Meta” to “noindex.”

deindex site from google, discourage search engines from indexing this site, discourage search engines in wordpress, search engine visibility wordpress
Setting Up Noindex in Google Using RankMath SEO Plugin

Once you checked the “noindex” option in the “Robots Meta” box, it instantly adds a noindex meta code for the post. Thus google or any other search engines can’t index those pages.

You can also discourage search engines from indexing specific blog posts using the robots.txt file (as we shared in the steps above).

Wrapping Up!

I hope you liked this article on discouraging search engines from indexing the site and specific WordPress posts. The method we shared above is easy to follow, but I don’t recommend following the steps on a live blog or a blog that receives traffic from Google and other search engines.

If you discourage or deindexed the site from search engines, you’ll completely lose the ranking of your website, and it will again take a lot of time to index and rank your website. Only use these methods on the website which are under development phase or on the posts you don’t want to be indexed in Google, Bing, or any other search engines.

TheWPX is a blog where we cover topics related to Blogging, WordPress, SEO, and Affiliate Marketing.
All the information which is published on our website is collected through independent research and tests. The data is being verified through consultation with reliable sources.
Also, We constantly update the content on our website, keeping it fresh and relevant to our readers at all times.

Leave a Comment

Sunny Kumar

Sunny Kumar, Founder of TheWPX

Hi, I'm Sunny Kumar, a passionate tech enthusiast and blogger from New Delhi, India. I usually writes about PC building, WordPress, and SEO on my blogs. And I hold certifications in SEO, Cloud Computing, Telecom & Networking, and CEH, along with an IT degree from IIT-D.