If you’re wondering how to keep search engines away from your WordPress blog, you’re not alone. Many folks prefer to keep their blogs private from search engines like Google, Bing, and Yahoo.
As someone who understands this need very well, I’m here to help you out. Let me share some simple steps that will allow you to ensure your blog stays off these search engines’ radar.
But before we share the steps to discourage search engines from indexing the site, here are a few things that you need to keep in mind:
- The first thing to take care of is that doing this on a live site with a lot of traffic from organic methods is not recommended. If you do, your site will lose ranking and traffic from search engines.
- Another thing you need to take care of is to only apply these settings on sites that are still in the development phase.
But why do we discourage search engines from indexing our site?
When we’re working on a project or developing a website, we don’t want our dummy pages to get indexed by search engines – because, in most cases, it is difficult to deindex those pages, and indexing dummy or low-quality pages can be harmful to the SEO of our website.
Thus, we discourage search engines from indexing our site during this phase. Once our website is fully ready to be indexed, we remove this restriction.
Having learned about when to use this option for deindexing our site from search engines, now we’ll learn…
How to Discourage Search Engines from Indexing this Site?
There are several methods you can use to discourage or prevent search engines from indexing your website. In this guide, we will share only the easiest ways to stop search engines from indexing your site, which include:
- Discourage Search Engines Using the WordPress Admin Panel
- Discourage Search Engines Using Robots.txt File
Here are some of the simplest ways to prevent your site from being indexed by search engines.
Next, we’ll share the steps you can take to easily stop your site from being indexed by Google or any other search engine using the methods previously mentioned.
1. Discourage Search Engines Using WordPress Admin Panel
So the first method we’re going to cover in this article is how to discourage search engines using the WordPress admin panel. This step is one of the easiest ways to completely deindex your site from Google or any other search engine.
You have to log in to your WordPress admin panel and then click on the “Settings > Reading” option (as shown in the image below).
Once you’re on the “Reading” page of the settings, you’ll see the “Search engine visibility” option at the bottom.
Just check the box next to “Search Engine Visibility” and then click the “Save Changes” button displayed in blue.
Once you click “Save Changes,” the adjustments will be applied to your site, and it will be completely deindexed from Google and other search engines until you reverse the changes.
2. Discourage Search Engines Using Robots.txt File
Discouraging search engines using robots.txt is another method preferred by bloggers and developers. This method is simple but requires a bit of technical knowledge. If you’re comfortable with technical tasks, you might also prefer this method.
In this method, you need to log in to your cPanel (or any other control panel you have) and then open the File Manager > public_html folder.
Once you’re in the public_html folder, create a file named “robots.txt.” This is essentially a text file that a webmaster creates to instruct Google and other search engines on how to crawl pages on their website.
Once the file is created, add the following lines to discourage search engines from indexing your site.
Pro Tips: Some SEO plugins, such as Rank Math and Yoast, create a robots.txt file for you. You can easily find them on the “Settings” page of your tool.
For example, in Rank Math, you can find the robots.txt file under “Rank Math > General Settings > Edit Robots.txt“.
If you want, you can also include the code here, and it will dynamically create the robots.txt file for you with the code we shared above. 🙂
Bonus: How to Discourage Search Engines from Indexing Specific Posts
So far, we have learned how to prevent search engines from indexing the entire site. But how can we prevent search engines from indexing specific posts?
There are a few methods to discourage search engines from indexing specific posts. The simplest way is to use SEO plugins such as Rank Math or Yoast SEO to disable indexing on specific posts.
You just need to open the post you want to noindex in the WordPress editor, then click on the “Advanced Settings” of your SEO tool and set the “Robots Meta” to “noindex.”
Once you check the “noindex” option in the “Robots Meta” box, it immediately adds a noindex meta tag to the post. Consequently, Google or any other search engines can’t index those pages.
You can tell search engines not to show some of your blog posts in their search results by using the robots.txt file, just like we explained earlier. I’ve guided many people on how to do this, and I can help you with it too.
Wrapping Up!
I hope you found the article useful where I explained how to stop search engines from showing your site or certain WordPress posts. The steps I’ve shared are simple to use, but I would say don’t try them on a website that’s already popular on Google or other search engines.
If you block search engines from finding your site, you’ll lose your website’s ranking. It will also take quite a bit of time to get your site indexed and ranked again. So, you should only use these tips for a website that’s still being set up, or for posts that you don’t want to show up in Google, Bing, or any other search engines.
