If you want to edit the robots.txt file on your blog, there are a few things you need to know. First, you need to know where the robots.txt file is located.
Second, you need to know how to edit it. Finally, you need to know what to do if there are any problems with the file. .
Where Is the Robots.txt File Located?
The robots.txt file is located at the root of your blog’s document folder.
If you are using a blog platform such as WordPress, the document folder will be located in your WordPress install directory (usually /wp-content/). If you are not using a blog platform, then the document folder may be located elsewhere on your computer (usually in your user directory).
How To Edit The Robots.txt File?
To edit the robots.txt file on your blog, open it in a text editor such as Notepad or Wordpad. The first thing you will want to do is identify the name of your blog’s domain (i.e., www.yourdomainname.com).
Next, find a line that begins with “#” and includes the name of your blog (i.e., #WordPress). You will then want to replace all instances of “WordPress” with “yourblog” on this line. This line should now look like this:.
#WordPress
Next, find a line that begins with “User-agent:” and includes the name of your web browser (i.e., Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.11). You will then want to replace all instances of “Mozilla/5.11 (KHTML, like Gecko) Chrome/19.
1084.52 Safari/536.11” with “yourblogbot” on this line (the first letter of each word should be capitalized).
User-agent: Yourblogbot
Next, find a line that begins with “Disallow:” and includes any phrases that you would like to exclude from search engines from visiting your blog (elements such as images or external links). You will then want to replace all instances of “Disallow: *” with “Disallow:yourblog” on this line (the asterisk character should be replaced with an uppercase letter). This line should now look like this:
Disallow: yourblog.