July 9, 2006 at 3:32 am
#697921
Inactive
I never use to use a robots.txt file, but then I noticed many errors in my logs caused from search engines looking for one. Now even though I also do not disallow anything, I do have one which has stopped serving up an error to the robots.
I place the robots.txt in the public_html folder.
As far as the anchor text, I never link back to my home page as “index.html” I always go with the full url. (read it somewhere?)