- This topic is empty.
-
Posts
-
dont know if im missing the question here but your robots file doesnt disallow anything and its would have no effect on outgoing links, the robots.txt just tells(suggests) bots what to crawl and what not too, the rel=”nofollow” just suggests that bots dont follow the link out of your site, they will if they want to though
You should remove the “Allow: /” from your robots.txt file as it’s not necessary and even though you nofollow links, the SE spiders can still access them through toolbars, analytics, other links, etc.
You can “Disallow:” certain sections of your site but again that is only a recommendation to the search engines, not a guarantee.
Hope this helps.
Google webmaster tools recommends and suggests a robots txt file. That is why i added one. So do you say that robots txt file and nofollow links conflict each other ? Thanks
@Warren 230002 wrote:
You should remove the “Allow: /” from your robots.txt file as it’s not necessary and even though you nofollow links, the SE spiders can still access them through toolbars, analytics, other links, etc.
You can “Disallow:” certain sections of your site but again that is only a recommendation to the search engines, not a guarantee.
Hope this helps.
Do you suggest that robots txt file and nofollow links conflict ?
Thanks
@itsblitz 230001 wrote:dont know if im missing the question here but your robots file doesnt disallow anything and its would have no effect on outgoing links, the robots.txt just tells(suggests) bots what to crawl and what not too, the rel=”nofollow” just suggests that bots dont follow the link out of your site, they will if they want to though
@casinorecommender 230004 wrote:
Google webmaster tools recommends and suggests a robots txt file. That is why i added one. So do you say that robots txt file and nofollow links conflict each other ? Thanks
You should definitely have a robots.txt file. I am just telling you that “Allow: /” specifically is not needed within the robots.txt file as it’s redundant. The nofollow on a link will work in conjunction with your robots.txt file.
@casinorecommender 230000 wrote:
I am currently using Robot.txt file with (User-agent: *
Allow: /) and also for the outgoing links i use “link nofollow”.The question is : Is using Robots.txt file with this type of code disturbs the functionality of “link no follow” codes for outgoing links ?
Thnx
You need to edit your HTACCESS file on the root of your web directory. This is also the preferred Google and Yahoo method. robot.txt file is not very accurate. HTACCESS is the gate keeper the the Server, you might say. You can find information on the internet about it. I tried to post some code here for you, but this website will not allow me to put code..
I know this is not exactly what you were talking about, but your HTACCESS file is also the most accurate way to prevent search engines and other sources from archiving your pages.
Happy programming..
Fixed the issue. Many thanks for your contribution.
@Warren 230006 wrote:
You should definitely have a robots.txt file. I am just telling you that “Allow: /” specifically is not needed within the robots.txt file as it’s redundant. The nofollow on a link will work in conjunction with your robots.txt file.
