Google Update Anyone Hit

Google bot is ignoring my robots.txt file and crawling in directories it specifically is disallowed. I have a bad-bot trap, and the ip’s were being logged and automatically blocked. I had to turn that off when I seen google bot in there.

Ignoring world wide standards means one of three things. Either google is busted, google is “do as we say not as we do,” or I’ve lost my mind and don’t know what the hell I’m doing anymore.

I guess it could be all three too.

Maybe I’m butting in where I shouldn’t but I figured I would give it a shot.

Here’s Google’s terms on the use of Robots.txt files.

http://www.google.ca/support/webmasters/bin/answer.py?answer=40364

It’s pretty interesting stuff. I hope it helps.

As for the dance.. It’s always incredibly unpredictable and hurts so many. Unfortunatly, that’s what happens when most people only use one or two search engines. It makes or kills your business is an instant. ” title=”” class=”bbcode_smiley” />

Google haven’t targeted you or anyone else – these spam sites have simply been created with metrics that make their sites appear better than yours.

Theres no mystery – and unfortunately with the level of cash floating around in this business it’s not a one-off thing either – you’ll have to grow other sources of traffic too.
:huh:

Take this one example, for keywords “poker sites”. The 1st page alone (out of 10 entries) yields 4 spam sites (all the same person, I’m sure):

espacioblog.com/myfiles/casino/poker-28.html

xoomer.alice.it/gambling777/poker-27.html

m4l.berlios.de/pub/Main/GamblinG/poker-16.html

galeon.hispavista.com/gambling777/poker-2.html

This is absolutely ludicrous, and is totally devaluing the net worth of Google.