- This topic is empty.
-
Posts
-
Great list of bad bots by IP below, I usually check it if I see an unusual activity (i.e. a bot totally ignores robots.txt and goes into directories I disallowed) and when I found its IP on this list, I permanently deny that IP on a server level.
Why are they doing this anyway? What is the benefit to them?
These bots are usually scrapers, data miners or email harvesters.
You can find some related articles on this blog: http://www.spam-whackers.com/blog/Other than that they are eating up your bandwidth so cost you money
Beside the bad bot IP’s list in my previous post my first todo is deny every traffic from *amazonaws.com when I set up a new domain. Although it is a service from amazon, it is mostly used to host bots and hacker bots. See more here: http://forums.oscommerce.com/lofiversion/index.php?t303607.html
