Folks.
We've been suffering from some major issues recently with performance of the database. I've tracked it down to a couple of rogue spiders / bots hitting us, causing very long running queries and gobs of data heading outbound.
We have a robots.txt file, but it looks like at least one of them doesn't respect the robots.txt file so I may need to brute force it out.
Please bear with us for a while as we try to keep the bots out which will hopefully put the system back in better shape.
Thanks.
Bots are running amok
- caughtatwork
- Posts: 17015
- Joined: 17 May 04 12:11 pm
- Location: Melbourne
- Contact:
Re: Bots are running amok
Spiders are an occupational hazard when caching ...
Love your work; good luck with the pest eradication.
Love your work; good luck with the pest eradication.
- CraigRat
- 850 or more found!!!
- Posts: 7015
- Joined: 23 August 04 3:17 pm
- Twitter: CraigRat
- Facebook: http://facebook.com/CraigRat
- Location: Launceston, TAS
- Contact:
- caughtatwork
- Posts: 17015
- Joined: 17 May 04 12:11 pm
- Location: Melbourne
- Contact:
Re: Bots are running amok
I've 403'd the most obnoxious and robots.txt'd the other 2, so they should die off in a day or so.
All things look back to normal.
All things look back to normal.
- WazzaAndWenches
- 5000 or more caches found
- Posts: 395
- Joined: 08 April 07 10:28 pm
- Location: Echuca, Vic
Re: Bots are running amok
Love your work C@W.
Re: Bots are running amok
Yep I vote you 2 guys are still as wonderful as when I first started caching