I caught this data miner pulling data from the forums yesterday, even after I had set them up to be hidden to any non registered users. I'd really like to know how this goddam thing works and how it does this. Look at how much bandwidth its cost some people in transferring all the pages to ghay Microsoft.
I for one didn't want the contents of my forums indexed, but unfortunately banning the IP address from the forums didn't keep it out. I ended up banning the damn thing from the whole site. Good riddance.
http://www.webmasterworld.com/forum97/73.htm
