On the evilness of User-Agent sniffing
Jeremy Visser writes about the evilness of User-Agent sniffing
I tend to disagree with him, particularly after reading Wikipedia's robots.txt file, and some of the comments in there.
wget can be quite a nasty program if used incorrectly. Chances are, if you know what you're doing, you'll know how to change the User-Agent string, and therefore you're probably not going to do something dumb like try to download the entire site recursively.
If you don't, you're probably Doing It Wrong.