Live data from Hacker News

HTTP Headers For Fun & Profit

loopj.com

31–40 of 53 posts

Re: HTTP Headers For Fun & Profit

#36
post #4

There's also a lot of fun robots.txt. I forget where it was mentioned (I didn't find it myself) but this one always made me laugh: # robots.txt for http://www.palm.com/ modified 7/28/09 User-agent: Vampires Disallow: /neck

From reddit.com: User-Agent: bender Disallow: /my_shiny_metal_ass User-Agent: Gort Disallow: /earth

Better still: http://www.last.fm/robots.txt

    Disallow: /harming/humans
    Disallow: /ignoring/human/orders
    Disallow: /harm/to/self

Re: HTTP Headers For Fun & Profit

#40
post #12

Is this not a terrible waste of bandwidth though ? At about 10 bytes per header (on the low end..) and say 100 million requests per day, that amounts to 1Gb of outbound bandwidth, if you count inbound bandwidth then that comes to 2gb in total. Not to mention the cumulative time spent by users downloading those bytes, thereby delaying resource display. Okay, I see that I sucked the fun out of it :P

1994 called, they want their bandwidth back! :)
Post reply on HN