Live data from Hacker News

Ask HN: Website with 6^16 subpages and 80k+ daily bots

news.ycombinator.com

81–90 of 212 posts

Re: Ask HN: Website with 6^16 subpages and 80k+ daily bots

#81

First off, make a website defend mode that can be triggered to serve different content. Then, do the following: 1. Add a robots.txt, make it look like it's wordpress (Disallow: /wp-admin etc) 2. If any client requests /wp-admin, flag their IP ASN as bot. 3. If a client is a bot, send it a gzip bomb (100kB size, unpacked around 20GB), use Transfer-Encoding: gzip and Content-Encoding: gzip to also punish malicious web…

Can you also smash adsense in there? just for good measure :)

Re: Ask HN: Website with 6^16 subpages and 80k+ daily bots

#82

First off, make a website defend mode that can be triggered to serve different content. Then, do the following: 1. Add a robots.txt, make it look like it's wordpress (Disallow: /wp-admin etc) 2. If any client requests /wp-admin, flag their IP ASN as bot. 3. If a client is a bot, send it a gzip bomb (100kB size, unpacked around 20GB), use Transfer-Encoding: gzip and Content-Encoding: gzip to also punish malicious web…

This is deliciously evil. Favourited.

Re: Ask HN: Website with 6^16 subpages and 80k+ daily bots

#83
post #72

Fun. \\Your site is pretty big, but this one has you beat: http://www.googolplexwrittenout.com/ Contains downloadable PDF docs of googolplex written out in long form. There are a lot of PDFs, each with many pages.

Even Wikipedia has it comfortably beaten:

6,895,000+ articles + 1,433,000+ 記事 + 2 004 000+ статей + 1.983.000+ artículos + 2.950.000+ Artikel + 2 641 000+ articles + 1,446,000+ 条目 / 條目 + 1.886.000+ voci + ۱٬۰۱۵٬۰۰۰+ مقاله + 1.134.000+ artigos = 23387000 > 16^6 = 0xffffff+1

Re: Ask HN: Website with 6^16 subpages and 80k+ daily bots

#84

Sell it to someone inexperienced who wants to pick up a high traffic website. Show the stats of visitors, monthly hits, etc. DO NOT MENTION BOTS. Easiest money you'll ever make. (Speaking from experience ;) )

As an inexperienced buyer I'd ask you to prove me those are real visitors and at least give me a breakup by country and region, if not also age, sex, income category.

Also I would ask you to show me how much profit have you made from those visitors. I have no need for a high number of visitors if that doesn't translate into profit.

Re: Ask HN: Website with 6^16 subpages and 80k+ daily bots

#85
post #74

First off, make a website defend mode that can be triggered to serve different content. Then, do the following: 1. Add a robots.txt, make it look like it's wordpress (Disallow: /wp-admin etc) 2. If any client requests /wp-admin, flag their IP ASN as bot. 3. If a client is a bot, send it a gzip bomb (100kB size, unpacked around 20GB), use Transfer-Encoding: gzip and Content-Encoding: gzip to also punish malicious web…

The gzip idea is giving me goosebumps however this must be a solved problem, right? I mean, the client device can also send zip bombs so it sounds like it should be DDOS 101?

>I mean, the client device can also send zip bombs

A GET request doesn't have a body. There's nothing to gzip.

Re: Ask HN: Website with 6^16 subpages and 80k+ daily bots

#86
post #74

Earlier quoted context omitted.

The gzip idea is giving me goosebumps however this must be a solved problem, right? I mean, the client device can also send zip bombs so it sounds like it should be DDOS 101?

>I mean, the client device can also send zip bombs A GET request doesn't have a body. There's nothing to gzip.

What if they send a POST request?

Re: Ask HN: Website with 6^16 subpages and 80k+ daily bots

#87

Earlier quoted context omitted.

If you ban an IP or even an ASN, there could be (many) thousands sharing that same identifier. Some kid will unknowingly run some free game that does some lightweight scraping in the background as monetization and you ban the whole ISP?

> some free game that does some lightweight scraping in the background as monetization What in the flying **. Is this a common thing?

For some definition of "common", yes. Some try to be less shady by asking for consent (e.g. in exchange for in-game credits), others are essentially malware.

For example: https://bright-sdk.com/

> Bright SDK is approved by Apple, Amazon, LG, Huawei, Samsung app stores, and is whitelisted by top Antivirus companies.

Re: Ask HN: Website with 6^16 subpages and 80k+ daily bots

#88

First off, make a website defend mode that can be triggered to serve different content. Then, do the following: 1. Add a robots.txt, make it look like it's wordpress (Disallow: /wp-admin etc) 2. If any client requests /wp-admin, flag their IP ASN as bot. 3. If a client is a bot, send it a gzip bomb (100kB size, unpacked around 20GB), use Transfer-Encoding: gzip and Content-Encoding: gzip to also punish malicious web…

I would probably just stop at the gzip bomb but this is all great.

Re: Ask HN: Website with 6^16 subpages and 80k+ daily bots

#89

First off, make a website defend mode that can be triggered to serve different content. Then, do the following: 1. Add a robots.txt, make it look like it's wordpress (Disallow: /wp-admin etc) 2. If any client requests /wp-admin, flag their IP ASN as bot. 3. If a client is a bot, send it a gzip bomb (100kB size, unpacked around 20GB), use Transfer-Encoding: gzip and Content-Encoding: gzip to also punish malicious web…

Was the Sneed incident real ?

Re: Ask HN: Website with 6^16 subpages and 80k+ daily bots

#90
post #86

Earlier quoted context omitted.

>I mean, the client device can also send zip bombs A GET request doesn't have a body. There's nothing to gzip.

What if they send a POST request?

Most servers will limit posts to a fairly low size by default.
Post reply on HN