Live data from Hacker News

I use zip bombs to protect my server

idiallo.com

171–180 of 467 posts

Re: I use zip bombs to protect my server

#171

I deployed this, instead of my usual honeypot script. It's not working very well. In the web server log, I can see that the bots are not downloading the whole ten megabyte poison pill. They are cutting off at various lengths. I haven't seen anything fetch more than around 1.5 Mb of it so far. Or is it working? Are they decoding it on the fly as a stream, and then crashing? E.g. if something is recorded as having read…

Try content labyrinth. I.e. infinitely generated content with a bunch of references to other generated pages. It may help against simple wget and till bots adapt. PS: I'm on the bots side, but don't mind helping.

This doesn't work if you pay bandwidth and CPU usage for your servers though.

Re: I use zip bombs to protect my server

#172
post #114

I'm curious why a 10GB file of all zeroes would compress only to 10MB. I mean theoretically you could compress it to one byte. I suppose the compression happens on a stream of data instead of analyzing the whole, but I'd assume it would still do better than 10MB.

A compressed file that is only one byte long can only represent maximally 256 different uncompressed files. Signed, a kid in the 90s who downloaded some "wavelet compression" program from a BBS because it promised to compress all his WaReZ even more so he could then fit moar on his disk. He ran the compressor and hey golly that 500MB ISO fit into only 10MB of disk now! He found out later (after a defrag) that the "co…

> He found out later (after a defrag) that the "compressor" was just hiding data in unused disk sectors and storing references to them

So you could access the files until you wrote more data to disk?

Re: I use zip bombs to protect my server

#173
post #119
post #114

I'm curious why a 10GB file of all zeroes would compress only to 10MB. I mean theoretically you could compress it to one byte. I suppose the compression happens on a stream of data instead of analyzing the whole, but I'd assume it would still do better than 10MB.

I get your point(and have no idea why it isn't compressed more), but is the theoretical value of 1 byte correct? With just one single byte, how does it know how big should the file be after being decompressed?

In general, this theoretical problem is called the Kolmogorov Complexity of a string: the size of the smallest program that outputs a the input string, for some definition of "program", e.g., an initial input tape for a given universal turing machine. Unfortunately, Kolmogorov Complexity in general is incomputable, because of the halting problem.

But a gzip decompressor is not turing-complete, and there are no gzip streams that will expand to infinitely large outputs, so it is theoretically possible to find the pseudo-Kolmogorov-Complexity of a string for a given decompressor program by the following algorithm:

Let file.bin be a file containing the input byte sequence.

1. BOUNDS=$(gzip --best -c file.bin | wc -c)

2. LENGTH=1

3. If LENGTH==BOUNDS, run `gzip --best -o test.bin.gz file.bin` and HALT.

4. Generate a file `test.bin.gz` LENGTH bytes long containing all zero bits.

5. Run `gunzip -k test.bin.gz`.

6. If `test.bin` equals `file.bin`, halt.

7. If `test.bin.gz` contains only 1 bits, increment LENGTH and GOTO 3.

8. Replace test.bin.gz with its lexicographic successor by interpreting it as a LENGTH-byte unsigned integer and incrementing it by 1.

9. GOTO 5.

test.bin.gz contains your minimal gzip encoding.

There are "stronger" compressors for popular compression libraries like zlib that outperform the "best" options available, but none of them are this exhaustive because you can surely see how the problem rapidly becomes intractable.

For the purposes of generating an efficient zip bomb, though, it doesn't really matter what the exact contents of the output file are. If your goal is simply to get the best compression ratio, you could enumerate all possible files with that algorithm (up to the bounds established by compressing all zeroes to reach your target decompressed size, which makes a good starting point) and then just check for a decompressed length that meets or exceeds the target size.

I think I'll do that. I'll leave it running for a couple days and see if I can generate a neat zip bomb that beats compressing a stream of zeroes. I'm expecting the answer is "no, the search space is far too large."

Re: I use zip bombs to protect my server

#174
post #166

Earlier quoted context omitted.

I made a 64kx64k JPEG once by feeding the encoder the same line of macro blocks until it produce the entire image. Years later I was finally able to open it.

I had a ton of trouble opening a 10MB or so png a few weeks back. It was stitched together screenshots forming a map of some areas in a game, so it was quite large. Some stuff refused to open it at all as if the file was invalid, some would hang for minutes, some opened blurry. My first semi-success was Fossify Gallery on my phone from F-Droid. If I let it chug a bit, it'd show a blurry image, a while longer it'd foc…

That's a 36,000x20,000 PNG, 720 megapixels. Many decoders explicitly limit the maximum image area they'll handle, under the reasonable assumption that it will exceed available RAM and take too long, and assume the file was crafted maliciously or by mistake.

Re: I use zip bombs to protect my server

#175
post #75
post #54

Is there any legal exposure possible? Like, a legitimate crawler suing you and alleging that you broke something of theirs?

Disclosure: IANAL The CFAA[1] prohibits: > knowingly causes the transmission of a program, information, code, or command, and as a result of such conduct, intentionally causes damage without authorization, to a protected computer; As far as I can tell (again, IANAL) there isn't an exception if you believe said computer is actively attempting to abuse your system[2]. I'm not sure if a zip bomb would constitute intenti…

There is IMO no legal use case for an external computer system to initiate a connection with my system without prior legal agreement. It all happens on good will.

Re: I use zip bombs to protect my server

#176
post #75
post #54

Is there any legal exposure possible? Like, a legitimate crawler suing you and alleging that you broke something of theirs?

Disclosure: IANAL The CFAA[1] prohibits: > knowingly causes the transmission of a program, information, code, or command, and as a result of such conduct, intentionally causes damage without authorization, to a protected computer; As far as I can tell (again, IANAL) there isn't an exception if you believe said computer is actively attempting to abuse your system[2]. I'm not sure if a zip bomb would constitute intenti…

There is IMO no legal use case for an external computer system to initiate a connection with my system without prior legal agreement. It all happens on good will and therefore can be terminated at any time.

Re: I use zip bombs to protect my server

#177
post #166

Earlier quoted context omitted.

I made a 64kx64k JPEG once by feeding the encoder the same line of macro blocks until it produce the entire image. Years later I was finally able to open it.

I had a ton of trouble opening a 10MB or so png a few weeks back. It was stitched together screenshots forming a map of some areas in a game, so it was quite large. Some stuff refused to open it at all as if the file was invalid, some would hang for minutes, some opened blurry. My first semi-success was Fossify Gallery on my phone from F-Droid. If I let it chug a bit, it'd show a blurry image, a while longer it'd foc…

Safari on my MacBook Air opened it fine, though it took about four seconds. Zooming works fine as well. It does take ~3GB of memory according to Activity Monitor.

Re: I use zip bombs to protect my server

#178
post #166

Earlier quoted context omitted.

I made a 64kx64k JPEG once by feeding the encoder the same line of macro blocks until it produce the entire image. Years later I was finally able to open it.

I had a ton of trouble opening a 10MB or so png a few weeks back. It was stitched together screenshots forming a map of some areas in a game, so it was quite large. Some stuff refused to open it at all as if the file was invalid, some would hang for minutes, some opened blurry. My first semi-success was Fossify Gallery on my phone from F-Droid. If I let it chug a bit, it'd show a blurry image, a while longer it'd foc…

IrfanView was able to load it in about 8 seconds (Ryzen 7 5800x) using 2.8GB of RAM, but zooming/panning is quite slow (~500ms per action)

Re: I use zip bombs to protect my server

#179

I deployed this, instead of my usual honeypot script. It's not working very well. In the web server log, I can see that the bots are not downloading the whole ten megabyte poison pill. They are cutting off at various lengths. I haven't seen anything fetch more than around 1.5 Mb of it so far. Or is it working? Are they decoding it on the fly as a stream, and then crashing? E.g. if something is recorded as having read…

Perhaps need to semi-randomize the file size? I'm guessing some of the bots have a hard limit to the size of the resource they will download.

Many of these are annoying LLM training/scraping bots (in my case anyway). So while it might not crash them if you spit out a 800KB zipbomb, at least it will waste computing resources on their end.

Re: I use zip bombs to protect my server

#180

I deployed this, instead of my usual honeypot script. It's not working very well. In the web server log, I can see that the bots are not downloading the whole ten megabyte poison pill. They are cutting off at various lengths. I haven't seen anything fetch more than around 1.5 Mb of it so far. Or is it working? Are they decoding it on the fly as a stream, and then crashing? E.g. if something is recorded as having read…

Try content labyrinth. I.e. infinitely generated content with a bunch of references to other generated pages. It may help against simple wget and till bots adapt. PS: I'm on the bots side, but don't mind helping.

Wouldn't this just waste your own bandwidth/resources?
Post reply on HN