“Should you encrypt or compress first?” • Hacker News Board

Re: “Should you encrypt or compress first?”

#151

Post by lisper » Tue, Jun 28, 2016, 6:27 PM UTC

jrockway wrote: post #145

lisper wrote: post #39

Earlier quoted context omitted.

> If security is important, the answer to that is no It's a little more nuanced than that. Compression may cause information leaks or it can prevent them depending on the circumstances. If you're encrypting an audio stream, then compressing it first can cause leaks. If you're encrypting a document, then compressing it first may prevent leaks.

I think compression is a read herring here. Think about keyboard-interactive ssh authentication. Whether or not the data is compressed, the timing of the packets depends on the timing between when you pressed the keys on your keyboard. Thus information is leaked, and compression was not involved. Therefore, I feel like compression complicates the discussion unnecessarily. Information that's not encrypted (the time a…

Well, yeah, but the point is that compression can cause the leakage of information through timing that would not otherwise have leaked.

Re: “Should you encrypt or compress first?”

#152

Post by duskwuff » Tue, Jun 28, 2016, 6:29 PM UTC

olliej wrote: post #109

DanBlake wrote: post #91

Earlier quoted context omitted.

Feel free to upload a 5MB file under your narrow restrictions and I will see if I can reduce it with one of the 20 different compression systems available to me.

Ooh, the encryption comment I just made reminded me of an example of compression potentially working on encrypted content. AES-ECB mode will actually compress fairly well for some inputs (the canonical example being a bitmap image). The reason the compression can work? Because the crypto is broken. In the image case you can actually just visually (no maths or anything) see a large amount of the details of the source…

Yep. Wikipedia uses this example image:

https://upload.wikimedia.org/wikipedia/commons/f/f0/Tux_ecb....

Re: “Should you encrypt or compress first?”

#153

Post by mikeash » Tue, Jun 28, 2016, 6:32 PM UTC

olliej wrote: post #144

mikeash wrote: post #125

Earlier quoted context omitted.

Here you go: https://mikeash.com/tmp/randomfile.bin If you manage to get that under 5MB by applying any tool which allows recovering the original data, I will be most interested to know how you did it.

Success: compress.sh: #!/bin/bash echo "#!/bin/bash" echo curl $1 ./compress.sh https://mikeash.com/tmp/randomfile.bin > randomfile.compressedzomg :D /me hides

You're lucky I'm too lazy to edit a byte on the server and then ask you if it still matches.

Re: “Should you encrypt or compress first?”

#154

Post by mikeash » Tue, Jun 28, 2016, 6:33 PM UTC

x1798DE wrote: post #146

mikeash wrote: post #78

Earlier quoted context omitted.

It's pointless. If you sign the encrypted data, then once the signature is verified in the receiver, you know that the decrypted data is also good. Repeating the signature just wastes time and space.

Except what about the case where someone can spoof that an encrypted message came from them? In that case, you want the signature somewhere inaccessible, so that they can't selectively strip it off.

I don't understand what you mean. Your signature scheme is either secure or it isn't. If an attacker can spoof or strip the signature and have you accept it, then it isn't secure and it doesn't matter where you put it. If they can't, then you can have it be in the outermost layer.

Re: “Should you encrypt or compress first?”

#155

Post by Dylan16807 » Tue, Jun 28, 2016, 6:35 PM UTC

mikeash wrote: post #95

DanBlake wrote: post #88

Earlier quoted context omitted.

Here it is encrypted with AES256 - Pass is wasteoftime http://pasted.co/wasteoftime.txt 4,732 bytes http://pasted.co/wasteoftime.rar 3,753 bytes Also, if there is issues with using the 'wrong' encryption, I feel thats kind of a straw man argument. Please feel free to upload a file over 5MB which cant be reduced in size through any of the various compression tools. Also, keep in mind that I never said it would be a hu…

That's not encrypted with AES256, that's encrypted with AES256 and then encoded with base64. Do you understand the difference between base64 and binary data? Base64 only uses six bits per byte. It's designed to allow data to transit cleanly through places which only allow text, such as JSON. Binary data is eight bits per byte. Binary data is what encryption and compression algorithms output. Naturally, if you take da…

And as a note: It's fine to use base64 if you want. But then you also need to output the compressed file as base64. And you'll see that 3753 octects equate to 5004 base64 characters, and the file is actually significantly larger than it was before compression.

Re: “Should you encrypt or compress first?”

#156

Post by danielweber » Tue, Jun 28, 2016, 6:36 PM UTC

hinkley wrote: post #65

vog wrote: post #26

Earlier quoted context omitted.

This is exactly what the article says.

Does this really need to be said though? I may be too close to the problem. I've had to explain this to project managers and customers of course, but this is hacker news. It feels like a three page article on why you should put your socks on before your shoes and not after.

Even a few years ago, the "obvious answer" in the infosec space was that you should compress first.

Re: “Should you encrypt or compress first?”

#157

Post by dang » Tue, Jun 28, 2016, 6:37 PM UTC

loeg wrote: post #30

vog wrote: post #26

Earlier quoted context omitted.

This is exactly what the article says.

The troublesome part is the headline.

By putting quotation marks around it I think we can eliminate the misleading bit.

Re: “Should you encrypt or compress first?”

#158

Post by em3rgent0rdr » Tue, Jun 28, 2016, 6:37 PM UTC

What if you compress and then only send data at regular periods and regular packet sizes? That way no information can be gleaned. E.g. after compressing you pad the data if it is unusually short, or you include other compressed data too, or you only use constant bit-rate compression algorithm.

Re: “Should you encrypt or compress first?”

#159

Post by spatulon » Tue, Jun 28, 2016, 6:38 PM UTC

That was a fun read. Do I detect a nod to tptacek's "If You’re Typing the Letters A-E-S Into Your Code You’re Doing It Wrong"?

https://www.nccgroup.trust/us/about-us/newsroom-and-events/b...

Re: “Should you encrypt or compress first?”

#160

Post by belorn » Tue, Jun 28, 2016, 6:46 PM UTC

AlyssaRowan wrote: post #105

belorn wrote: post #80

Earlier quoted context omitted.

Could you elaborate what kind of scenario would work on a audio stream but not on a document? In order to use CRIME, the attacker would need to repeatably injected content before the compression stage in order to test if it matches some part of the source. Audio is kind of heavy uncompressed and aren't normally streamed as such.

Imagine a voice call, using a variable-bitrate codec, broken into small packets (not necessarily over the internet), of interest to a passive eavesdropper who cannot decrypt the content, but can observe metadata: the timing of packets and their sizes. Feed that information into a prebuilt phoneme-in-context model trained on that codec and language, and said eavesdropper could probably reconstruct a pretty good estima…

Thanks, its good to have specifics when thinking about security.

I would guess that the phones and voip is a perfect target to do this, as those system are often designed to filter out noise. I guess it also would be almost impossible to use this to transcribe a encrypted movie or music, as the unpredictable sounds would break the prediction model.