Earlier quoted context omitted.
Are you talking about compressing plain text larger than a block, then encrypting per each block of the compressed output?
@lisper Stream ciphers.
Why would you use a stream cipher to encrypt a document?
“Should you encrypt or compress first?”
71–80 of 249 posts
Re: “Should you encrypt or compress first?”
#72Earlier quoted context omitted.
But in almost all cases it's right.
"Don't compress at all" is the better default answer. Noting that "compression after encryption is stupid."
If security is the top concern, making all encrypted messages the same length would be ideal as far as I can tell. That way, all you are giving away is an upper bound on the message size. Padding with random noise to a uniform length (with the payload either compressed or not) and then encrypting should be the most secure option.
Re: “Should you encrypt or compress first?”
#73Earlier quoted context omitted.
No. If the encrypted output is randomly distributed, as it should be, then the expected number of bits cannot be reduced through compression. If it successfully compresses some files, it will make some files larger. For random data, the change will be negligible with overwhelming probability, plus there will be overhead.
Using a random text generator http://pasted.co/encrypted.txt ( 10,002 bytes ) Then compressed it with winrar http://pasted.co/compressed.rar ( 7,743 bytes ) Roughly 20% compression isnt meaningless- so I fail to see why you are just giving a flat 'no' when its obvious what you are saying is untrue.
That's a base64 encoding of random data, which by definition has only 6 bits of entropy per byte. You should be able to compress it 25% but you only managed 22.6%
Re: “Should you encrypt or compress first?”
#74Earlier quoted context omitted.
@DanBlake That's a text file, which doesn't use the full range of a byte (0-255), so each character takes less than a byte.
Im happy to reproduce in other formats. The contents are still random which is the crux of the issue here.
The contents are not purely random. For example, I can predict with 100% certainty that any given byte in the file is not the value zero (ASCII NUL).
Try compressing the output of /dev/urandom on your nearest convenient UNIX-like system. If you figure out a way to reliably and significantly compress that, please report back.
Re: “Should you encrypt or compress first?”
#75Earlier quoted context omitted.
This is exactly what the article says.
Does this really need to be said though? I may be too close to the problem. I've had to explain this to project managers and customers of course, but this is hacker news. It feels like a three page article on why you should put your socks on before your shoes and not after.
It's not just you. I've never used encryption or compression in any serious way, but the right answer seems obvious if you know the definitions of encryption and compression.
Re: “Should you encrypt or compress first?”
#76Earlier quoted context omitted.
No. If the encrypted output is randomly distributed, as it should be, then the expected number of bits cannot be reduced through compression. If it successfully compresses some files, it will make some files larger. For random data, the change will be negligible with overwhelming probability, plus there will be overhead.
Using a random text generator http://pasted.co/encrypted.txt ( 10,002 bytes ) Then compressed it with winrar http://pasted.co/compressed.rar ( 7,743 bytes ) Roughly 20% compression isnt meaningless- so I fail to see why you are just giving a flat 'no' when its obvious what you are saying is untrue.
[deleted]
Re: “Should you encrypt or compress first?”
#77There's no compress or encrypt _first_. It's just compress or not, before encrypting. If security is important, the answer to that is no, unless you're an expert and familiar with CRIME and related attacks. Compression after encryption is useless, as there should be NO recognizable patterns to exploit after the encryption.
[deleted]
Re: “Should you encrypt or compress first?”
#78Earlier quoted context omitted.
Good catch! Although the article talks about encrypt+sign versus sign+encrypt, the same argument goes for compress+sign versus sign+compress.
Why is the debate about "compress/encrypt then sign" vs "sign then compress/encrypt"? Is there a non obvious problem with sign then compress/encrypt then sign again? (overcomplicated or unnecessary?)
It's pointless. If you sign the encrypted data, then once the signature is verified in the receiver, you know that the decrypted data is also good. Repeating the signature just wastes time and space.
Re: “Should you encrypt or compress first?”
#79Where everyone seems to be getting confused is handling a live flow versus handling a finalized flow (a file).
* Always pad to combat plain-text attacks, padding in theory shouldn't compress well so there's no point making the compression less effective by processing it.
* Always compress a 'file' first to reduce entropy.
* Always pad-up a live stream, maybe this data is useful in some other way, but you want interactive messages to be of similar size.
* At some place in the above also include a recipient identifier; this should be counted as part of the overhead not part of the padding.
* The signature should be on everything above here (recipients, pad, compressed message, extra pad).
. It might be useful to include the recipients in the un-encrypted portion of the message, but there are also contexts where someone might choose otherwise; an interactive flow would assume both parties knew a key to communicate with each other on and is one such case.
* The pad, message, extra-pad, and signature /must/ be encrypted. The recipients /may/ be encrypted.
I did have to look up the sign / encrypt first question as I didn't have reason to think about it before. In general I've looked to experts in this field for existing solutions, such as OpenPGP (GnuPG being the main implementation). Getting this stuff right is DIFFICULT.
Re: “Should you encrypt or compress first?”
#80There's no compress or encrypt _first_. It's just compress or not, before encrypting. If security is important, the answer to that is no, unless you're an expert and familiar with CRIME and related attacks. Compression after encryption is useless, as there should be NO recognizable patterns to exploit after the encryption.
> If security is important, the answer to that is no It's a little more nuanced than that. Compression may cause information leaks or it can prevent them depending on the circumstances. If you're encrypting an audio stream, then compressing it first can cause leaks. If you're encrypting a document, then compressing it first may prevent leaks.
Could you elaborate what kind of scenario would work on a audio stream but not on a document? In order to use CRIME, the attacker would need to repeatably injected content before the compression stage in order to test if it matches some part of the source. Audio is kind of heavy uncompressed and aren't normally streamed as such.