Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Should you encrypt or compress first?”

blog.appcanary.com

Post reply on HN
Print view View original

131–140 of 249 posts

Re: “Should you encrypt or compress first?”

#131

Post by ghshephard »

yellowstuff wrote: post #75
hinkley wrote: post #65

Earlier quoted context omitted.

Does this really need to be said though? I may be too close to the problem. I've had to explain this to project managers and customers of course, but this is hacker news. It feels like a three page article on why you should put your socks on before your shoes and not after.

It's not just you. I've never used encryption or compression in any serious way, but the right answer seems obvious if you know the definitions of encryption and compression.

Really? I've been reading, and following encryption news for 10+ years, and it had never occurred to me that you should not compress prior to encrypting. The article was an eye opener for me - in hindsight, maybe obvious, but I bet 90% of your average technical audience wouldn't realize that you should not compress prior to encrypting.

Re: “Should you encrypt or compress first?”

#132

Post by shawnz »

vog wrote: post #26
orlp wrote: post #14

There's no compress or encrypt _first_. It's just compress or not, before encrypting. If security is important, the answer to that is no, unless you're an expert and familiar with CRIME and related attacks. Compression after encryption is useless, as there should be NO recognizable patterns to exploit after the encryption.

This is exactly what the article says.

The article seems to suggest that encrypt-then-compress would be the right solution in some circumstances, but really that's just as effective as encryption only with no compression. I don't feel the article is totally clear about this.

Re: “Should you encrypt or compress first?”

#133

Post by danbruc »

kbenson wrote: post #110
AlyssaRowan wrote: post #105

Earlier quoted context omitted.

Imagine a voice call, using a variable-bitrate codec, broken into small packets (not necessarily over the internet), of interest to a passive eavesdropper who cannot decrypt the content, but can observe metadata: the timing of packets and their sizes. Feed that information into a prebuilt phoneme-in-context model trained on that codec and language, and said eavesdropper could probably reconstruct a pretty good estima…

So, it's not just compression per se, but compression plus real-time plus some knowledge about the underlying information. These sorts of attacks don't matter against a compressed and then encrypted file of indeterminate data, correct? Similarly, an amalgamation of many types of data, such as a disk or archive?

If you know absolutely nothing about the uncompressed original, then it seems indeed pretty hard to infer anything from the compressed version. But as soon as you know something, for example the rough size of the uncompressed original, you can for example start to infer that the content is either more like AAA or more like ABC because repetitions will compress better. And in reality it is really unlikely that an attacker knows nothing at all about the content. You know the type of data, you know what the header structure of the file format looks like. Or you could try to learn something from the statistics of the observed messages. Maybe the data contains a time stamp that correlates with the time you observe the message. The possibilities are endless.

Re: “Should you encrypt or compress first?”

#134

Post by cellularmitosis »

kbenson wrote: post #110
AlyssaRowan wrote: post #105

Earlier quoted context omitted.

Imagine a voice call, using a variable-bitrate codec, broken into small packets (not necessarily over the internet), of interest to a passive eavesdropper who cannot decrypt the content, but can observe metadata: the timing of packets and their sizes. Feed that information into a prebuilt phoneme-in-context model trained on that codec and language, and said eavesdropper could probably reconstruct a pretty good estima…

So, it's not just compression per se, but compression plus real-time plus some knowledge about the underlying information. These sorts of attacks don't matter against a compressed and then encrypted file of indeterminate data, correct? Similarly, an amalgamation of many types of data, such as a disk or archive?

Thank you, that was the tldr I was looking for in this thread.

Re: “Should you encrypt or compress first?”

#135

Post by IncRnd »

lisper wrote: post #59
IncRnd wrote: post #55

Earlier quoted context omitted.

Are you talking about compressing plain text larger than a block, then encrypting per each block of the compressed output?

Yes, of course. What would be the alternative?

Thank you. Your comment wasn’t clear for several reasons. Stream ciphers are regularly blocked in TLS, compression can occur per block. Compression may be done per block, not just against a file larger than a single block.

Re: “Should you encrypt or compress first?”

#136

Post by jessriedel »

canttestthis wrote: post #119
hinkley wrote: post #65

Earlier quoted context omitted.

Does this really need to be said though? I may be too close to the problem. I've had to explain this to project managers and customers of course, but this is hacker news. It feels like a three page article on why you should put your socks on before your shoes and not after.

Cut the arrogance. It's not obvious to all of us, we're not all cryptonerds.

Arrogance is probably better than name calling.

Re: “Should you encrypt or compress first?”

#139

Post by IncRnd »

Despite the question being flawed. The correct answer is a series of questions: Who is the attacker? What are you guarding? What assumptions are there about the operating environment? What invariants (regulations, compliance, etc) exist?

There may be compensating controls that invalidate the perceived needs for encryption or compression, for example. i.e. don't design in the dark.

Of course, the interviewer may just want a canned scripted answer - but the interview is your chance to shine, showing how you can discuss all the angles.

Re: “Should you encrypt or compress first?”

#140

Post by jakobegger »

DanBlake wrote: post #90
IanCal wrote: post #58

Earlier quoted context omitted.

> It wont be a huge difference in file size, but it will be smaller. may be smaller, not all outputs will be smaller, this is true of all lossless compression algorithms: https://en.wikipedia.org/wiki/Pigeonhole_principle

This is literally all I was saying and people went on a downvote frenzy. People are literally saying that a encrypted output can NEVER be compressed. I was saying that it can (although not all the time!) and the gains would be minimal. Ive corrected the first post with the word 'may'

Monkeys hitting random keys on a typewriter can write novels (although not all the time!)
Post reply on HN