Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Curious Case of MD5

katelynsills.com

Post reply on HN
Print view View original

101–110 of 173 posts

Re: The Curious Case of MD5

#101

Post by heads »

I can see the lawyers’ point. Any old checksum is good enough for spotting random data corruption. If you do happen to have a crooked lawyer who is submitting tampered evidence then you would hope that there are better systems in place to weed out this ethical corruption.

For instance: law school training to strengthen ethics, well known punishments to act as a deterrent, hiring processes to filter out unscrupulous actors, and whistleblower protections to encourage and reward vigilance.

It’s not an opinion that adheres to the cynical zeitgeist, but in my experience most members of this profession are extremely trustworthy. I’m sure they dislike the stereotype of lawyer=rotter just as much as hackers are tired of being typecast as Newman… sorry, Dennis from Jurassic Park!

Re: The Curious Case of MD5

#102

Post by wolf550e »

colmmacc wrote: post #41
tptacek wrote: post #10

The unsatisfying answer to this is probably that it just doesn't matter. It's not as if evidence chain of custody is assured cryptographically; it's assured by rules and regulations and an adversarial system. If you tried to submit as evidence a forged document vouchsafed with a colliding MD5 hash, you'd be putting your own freedom at risk, because the forgery will be straightforwardly detectable (the real document w…

You say to use SHA2, but TFA says to use SHA3 or Blake. I think your recommendation is the better one, but I feel like teasing out why because it's interesting. Firstly ... the NIST recommendation TFA links doesn't just recommend SHA-3, it actually says "Federal agencies should use SHA-2 or SHA-3 as an alternative to SHA-1." SHA-2 and SHA-3 are both valid and recommended hash functions by NIST. And while 3 is higher…

BLAKE3 is faster than hardware accelerated SHA-2 because the tree mode used in BLAKE3 allows hashing parts of a single message in parallel (with SHA-2, parts of a single message have to be hashed one after another, and parallelism is only used in workloads where you process multiple messages at the same time).

https://github.com/minio/sha256-simd

https://github.com/BLAKE3-team/BLAKE3

Re: The Curious Case of MD5

#103

Post by adrian_b »

robertlagrant wrote: post #97

> Yes, they say, MD5 is broken for encryption, but since they’re not doing encryption, it’s fine for them to use it. Unless I missed it, this article seems to not refute the most fundamental point: MD5 was never broken for encryption. Hashing is not encryption.

While hashing is not encryption, any secure hashing function can be used for encryption, even when used as a black box, (by making an unpredictable PRNG with it).

Moreover, MD5, SHA-1 and SHA-2 contain a block cipher function used in the Davies-Meyer mode of operation.

The internal block cipher function can be extracted and used in any other mode of operation possible for block cipher functions.

Because of these possibilities, many older laws that have existed in various places, prohibiting the inclusion of encryption in software products, but allowing secure hashing functions, have been completely misguided.

Re: The Curious Case of MD5

#104

Post by adrian_b »

OhMeadhbh wrote: post #74
bawolff wrote: post #56

Earlier quoted context omitted.

So if the document is evidence , then its probably created by the attacker. This seems like a setup where collision is more relavent than 2nd preimage.

How so?

Second preimage attacks are relevant for the documents that you create and give to others.

Keeping a hash of the document ensures that you can prove that any altered document shown by someone else is not the original.

Collision attacks are relevant for the documents created by others, which you receive.

If you have a hash of the document that is collision-resistant, you can trust that the creator does not have other variants of the document with the same hash.

If the hash is not collision resistant, i.e. it is MD5 or SHA-1, you cannot know if the creator of the document has not also created another variant of the document than the one handed to you, which has the same hash.

That is why a digital signature on a document received from others is meaningful only if it is based on a collision-resistant hash.

If you sign and verify your own documents, for detecting modifications, a second preimage attack resistant hash would be enough.

Re: The Curious Case of MD5

#105

Post by ivanbakel »

masklinn wrote: post #75
wyldfire wrote: post #45

Earlier quoted context omitted.

> there's not really any relevant attack space. Then why use a cryptographic hash at all? much better hashes out there that only strive for distribution/avalanche. https://en.wikipedia.org/wiki/Non-cryptographic_hash_functio...

Because they’re known to be collision resistant (it’s a primary requirement), whereas non-cryptographic hashes are not, so now you need to evaluate each function individually for this property which is a hassle. And an unnecessary one, I doubt the computation of the hash is what genomics are bound on.

But what's the relevance of collision resistance, without a meaningful attack surface?

Re: The Curious Case of MD5

#106

Post by adrian_b »

moyix wrote: post #59
gruez wrote: post #55

Earlier quoted context omitted.

Generating 2^64 hashes isn't guaranteed to produce a collision, and even if a collision did exist in that set, you're not going to find it by getting a bunch of GPUs to compute 2^64 hashes. There's a huge difference between a haystack that maybe contains a needle, and a needle that's been pulled from the haystack and presented to you. To actually find and identify the collisions you'll need to hook those GPUs up to s…

You can generate pairs of hashes for random inputs and check for collision without storing all of the outputs, no?

True, which is why 256-bit or bigger hashes are recommended to make sure that the time for finding a collision is alone great enough to make this impossible.

Finding a collision for an 160-bit hash by brute force, on a big cloud or supercomputer, is about the maximum that someone with unlimited financial resources could do.

Re: The Curious Case of MD5

#107

Post by laserbeam »

> MD5 should be considered broken and unsuitable for further use.

Ya know... It's 2024 and Azure's blob storage ONLY supports MD5 for integrity checks when writing blobs. There are no other hash functions supported there. The default cloud storage solution implemented by one of the largest cloud providers out there ONLY uses MD5.

I really want to use something else, but whenever I have to interact with them I must fall back to MD5. It's not up to me as a dev to use something better if I need to interact with Azure. Yes, I can use other hashes alongside MD5, but if I want integrity checks with the storage provider I can't completely abandon MD5.

Re: The Curious Case of MD5

#108

Post by shpx »

MD5 hashes are half the length of the recommended hashing algorithms. This convenience (and the switching cost) is worth more than the theoretical security considerations.

Re: The Curious Case of MD5

#109

Post by Pikamander2 »

laserbeam wrote: post #107

> MD5 should be considered broken and unsuitable for further use. Ya know... It's 2024 and Azure's blob storage ONLY supports MD5 for integrity checks when writing blobs. There are no other hash functions supported there. The default cloud storage solution implemented by one of the largest cloud providers out there ONLY uses MD5. I really want to use something else, but whenever I have to interact with them I must fa…

WordPress still uses MD5 for database passwords to this very day with no immediate plans to change it.

That said, they apparently use eight passes of MD5 hashing along with salting, which they claim is a sufficiently secure combo.

WordPress's core and default themes are known to be fairly secure, so I'd like to believe they know what they're talking about, but if nothing else it feels icky.

Re: The Curious Case of MD5

#110

Post by denton-scratch »

> only broken for encryption

It's broken in an adversarial situation: given the hash of evidence-file A, it's possible to construct a file B that gives the same hash.

But it would be a different matter entirely to construct a file B that actually looked like a file of evidence relevant to the case. I don't know how lawyers use these hashes, but unless they're being used to detect malicious tampering, I don't see what's wrong with MD5. And since the files to be hashed are evidence, they're in the custody of a court; things have got quite bad if court officials might be tampering with evidence.

Post reply on HN