Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Curious Case of MD5

katelynsills.com

Post reply on HN
Print view View original

131–140 of 173 posts

Re: The Curious Case of MD5

#131

Post by DannyBee »

tptacek wrote: post #10

The unsatisfying answer to this is probably that it just doesn't matter. It's not as if evidence chain of custody is assured cryptographically; it's assured by rules and regulations and an adversarial system. If you tried to submit as evidence a forged document vouchsafed with a colliding MD5 hash, you'd be putting your own freedom at risk, because the forgery will be straightforwardly detectable (the real document w…

Lawyer here. This is right. Unlike some of the typical software use cases, md5 is usually not the only thing stopping you from falsifying evidence.

The records being produced almost always exist and the hash is being used to differentiate which is which and whether you got are correct copies.

If evil litigant produces fake documents instead of real ones, and you can't get access to the system they came from, no hash will save you - they can produce any hash they want and you can't verify against an original.

If you can verify against the original, forensics isn't going to look at a hash and say "hash matches my job is done".

Most of the time what matters is whether the evidence exists or not.

The tobacco companies lied and claimed they never had any evidence in the first place.

assume instead they wanted to falsify the data to say that as far as they could tell smoking did not cause cancer.

this is not a hashing problem. This is a problem of making up fake studies that look real.

Re: The Curious Case of MD5

#132

Post by LgLasagnaModel »

“A cryptographic hash should uniquely identify a file.”

This is not true. It’s not possible to guarantee this. One can be certain that two files DON’T match if they have different hashes, but one cannot be certain that two files DO match based ONLY on the fact that they have the same hash.

Re: The Curious Case of MD5

#133

Post by DannyBee »

woodruffw wrote: post #13
tptacek wrote: post #10

The unsatisfying answer to this is probably that it just doesn't matter. It's not as if evidence chain of custody is assured cryptographically; it's assured by rules and regulations and an adversarial system. If you tried to submit as evidence a forged document vouchsafed with a colliding MD5 hash, you'd be putting your own freedom at risk, because the forgery will be straightforwardly detectable (the real document w…

This is more or less what I learned when I worked on forensics software, the kind that was supposed to maintain this kind of chain of custody/integrity. Like most things that touch the legal system, the presumption is that dishonesty or unsoundness in the chain of custody is fundamentally a legal problem with legal recourses, not something that can be solved with math.

Right. If you don't have access to the original system (whether it's the thing that stored the emails or a computer with evidence on it), just an image, the hash doesn't matter anyway. They can just put fake data in the evidence vault system to begin with.

If you do have access to the original system, md5 is usually not the thing in the way of falsifying evidence.

most of the time they will claim, the evidence does not exist at all, rather than try to falsify it.

falsification requires making lots of things that makes sense historically, and humans to swear to them.

you also often have to falsify more than one system in a consistent way.

you have to do all of these things in a way that the forensic specialist is not going to think that everything looks really weird

Re: The Curious Case of MD5

#134

Post by tesdinger »

throwaway89201 wrote: post #19

Another unfortunately place where MD5 is widely used: pirate libraries such as Library Genesis and Anna's Archive. While content is distributed at large in torrents with SHA1-summed shards, and Anna's Archive at least offers some structured metadata which would allow to slowly migrate away from MD5, files are still indexed using MD5 as primary key, and any other kind of file hash is nowhere to be found. Pirate librar…

I do not need to do a hash collision to upload malware to Library Genesis. I could just upload malware with a slightly different name than a popular book and claim it is a different release, like book_high_quality. To securely view content downloaded from such sites, update your software and sandbox the application.

Re: The Curious Case of MD5

#135

Post by denton-scratch »

aljarry wrote: post #125
denton-scratch wrote: post #110

> only broken for encryption It's broken in an adversarial situation: given the hash of evidence-file A, it's possible to construct a file B that gives the same hash. But it would be a different matter entirely to construct a file B that actually looked like a file of evidence relevant to the case. I don't know how lawyers use these hashes, but unless they're being used to detect malicious tampering, I don't see what…

> It's broken in an adversarial situation: given the hash of evidence-file A, it's possible to construct a file B that gives the same hash. No, that's a second preimage attack. MD5 is safe against preimage & second preimage attacks. What MD5 is not safe against, is a collision attack: you can create two messages/files with different content, that end up having the same hash.

Yeah, sorry. TFA made that clear.

So to exploit the vulnerability, you have to be able to manipulate file A, the original piece of evidence, to construct a file B that has a matching hash. I still fail to see how this impacts files submitted to a court in evidence.

Re: The Curious Case of MD5

#136

Post by ddtaylor »

This is actually something I discussed with my legal team and was prepared to bring up at trial with our own forensic expert. My team was (correctly, IMO) not expecting much to happen because the amount of precedent that exists with MD5 being used as a way to say these documents haven't been swapped or tampered.

Re: The Curious Case of MD5

#137

Post by gsu2 »

hannob wrote: post #118
pxx wrote: post #89

Earlier quoted context omitted.

> I can't count the number of times I've seen people say "md5 is fine for use case xyz" where in some counterintuitive way it wasn't fine. Help us out by describing a time when this happened. MD5's weaknesses are easily described, and importantly, it is still (second) preimage resistant. I agree that upgrade is likely your best bet. But I've found the other direction of bad reasoning is a more pernicious trap to fall…

> Help us out by describing a time when this happened. Linus Torvalds saying that SHA-1 is okay for git, while it is used for Git signatures as well. Signatures are a classic "you need collission resistance to have safe signatures, but people are often confused about it" case.

I might be mistaken, but wouldn't a git signature already be signing trusted things (i.e. the person making the original signature is trusted), making any attack enabled by the input hash function a second preimage attack (i.e. an attacker onky knows the trusted input, not anything private like the signing key)?

Hash collisions mean you can't trust signatures from _untrusted_ sources, but git signatures don't seem to fit that situation.

Re: The Curious Case of MD5

#138

Post by ivanbakel »

masklinn wrote: post #120
ivanbakel wrote: post #105

Earlier quoted context omitted.

But what's the relevance of collision resistance, without a meaningful attack surface?

Ensuring every sequence is uniquely identified. Although they still want to avoid those, non-cryptographic hash functions often care a lot less about collision resistance, which is a problem when fingerprinting, which is the use case here. The alternative to a CHF in this case is not a non-cryptographic hash function, it's a dedicated fingerprinting scheme (like Rabin fingerprints). But a CHF is a perfectly good fing…

Those properties are not a direct result of a function being collision-resistant, which is a property that only makes sense in adversarial contexts. If nobody is trying to produce collisions, it doesn't matter if they're easy or hard to find.

You might care that the output hashes are well-distributed for your closely-related input data, but as the comment you replied to above points out, there are non-cryptographic functions with good avalanche properties which would satisfy that need without being collision-resistant.

Re: The Curious Case of MD5

#139

Post by Moru »

tesdinger wrote: post #130
NoboruWataya wrote: post #117

Earlier quoted context omitted.

If someone commits a crime, it's not the victim of the crime that has to pay for the lawyer to prosecute them.

Kim Yong Un is not going to make a bank transfer out of North Korea to pay for your Lawyers after they hacked you.

Exactly

Re: The Curious Case of MD5

#140

Post by oconnor663 »

adrian_b wrote: post #83
colmmacc wrote: post #41

Earlier quoted context omitted.

You say to use SHA2, but TFA says to use SHA3 or Blake. I think your recommendation is the better one, but I feel like teasing out why because it's interesting. Firstly ... the NIST recommendation TFA links doesn't just recommend SHA-3, it actually says "Federal agencies should use SHA-2 or SHA-3 as an alternative to SHA-1." SHA-2 and SHA-3 are both valid and recommended hash functions by NIST. And while 3 is higher…

SHA-3 (also BLAKE2 or BLAKE3) is definitely more secure for very large documents than SHA-2. The security (i.e. the difficulty in finding collisions) decreases with the length of the document for SHA-2 and it stays constant for SHA-3. Nevertheless, it is unlikely that typical legal documents are big enough for this to matter, except when the hashes would be e.g. for entire seized HDDs or SSDs, so SHA-2 is an acceptab…

> The security (i.e. the difficulty in finding collisions) decreases with the length of the document for SHA-2

Could you spell this part out for me?

Post reply on HN