The beginning of Git supporting other hash algorithms • Hacker News Board

Re: The beginning of Git supporting other hash algorithms

#121

Post by CopperWing » Mon, Mar 20, 2017, 10:34 AM UTC

snakeanus wrote: post #78

dahart wrote: post #56

Earlier quoted context omitted.

Linus has explained why he picked SHA-1. I'm not Linus, and I'm not defending his choice, but he has said repeatedly that git's hash is primarily for indexing and error correction, and not primarily for security. Clearly he felt like SHA-1 was "good enough". And if you have something that's "good enough" there are reasons not to write code for alternatives you're not going to use.

>but he has said repeatedly that git's hash is primarily for indexing and error correction, and not primarily for security And he was wrong as openpgp signatures on commits and tags are a thing. Not sure when that feature was introduced however, I doubt that it existed in the first version of git. That being said he should have changed the hash function the moment that feature was introduced.

Signatures were introduced in git as part of the response to the kernel.org hack in 2011.

Re: The beginning of Git supporting other hash algorithms

#122

Post by SQLite » Mon, Mar 20, 2017, 11:25 AM UTC

david-given wrote: post #120

dchest wrote: post #93

Earlier quoted context omitted.

I'm a long time fan of Fossil and contributed a bit to its development (in particular, TLS support and some protections against timing attacks). I'm not sure where you found provocative language, but let me try to explain it here more clearly. Design deficiency (This is unrelated to the choice of hash.) Fossil stores blobs as-is. A file containing "hello world" will be stored as "hello world" and referenced as HASH("…

Thank you very much for posting this --- this is exactly the information which was missing from the Twitter thread! (My concern was that the twitter basically contained nothing of any content: no technical details, no link to a blog post, nothing which can be checked or verified... which makes it indistinguishable from insinuation; and so I have to dismiss what you said out of hand.) (Regarding provocative language:…

When adding an attachment in Fossil, if the attachment is syntactically similar to a structural artifact (such as a manifest), then the attachment is compressed prior to being hashed and stored, thus making it very dissimilar to a manifest and incapable of being confused with a manifest. Hence, it is not possible for someone to add a new manifest as an attachment and have that confuse the system. Furthermore, there is an audit trail so that should an attacker discover and exploit some bug in the previous mechanism and manage to get a manifest inserted via an attachment, then the rogue manifest can be easily identified and "shunned".

Users with commit privileges are granted more trust and do have the ability to forge manifests. But as before, there is an audit trail and rogue manifests (and the users that insert them) can be detected and dealt with after the fact.

Structural artifacts have a very specific and pedantic format. You can forge a structural artifact, but you will never generate one by accident during normal software development activities.

Re: The beginning of Git supporting other hash algorithms

#123

Post by harshreality » Mon, Mar 20, 2017, 5:23 PM UTC

copperx wrote: post #110

harshreality wrote: post #83

Earlier quoted context omitted.

The hash function may not matter for overall git performance in virtually all dev machine setups, but there will be a (maybe tiny, maybe larger, depending on the repo and disk io speed) difference in cpu utilization and heat generation, right?

That's a silly thing to worry about when you're developing Ruby or Java applications. My PC boots faster than the Rails console or IntelliJ.

The broader point is this: code repositories are not the only things git is used for.

Re: The beginning of Git supporting other hash algorithms

#124

Post by david-given » Mon, Mar 20, 2017, 5:55 PM UTC

SQLite wrote: post #122

david-given wrote: post #120

Earlier quoted context omitted.

Thank you very much for posting this --- this is exactly the information which was missing from the Twitter thread! (My concern was that the twitter basically contained nothing of any content: no technical details, no link to a blog post, nothing which can be checked or verified... which makes it indistinguishable from insinuation; and so I have to dismiss what you said out of hand.) (Regarding provocative language:…

When adding an attachment in Fossil, if the attachment is syntactically similar to a structural artifact (such as a manifest), then the attachment is compressed prior to being hashed and stored, thus making it very dissimilar to a manifest and incapable of being confused with a manifest. Hence, it is not possible for someone to add a new manifest as an attachment and have that confuse the system. Furthermore, there i…

Isn't the problem here, though, that they can be forged? If, by attaching a structural artifact in the correct format to a ticket, we're effectively allowing people without commit privileges to make commits --- potentially anonymous people.

I agree that this isn't likely to happen by accident, but Fossil servers are usually public facing, so we have to worry about malice as well.

Re: The beginning of Git supporting other hash algorithms

#125

Post by wolf550e » Tue, Mar 21, 2017, 12:53 PM UTC

harshreality wrote: post #82

wolf550e wrote: post #72

Earlier quoted context omitted.

Yeah, I would have gone with BLAKE2. It's much faster than SHA-256 and SHA3-256: https://blake2.net/skylake.png

sha3 will probably get hw accel eventually. Blake2 is less likely to. It's like the dilemma between chacha20 and a stream cipher mode for aes. An argument could be made for either, depending on application specifics and available hardware.

But like its ancestor chacha, blake2 is fast on anything that has SIMD.

Re: The beginning of Git supporting other hash algorithms

#126

Post by makomk » Thu, Mar 23, 2017, 12:14 AM UTC

rocqua wrote: post #31

amluto wrote: post #29

Earlier quoted context omitted.

> Technically MD5(128bits) and SHA1(160bits) lengths are sufficient for hashes, but they had cryptographic weaknesses Not true. A 128-bit hash gets collisions after ~2^64 tries. A big cluster can find targeted 128-bit collisions. To attack something like git, the entire attack can be done offline. The big MD5 X.509 break needed cryptanalysis to make it day I because the attack needed to happen in real time.

The threat models in which 64 bits of security (by birthday attack on 128bit hashes) is insufficient are really limited.

For comparison, this is about the same level of cost as the recent collision attack on SHA-1. So even a perfect 128-bit hash is at least as broken as SHA-1 is.

Re: The beginning of Git supporting other hash algorithms

#127

Post by amluto » Fri, Mar 24, 2017, 9:46 PM UTC

rocqua wrote: post #60

IncRnd wrote: post #50

Earlier quoted context omitted.

That is misleading, since a birthday attack is not required. The security strength of hashes is not measured by the length of the hashes. md5 was first "broken" in 1995. As of 10 years ago, a collision attack took only a few seconds. Plus, there are a _number_ of other attacks on the hash.

The argument I replied to concerned best-case for length. I.e. a perfect hash at 128 bits delivers 64 bits of security against collisions. (Note the 'perfect' part) 64 bits of security is good enough against most non-nation state actors. Obviously, MD5 (and sha-1) aren't anywhere near perfect hashes. And obviously, you need to look at more than length when judging a hash. Basically my point was that md5's hash length…

The bitcoin network does 2^64 hashes in about 10 seconds. 10 seconds of bitcoin network is a far cry from nation state only.

Re: The beginning of Git supporting other hash algorithms

#128

Post by IncRnd » Mon, Mar 27, 2017, 8:23 PM UTC

palunon wrote: post #118

IncRnd wrote: post #43

Earlier quoted context omitted.

Yes, Linus wrote that SHA1 isn't here for security, but that was a glaring misunderstanding of security on his part. Integrity protection of source code is a security function.

I think it's mainly due to a different threat model. Linus only pulls from his trusted lieutenants, who are unlikely to try to attack the source in that way (it's way easier to simply hide a bad commit in the lot, no need to fiddle with SHA1). They do the same. The rest of the code is sent through mailing lists as patches, so the hash is irrelevant. SHA1 here protects against "random" corruption (which is more than s…

I understand your point. However, that doesn't take into account defense in depth which says that more than a single control should be in place.