Live data from Hacker News

A new hash algorithm for Git

lwn.net

231–240 of 240 posts

Re: A new hash algorithm for Git

#231
post #193

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Merkle_tree

"blockchain" is self-descriptive, easier to pronounce (only two syllables instead of three), and easier to spell correctly. :-)

"blockchain technology" is a lot more syllables. Plus there's the downside of sounding like a loon.

Re: A new hash algorithm for Git

#233

I was interested in how fossil handled the SHA1 transition, and found this nicely explained as below: https://fossil-scm.org/home/doc/trunk/www/hashpolicy.wiki

Fossil's main author is chiming in the discussion of this on Fossil's forums: ( https://fossil-scm.org/forum/forumpost/50a5bea5fb ) > That's appalling. Fossil's implementation doesn't require a conversion. “This is a key point, that I want to highlight. I'm sorry that it wasn't made more clear in the LWN posting nor in the HN discussion. “With Fossil, to begin using the new SHA3 hash algorithm, you just upgrade your…

So if a repo has anyone commit to it using a new binary, then anyone accessing the repo will need the new binary as well?

Re: A new hash algorithm for Git

#234

Earlier quoted context omitted.

1.5 Chosen-prefix collision: Given a prefix A, generate two values AB and AC, where B and C differ but are both prefixed with A. (AX is A concatenated with X). This exists for SHA1. It's more powerful than a basic collision wheri you can't pick the prefix, but weaker than either type of pre-image.

It's worth noting that this attack is a property of the Merkle–Damgård hash construction, not of SHA-1 specifically, which means SHA-2 (Git's path forward) is also vulnerable: https://en.wikipedia.org/wiki/Merkle%E2%80%93Damg%C3%A5rd_co... https://www.reddit.com/r/crypto/comments/44p5jc/eli5_why_are... Fossil uses SHA-3, which has an entirely different construction, which is not at this time known to have a similar w…

No, it's not the same as length extension. SHA1 is vulnerable to Chosen Prefix collisions. SHA2 doesn't have any known collision attacks faster than the birthday bounded brute force attack, let alone any chosen prefix collisions, but both do have length extension attacks. Also length extension isn't specific to Merkle–Damgård, though all Merkle–Damgård hashes are vulnerable to it without mitigations (like truncation of the output).

Re: A new hash algorithm for Git

#235
post #202

Earlier quoted context omitted.

AMD Zen supports SHA extensions across all SKUs. Here are `openssl speed` numbers on an AMD EPYC 3201: type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes blake2s256 46720.33k 187461.21k 305314.65k 373840.55k 398207.66k 401528.15k blake2b512 38423.44k 155318.81k 422325.08k 592401.75k 674843.31k 681743.70k sha256 84620.44k 279840.47k 723573.76k 1199678.81k 1484693.50k 1510484.65k sha512 33854.38k 135674…

Assuming Blake3 will be across the board 43% faster (7 instead of 10 rounds) than 32-bit blake2s256, we would get: Blake3 SHA-256 66743 84620 Tiny 534057 1199679 Medium (1024 bytes) 573611 1510485 Largeish (16384 bytes) This is based on the parent’s numbers with a fudge factor to account for Blake3 being a faster version of blake2s256 (i.e. the 32-bit version of Blake2 which is the only version in Blake3) Of course,…

The BLAKE3 tree mode also takes advantage of SIMD parallelism on a single core, which ends up being a larger effect than the reduced number of rounds. At 2-4 KiB of input (depending on the implementation) it's 2x faster than BLAKE2s on my laptop. Where AVX2 and AVX-512 are supported, those kick in at 8 KiB and 16 KiB of input respectively, widening the difference further. The red bar chart at https://github.com/BLAKE3-team/BLAKE3 is a single-threaded measurement on a machine that supports AVX-512.

Re: A new hash algorithm for Git

#236
post #211
post #188

Earlier quoted context omitted.

> if I understand correctly then Fossil's migration is straightforward because they did not address the same issues Git chose to. I think more is at play here. (1) You can set Fossil to ignore all SHA1 artifacts using the "shun-sha1" hash policy. (2) The excess complication in the Git migration strategy is likely due to the inability of the underlying Git file formats to handle two different hash algorithms in the sa…

(2) The excess complication in the Git migration strategy is likely due to the inability of the underlying Git file formats to handle two different hash algorithms in the same repository at the same time. But, I could be wrong. Post a rebuttal if you have evidence to the contrary. It seems unfair to demand a rebuttal when you are the one who made the claim. According to the article at least, the difficulty stems main…

> the difficulty stems mainly from their migration strategy, for converting all existing SHA1 hashes.

That's essentially the same difficulty, since the only strategy for doing this that has been historically proven to work seamlessly and painlessly involves being able to handle both hash algorithms in the same repository at the same time.

Re: A new hash algorithm for Git

#237
post #111

Earlier quoted context omitted.

I like to start a repo with an "empty" commit, that is to say its tree is the magic 4b825dc. https://news.ycombinator.com/item?id=18342763 I wonder if it would still be practically possible to manipulate the commit id.

Wow! I wouldn't have guessed that Git had that vulnerability. Fossil solves it easily: creating a new repo involves generating a random project code (a nonce) which goes into the hash of the first commit, so that even two identical commit sequences won't produce identical blockchains. Fossil lets you force the project ID on creating the repo, but the capability only exists for special purposes.

It doesn't seem to be a vulnerability at all

Re: A new hash algorithm for Git

#238
post #192

Earlier quoted context omitted.

Honest question: what are the use cases in Git where hash computation speed is a meaningful optimization?

My experience in developing and maintaining Fossil is that the hashing speed is not a factor, unless you are checking in huge JPEGs or MP3s or something. And even then, the relative performance of the various hash algorithms is not enough to worry about.

Thanks for the insight. My intuition was kind of the same, but on modern hardware computing the digest-style (as opposed to cryptographic, slow-by-design) hash is essentially imperceptible for payloads in the low MBs -- and much above that is a use case for LFS.

Re: A new hash algorithm for Git

#239
post #71

Excuse my ignorance, but couldn’t they just add a SHA256 hash to commit objects (or some new commit-verify object) of the entire trees current concatenated content, leave everything else SHA1 and get the same benefit without rewriting the entire thing from the ground up? Git could even do that as part of the git gc step slowly over time - tag commits with a secondary hash. Rewriting the whole thing including every gi…

You can't change past commits to add that hash (without changing all commit hashes), so this method could only protect new commits. For any existing repo this would lead to a very weird security model: We admit that sha1 hashes are broken, and only guarantee that commits made by git versions newer than git x.x.x are safe from after-the-fact modification (or alternatively only commits made after date X).

[deleted]

Re: A new hash algorithm for Git

#240

Earlier quoted context omitted.

> That cute rhetoric will not fool anyone. Well, let's see, the Fossil equivalents are: 1. Do nothing at all for a conversion from the SHA-1 to SHA-3 — yes, 3, not 2 as in Git! — because it's automatic for months now and dead easy going back 3 years now. ( https://www.fossil-scm.org/fossil/doc/trunk/www/hashpolicy.w... ) 2. "fossil diff" 3. "fossil ci" 4. Why are you rebasing in the first place, again? https://www.fo…

Articles like this are eye opening to me, in a bad way. Every once in a while, I get really curious about giving Fossil a try, because it does have some legitimately cool ideas, and then I see the documentation saying things like: > Rebasing is the same as lying And I think, "Holy crud do I not want to be part of this community." The nice thing about Git is that (within reason) once I understood it, I was able to use…

>You've never accidentally committed a password to repo

You mean your password is something funny? In that case random.org can help you with password generation.

Post reply on HN