Live data from Hacker News

A new hash algorithm for Git

lwn.net

151–160 of 240 posts

Re: A new hash algorithm for Git

#151

I’m already seeing a lot of discussion both here and over at LWN about which hash algorithm to use. The Git team made the right choice: SHA2-256 is the best choice here; it has been around for 19 years and is still secure, in the sense that there are no known attacks against it. Both BLAKE[2/3] and SHA-3 (Keccak) have been around for 12 years and are both secure; just as BLAKE2 and BLAKE3 are faster reduced round var…

Honest question: what are the use cases in Git where hash computation speed is a meaningful optimization?

Rewriting all repos from SHA1 to hash-next?

Re: A new hash algorithm for Git

#152
post #86
post #55

Earlier quoted context omitted.

There is a downside that this would mean commit-prefixes remain sensitive to collisions. Hence anyone checking out a commit by a hash-prefix would still be vulnerable. Not a dealbreaker by far, but still a slight mark against this solution.

Does git have code to detect whether a hash prefix is ambiguous? I know that if you use a short prefix (which is more likely to be shared by multiple objects), git will output an error message staying that the object reference is ambiguous IIRC.

Yes.

Re: A new hash algorithm for Git

#153
post #117

Earlier quoted context omitted.

OP's link says it is "subscription-only content," but it is still publicly available. It says that it has been "made available by an LWN subscriber." How does that work?

Subscribers receive a sharing link which can be used to share articles with friends. They tolerate sharing on HN because it brings in new customers.

Damn, lwn is sweeet !! :)

Re: A new hash algorithm for Git

#154
post #139

Earlier quoted context omitted.

Perhaps suggest an alternative? It may help understand why this was chosen.

One alternative would be to just do lookups in both hash databases (until SHA1 is fully migrated away from), and reject invocations that conflict. Git's CLI already rejects ambiguous short hash prefixes for SHA1, it could easily reject ambiguous prefixes between SHA1 and SHA256 and otherwise allow unique prefixes for either hash. This would be pretty ergonomic for users.

For most cases that would suffice and would be ergonomic, but what if a full SHA1 also qualifies as a prefix of one or more SHA256, and you want the SHA1? There's still a need for a mechanism to disambiguate for these cases, even if it ends up very rarely needed.

Re: A new hash algorithm for Git

#155
post #111

I'll have to update my program which generates vanity hashes. I do enjoy starting projects with an obligatory "Initial Commit" with a deadbeef SHA-1

I like to start a repo with an "empty" commit, that is to say its tree is the magic 4b825dc. https://news.ycombinator.com/item?id=18342763 I wonder if it would still be practically possible to manipulate the commit id.

Yep. You can inject arbitrary metadata into the git commit object and the git cli ignores it, other than including it in the hash. E.g., https://github.com/kevinwallace/gitbrute , https://github.com/kevinwallace/gitbrute/commit/0001111 .

Re: A new hash algorithm for Git

#156
post #154
post #139

Earlier quoted context omitted.

One alternative would be to just do lookups in both hash databases (until SHA1 is fully migrated away from), and reject invocations that conflict. Git's CLI already rejects ambiguous short hash prefixes for SHA1, it could easily reject ambiguous prefixes between SHA1 and SHA256 and otherwise allow unique prefixes for either hash. This would be pretty ergonomic for users.

For most cases that would suffice and would be ergonomic, but what if a full SHA1 also qualifies as a prefix of one or more SHA256, and you want the SHA1? There's still a need for a mechanism to disambiguate for these cases, even if it ends up very rarely needed.

You're talking about a 160 bit truncated hash collision on SHA256, which is extraordinarily unlikely if SHA256 is not itself completely broken (moreso than SHA1 already is!). I don't think any syntax is needed for that in the porcelain CLI; it could be handled with non-user-facing commands if it ever came up (it won't).

Re: A new hash algorithm for Git

#158
post #156
post #154

Earlier quoted context omitted.

For most cases that would suffice and would be ergonomic, but what if a full SHA1 also qualifies as a prefix of one or more SHA256, and you want the SHA1? There's still a need for a mechanism to disambiguate for these cases, even if it ends up very rarely needed.

You're talking about a 160 bit truncated hash collision on SHA256, which is extraordinarily unlikely if SHA256 is not itself completely broken (moreso than SHA1 already is!). I don't think any syntax is needed for that in the porcelain CLI; it could be handled with non-user-facing commands if it ever came up (it won't).

> extraordinarily unlikely if SHA256 is not itself completely broken (moreso than SHA1 already is

I was hoping I captured that by saying "very rarely". However, if SHA1 collisions can be made willingly, doesn't that mean that one can also willingly make a SHA1 hash that matches with the prefix of an existing SHA256 hash?

Re: A new hash algorithm for Git

#159
post #71

Excuse my ignorance, but couldn’t they just add a SHA256 hash to commit objects (or some new commit-verify object) of the entire trees current concatenated content, leave everything else SHA1 and get the same benefit without rewriting the entire thing from the ground up? Git could even do that as part of the git gc step slowly over time - tag commits with a secondary hash. Rewriting the whole thing including every gi…

You can't change past commits to add that hash (without changing all commit hashes), so this method could only protect new commits. For any existing repo this would lead to a very weird security model: We admit that sha1 hashes are broken, and only guarantee that commits made by git versions newer than git x.x.x are safe from after-the-fact modification (or alternatively only commits made after date X).

What if we use the exploit to add the new data but keep the sha1 the same? :)

Re: A new hash algorithm for Git

#160

Earlier quoted context omitted.

That cute rhetoric will not fool anyone. Common git workflows use fairly succinct git commands: git diff git commit -p git rebase -i HEAD~3 The command quoted in my original comment is just this we strip away the SHA256 garbage: git log abac87a..f787cac (Or maybe it is: git log abac87a^..f787cac^ I cannot guess whether the ^ operator still has the same meaning or whether it is part of this ^{sha...} notation.) The ha…

You are now ignoring the fact that in the initial quote you objected to was the intentionally tongue-in-cheek: > 'For a Git user interface this is relatively straightforward and concise'. It kinda looks like you missed the joke and are now doubling-down on your disagreement. The author does not think the proposed example is reasonable. You're in agreement.

Since git is something that I rely on for everyday use, and long-term data stroage, and its development is being threatened by the inclusion of moronic changes I completely disagree with, I'm completely unreceptive to jokes. This is no laughing matter.
Post reply on HN