Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A new hash algorithm for Git

lwn.net

Post reply on HN
Print view View original

131–140 of 240 posts

Re: A new hash algorithm for Git

#131

Post by velcrovan »

brobdingnagians wrote: post #74

I was interested in how fossil handled the SHA1 transition, and found this nicely explained as below: https://fossil-scm.org/home/doc/trunk/www/hashpolicy.wiki

Fossil's main author is chiming in the discussion of this on Fossil's forums:

(https://fossil-scm.org/forum/forumpost/50a5bea5fb)

> That's appalling. Fossil's implementation doesn't require a conversion.

“This is a key point, that I want to highlight. I'm sorry that it wasn't made more clear in the LWN posting nor in the HN discussion.

“With Fossil, to begin using the new SHA3 hash algorithm, you just upgrade your fossil binary. No further actions, workflow changes, disruptions, or thought are required on the part of the user.

* “Old check-ins with SHA1 hashes continue to use their SHA1 hash names.”

* “New check-ins automatically get more secure SHA3 hash names.”

* “No repository conversions need to occur”

* “Given a hash prefix, Fossil automatically figures out whether it is dealing with a SHA1 or a SHA3 hash”

* “No human brain-cycles are wasted trying to navigate through a hash-algorithm cut-over.”

“Contrast this to Git, where a repository must be either all-SHA1 or all-SHA2. Hence, to cut-over a repository requires rebuilding the repository and in the process renaming all historical artifacts -- essentially rebasing the entire repository. The historical artifact renaming means that external links to historical check-ins (such as in tickets) are broken. And during the transition period, users have to be constantly aware of whether they are using SHA1 or SHA2 hash names. It is a big mess. It is no wonder, then, that few people have been eager to transition their repositories over to the newer SHA2 format.”

Re: A new hash algorithm for Git

#132

Post by jolmg »

kazinator wrote: post #125
microtherion wrote: post #123

Earlier quoted context omitted.

Note the qualifier "for a Git user interface". The average git command is along the lines of "git ph-nglui --mglw=nafh Cthulhu...R'lyeh -- wgah^nagl fhtagn"

That cute rhetoric will not fool anyone. Common git workflows use fairly succinct git commands: git diff git commit -p git rebase -i HEAD~3 The command quoted in my original comment is just this we strip away the SHA256 garbage: git log abac87a..f787cac (Or maybe it is: git log abac87a^..f787cac^ I cannot guess whether the ^ operator still has the same meaning or whether it is part of this ^{sha...} notation.) The ha…

> I cannot guess whether the ^ operator still has the same meaning or whether it is part of this ^{sha...} notation.)

This isn't the first ^{...} notation. The manpage gitrevisions(7) also mentions ^{/} for referencing a commit based on a regular expression of its commit message, like

  git checkout 'add-search^{/finished query builder}'
Though, this new notation is probably more in-line with the notation ^{}, which lets you disambiguate what you put in as in deadbeef^{tag}, so that it's not confused with deadbeef^{commit}.

EDIT: The article doesn't mention it, but I imagine one interpretation would take precedence and cause git to issue a warning when it's ambiguous. Right now, if I tag a commit with the hash of another commit, its interpretation as a tag takes precedence and I get a warning at the top, "warning: refname '368bc6e' is ambiguous." That would mean you'd only ever write ^{sha256} when the provided part of a sha256 hash is ambiguous with an existing sha1 hash or something else like a tag. That's also vice versa with ^{sha1}.

Re: A new hash algorithm for Git

#133

Post by Nullabillity »

OJFord wrote: post #120
gregmac wrote: post #112

Earlier quoted context omitted.

My inclination is that protecting only new commits might be enough, but it gets me thinking: What would a practical attack on this look like, assuming sha1 was broken? Let's say I'm trying to insert a line of code that does something nefarious, and that it's now trivial to generate "magic text" you can stick anywhere in a file (eg, inside a comment at the end of a line) to get any desired sha1 hash. Are all the other…

> My inclination is that protecting only new commits might be enough Why? It's not the same as saying 'versions after vX are safe', it's the same as saying 'any unsafety after vX was there before, not introduced since' (both with 'as a result of SHA-1 collision' qualifiers of course). > Can I push my replacement commit to another repository (eg, github)? Would even force push work? Implementation dependent I suppose,…

Git commits are snapshots, not diffs. Each commit contains a tree, which contains a list of files and their respective hashes. As long as its whole tree is SHA-256 then a commit should be safe, regardless of its history.

The downside to the migration would be that all unchanged files would be stored twice (once identified by SHA1, once identified by SHA-256). But you could work around that by hardlinking identical files.

Re: A new hash algorithm for Git

#135

Post by sunil_saini »

The above article suggests that Sha-1 collision is infeasible because attacker has to come up with code that not only generate same hash but also benefit him. But can't he just add some malicious code and add some random text in comments to produce same hash?

Re: A new hash algorithm for Git

#136

Post by zokier »

Does anyone know if a standard format for sort of tagged-union hash type, something similar as crypt format for passwords? Feels like everyone is needing to support multiple hash types at some point, and basically needs to reinvent that particular wheel again and again.

Re: A new hash algorithm for Git

#137

Post by tzs »

wongarsu wrote: post #81
donatj wrote: post #71

Excuse my ignorance, but couldn’t they just add a SHA256 hash to commit objects (or some new commit-verify object) of the entire trees current concatenated content, leave everything else SHA1 and get the same benefit without rewriting the entire thing from the ground up? Git could even do that as part of the git gc step slowly over time - tag commits with a secondary hash. Rewriting the whole thing including every gi…

You can't change past commits to add that hash (without changing all commit hashes), so this method could only protect new commits. For any existing repo this would lead to a very weird security model: We admit that sha1 hashes are broken, and only guarantee that commits made by git versions newer than git x.x.x are safe from after-the-fact modification (or alternatively only commits made after date X).

Couldn't they make a table that contains a list of all the old objects by SHA1 hash, for for each contains the new SHA256 hash of that object, and then commit this table in the repository?

Re: A new hash algorithm for Git

#138

Post by WorldMaker »

arve0 wrote: post #110
wongarsu wrote: post #81

Earlier quoted context omitted.

You can't change past commits to add that hash (without changing all commit hashes), so this method could only protect new commits. For any existing repo this would lead to a very weird security model: We admit that sha1 hashes are broken, and only guarantee that commits made by git versions newer than git x.x.x are safe from after-the-fact modification (or alternatively only commits made after date X).

I’m not known with the internal data structure of git, but couldn’t you add the new hash as a commit in a new format “on the side”, leaving the original commit as is?

Git does have an commit-related object called a note that you can attach as a separate object. [1]

Presumably the proposed "hash translation store" could use an approach similar to notes, and include the hash translations as objects in the git database (hopefully in a way that could be signed by a tag).

[1] http://alblue.bandlem.com/2011/11/git-tip-of-week-git-notes....

Re: A new hash algorithm for Git

#139

Post by loeg »

scarejunba wrote: post #130
kazinator wrote: post #114

> There is, of course, a way to unambiguously give a hash value in the new Git code, and they can even be mixed on the command line; this example comes from the transition document: git --output-format=sha1 log abac87a^{sha1}..f787cac^{sha256} > For a Git user interface this is relatively straightforward and concise No, it isn't. It's a complete and utter user interface clusterfuck. Just say no to this insanity.

Perhaps suggest an alternative? It may help understand why this was chosen.

One alternative would be to just do lookups in both hash databases (until SHA1 is fully migrated away from), and reject invocations that conflict. Git's CLI already rejects ambiguous short hash prefixes for SHA1, it could easily reject ambiguous prefixes between SHA1 and SHA256 and otherwise allow unique prefixes for either hash. This would be pretty ergonomic for users.

Re: A new hash algorithm for Git

#140

Post by strenholme »

_verandaguy wrote: post #119
strenholme wrote: post #97

I’m already seeing a lot of discussion both here and over at LWN about which hash algorithm to use. The Git team made the right choice: SHA2-256 is the best choice here; it has been around for 19 years and is still secure, in the sense that there are no known attacks against it. Both BLAKE[2/3] and SHA-3 (Keccak) have been around for 12 years and are both secure; just as BLAKE2 and BLAKE3 are faster reduced round var…

Honest question: what are the use cases in Git where hash computation speed is a meaningful optimization?

It’s actually not a big deal with Git, which is why SHA2-256 is the right choice.
Post reply on HN