Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A new hash algorithm for Git

lwn.net

Post reply on HN
Print view View original

141–150 of 240 posts

Re: A new hash algorithm for Git

#141

Post by loeg »

Nullabillity wrote: post #133
OJFord wrote: post #120

Earlier quoted context omitted.

> My inclination is that protecting only new commits might be enough Why? It's not the same as saying 'versions after vX are safe', it's the same as saying 'any unsafety after vX was there before, not introduced since' (both with 'as a result of SHA-1 collision' qualifiers of course). > Can I push my replacement commit to another repository (eg, github)? Would even force push work? Implementation dependent I suppose,…

Git commits are snapshots, not diffs. Each commit contains a tree, which contains a list of files and their respective hashes. As long as its whole tree is SHA-256 then a commit should be safe, regardless of its history. The downside to the migration would be that all unchanged files would be stored twice (once identified by SHA1, once identified by SHA-256). But you could work around that by hardlinking identical fi…

This doesn't protect subdirectories unless you rewrite the entire tree structure with SHA256. I don't know if Git does that now, or not. Git generally points to unmodified subdirectories with the existing content hash; if the SHA1 is pointed to by SHA256, which is implied by the transition plan proposed in the grand-grandparent comment, then those subdirectories are essentially unprotected.

Re: A new hash algorithm for Git

#142

Post by loeg »

tzs wrote: post #137
wongarsu wrote: post #81

Earlier quoted context omitted.

You can't change past commits to add that hash (without changing all commit hashes), so this method could only protect new commits. For any existing repo this would lead to a very weird security model: We admit that sha1 hashes are broken, and only guarantee that commits made by git versions newer than git x.x.x are safe from after-the-fact modification (or alternatively only commits made after date X).

Couldn't they make a table that contains a list of all the old objects by SHA1 hash, for for each contains the new SHA256 hash of that object, and then commit this table in the repository?

Yep.

Re: A new hash algorithm for Git

#143

Post by seniorsassycat »

velcrovan wrote: post #131
brobdingnagians wrote: post #74

I was interested in how fossil handled the SHA1 transition, and found this nicely explained as below: https://fossil-scm.org/home/doc/trunk/www/hashpolicy.wiki

Fossil's main author is chiming in the discussion of this on Fossil's forums: ( https://fossil-scm.org/forum/forumpost/50a5bea5fb ) > That's appalling. Fossil's implementation doesn't require a conversion. “This is a key point, that I want to highlight. I'm sorry that it wasn't made more clear in the LWN posting nor in the HN discussion. “With Fossil, to begin using the new SHA3 hash algorithm, you just upgrade your…

The way I read the fossil's authors comments, old commits continue to use sha1 hashes. A repository will be vulnerable to sha1 collision attacks as long as there is an object in the repository that has not been hashed with the new algorithm.

For example, floppy.c could be replaced in a repo with file with the same sha1 hash as long as the last commit that modifies floppy.c used a sha1 hash.

Right?

Re: A new hash algorithm for Git

#144

Post by strenholme »

KMag wrote: post #129
strenholme wrote: post #97

I’m already seeing a lot of discussion both here and over at LWN about which hash algorithm to use. The Git team made the right choice: SHA2-256 is the best choice here; it has been around for 19 years and is still secure, in the sense that there are no known attacks against it. Both BLAKE[2/3] and SHA-3 (Keccak) have been around for 12 years and are both secure; just as BLAKE2 and BLAKE3 are faster reduced round var…

SHA-256 is probably the right choice, but I don't think it's as obvious as you suggest, given SHA-512/256. SHA-512/256 is a standard peer-reviewed and well-studied way to run SHA-512 with a different initial state and then truncate output to 256 bits. This is heavy bikeshedding, but SHA-512/256 would be a more conservative choice than SHA-256. Under standard assumptions, SHA-256 is no weaker than SHA-512. The structu…

SHA-512/256 is a lot newer than SHA2-256 (usually called SHA-256, but I prefer the SHA2 prefix to make it clear that it’s a very different beast than SHA3-256), and its speed on 32-bit CPUs is less than optimal, so I don’t see it as being a more conservative choice. In terms of security, it uses the same 19-year-old unbroken algorithm as SHA2-256.

I am aware of the length extension issues, but they are not relevant for Git’s use case.

In terms of support, SHA-512/256 has, as you mentioned, less hardware acceleration support, and it’s also not supported in a lot of mainstream programs like GNU Coreutils. I also know that some companies mandate using SHA2-256 whenever a cryptographic hash is needed.

Git made the right choice with SHA2-256: It’s the most widely supported secure cryptographic hash out there.

Re: A new hash algorithm for Git

#145

Post by loeg »

arve0 wrote: post #110
wongarsu wrote: post #81

Earlier quoted context omitted.

You can't change past commits to add that hash (without changing all commit hashes), so this method could only protect new commits. For any existing repo this would lead to a very weird security model: We admit that sha1 hashes are broken, and only guarantee that commits made by git versions newer than git x.x.x are safe from after-the-fact modification (or alternatively only commits made after date X).

I’m not known with the internal data structure of git, but couldn’t you add the new hash as a commit in a new format “on the side”, leaving the original commit as is?

This is kind of what rewriting the repo is. Yes, you could leave the SHA1 commit tree around afterwards (i.e., for convenience of existing URLs), but you wouldn't want to keep SHA1 around as the authoritative hashname.

Re: A new hash algorithm for Git

#146

Post by gouggoug »

kazinator wrote: post #114

> There is, of course, a way to unambiguously give a hash value in the new Git code, and they can even be mixed on the command line; this example comes from the transition document: git --output-format=sha1 log abac87a^{sha1}..f787cac^{sha256} > For a Git user interface this is relatively straightforward and concise No, it isn't. It's a complete and utter user interface clusterfuck. Just say no to this insanity.

> For a Git user interface this is relatively straightforward and concise

You forgot to include the end of that sentence, that acknowledges your issue with it:

> , but one can still imagine that users might tire of it relatively quickly.

Re: A new hash algorithm for Git

#147

Post by allover »

kazinator wrote: post #125
microtherion wrote: post #123

Earlier quoted context omitted.

Note the qualifier "for a Git user interface". The average git command is along the lines of "git ph-nglui --mglw=nafh Cthulhu...R'lyeh -- wgah^nagl fhtagn"

That cute rhetoric will not fool anyone. Common git workflows use fairly succinct git commands: git diff git commit -p git rebase -i HEAD~3 The command quoted in my original comment is just this we strip away the SHA256 garbage: git log abac87a..f787cac (Or maybe it is: git log abac87a^..f787cac^ I cannot guess whether the ^ operator still has the same meaning or whether it is part of this ^{sha...} notation.) The ha…

You are now ignoring the fact that in the initial quote you objected to was the intentionally tongue-in-cheek:

> 'For a Git user interface this is relatively straightforward and concise'.

It kinda looks like you missed the joke and are now doubling-down on your disagreement.

The author does not think the proposed example is reasonable. You're in agreement.

Re: A new hash algorithm for Git

#148

Post by loeg »

zokier wrote: post #136

Does anyone know if a standard format for sort of tagged-union hash type, something similar as crypt format for passwords? Feels like everyone is needing to support multiple hash types at some point, and basically needs to reinvent that particular wheel again and again.

It isn't too bad to just exhaustively look up provided hashes in all your databases (at least, for Git). You should probably only support 1 primary hash at a time, and 1 additional legacy hash for migration purposes. This makes lookup twice as expensive; for git, this is not usually the slow part (the slow part is 'git status' having to compare the entire local filesystem checkout to the repo).

Re: A new hash algorithm for Git

#150

Post by jakeogh »

tialaramex wrote: post #61
jakeogh wrote: post #2

Is there an archive of crypto related future predictions? How long until a specified length preimage attack can break bittorrent blocks? I remember a paper published a ~decade ago estimating very short (well funded) ASIC sha1 collisons. Anyone have that ref? EDIT: Should I have not said preimage? My understanding is bittorrent is broken (by DDoS, not infohash(?)) if you can make a bad block that matches the length an…

> EDIT: Should I have not said preimage? My understanding is bittorrent is broken (by DDoS, not infohash(?)) if you can make a bad block that matches the length and sha1 of a target block. There are three different attacks 1. Collision, which is practical (expensive but practical) for SHA-1 today, lets somebody make two documents A and B which have the same hash. This is only useful if you can fool people somehow int…

Thanks. Is there a name for collision with the same preimage size?
Post reply on HN