The beginning of Git supporting other hash algorithms • Hacker News Board

Re: The beginning of Git supporting other hash algorithms

#41

Post by ufmace » Sun, Mar 19, 2017, 4:14 PM UTC

CydeWeys wrote: post #8

ebbv wrote: post #4

Earlier quoted context omitted.

Can it really be that simple though? If you are using a newer version of Git on your repo which is committing only with the newer hash and I try to clone your repo with an older version I will be unable to do so. I guess maybe that's acceptable though?

I think backwards incompatibility would be acceptable. Add read support for the new format to git, but then don't have widespread repositories using the new format until some period of time later. By the time they do become commonplace, everyone should already be running a version of git supporting them. It's not exactly hard to upgrade git in most situations anyway, just a simple invocation of: $ sudo $PKG_MGR upgra…

Well, it's not hard to update the command-line git client on Unix-y systems with package management. The trouble will be with the hundreds/thousands of other programs that use Git in various ways and are essential to development workflows in various places. Github themselves, Microsoft and Jetbrains IDEs, etc.

Re: The beginning of Git supporting other hash algorithms

#42

Post by taeric » Sun, Mar 19, 2017, 4:29 PM UTC

ufmace wrote: post #41

CydeWeys wrote: post #8

Earlier quoted context omitted.

I think backwards incompatibility would be acceptable. Add read support for the new format to git, but then don't have widespread repositories using the new format until some period of time later. By the time they do become commonplace, everyone should already be running a version of git supporting them. It's not exactly hard to upgrade git in most situations anyway, just a simple invocation of: $ sudo $PKG_MGR upgra…

Well, it's not hard to update the command-line git client on Unix-y systems with package management. The trouble will be with the hundreds/thousands of other programs that use Git in various ways and are essential to development workflows in various places. Github themselves, Microsoft and Jetbrains IDEs, etc.

Don't most of these go through the git codebase? Or just parse the output of standard commands? Curious what implications the hash has on those.

Re: The beginning of Git supporting other hash algorithms

#43

Post by IncRnd » Sun, Mar 19, 2017, 4:29 PM UTC

jlgaddis wrote: post #15

nuntius wrote: post #13

Earlier quoted context omitted.

Backwards compat requires that both old and new hashes work at the same time. A simple typedef is unlikely to handle all the semantics and space needed for such a change... It is often hard to generalize when N=1. Now that the N=1 use case is established and we are moving towards N=2, it is painfully obvious to all that a better abstraction is needed. Typedef or no, we would still need a full audit of the code to fin…

Perhaps you haven't read Linus' comments where he stated (more than a decade ago) that the usage of SHA1 here isn't for "security"? (Hint: that's why GPG signing commits is an option.)

Yes, Linus wrote that SHA1 isn't here for security, but that was a glaring misunderstanding of security on his part. Integrity protection of source code is a security function.

Re: The beginning of Git supporting other hash algorithms

#44

Post by mikeash » Sun, Mar 19, 2017, 4:30 PM UTC

dahart wrote: post #22

zoren wrote: post #10

Someone please remind me why the hash is not a type definition so the representation would only have to be changed in one place.

That's exactly what this change is. You mean why wasn't it that way before the change? Maybe because it wasn't ever needed before? Git's been good with only sha-1 for 12 years. Think about the flip side of your question... what were the alternatives 12 years ago, or 5 years ago? And why would someone write code for alternatives that aren't expected to be used and maybe don't exist? In my experience, generalizing ahea…

The first attack on SHA-1 was published in 2003. Git showed up in 2005. Not only should git have allowed for something else, it never should have used SHA-1 in the first place.

Re: The beginning of Git supporting other hash algorithms

#45

Post by koolba » Sun, Mar 19, 2017, 4:32 PM UTC

Since the majority of us are running x64 machines, will the hash be a truncated SHA-512/256 or will it be SHA-256? The former is significantly faster on x64 machines.

Re: The beginning of Git supporting other hash algorithms

#46

Post by joatmon-snoo » Sun, Mar 19, 2017, 4:35 PM UTC

koolba wrote: post #45

Since the majority of us are running x64 machines, will the hash be a truncated SHA-512/256 or will it be SHA-256? The former is significantly faster on x64 machines.

The RFC is still under discussion (there are a few plans going around) but the strong contender right now is SHA3-256, no truncation.

Re: The beginning of Git supporting other hash algorithms

#47

Post by evmar » Sun, Mar 19, 2017, 4:36 PM UTC

drostie wrote: post #36

bk2204 wrote: post #9

I'm the person who's been working on this conversion for some time. This series of commits is actually the sixth, and there will be several more coming. (I just posted the seventh to the list, and I have two more mostly complete.) The current transition plan is being discussed here: https://public-inbox.org/git/CA+dhYEViN4-boZLN+5QJyE7RtX+q6a...

I'm not quite so familiar with the Git internals, how do you deal with the problem of having different non-leaf nodes scattered through the directory tree? This might be a non-issue based on how Git stores the tree, but I can imagine one simple model where each directory would be a sort of "collection object", a binary encoding of a list of (filename, hash) pairs in filename order, and therefore the directory gets a…

https://git-scm.com/book/en/v2/Git-Internals-Git-Objects has descriptions of the objects. Both trees and commits are hashes over data that includes hashes of other objects so they must be different. The doc discusses converting them at transmission time, search for [convert to sha256] in it.

Re: The beginning of Git supporting other hash algorithms

#48

Post by lvh » Sun, Mar 19, 2017, 4:37 PM UTC

dahart wrote: post #22

zoren wrote: post #10

Someone please remind me why the hash is not a type definition so the representation would only have to be changed in one place.

That's exactly what this change is. You mean why wasn't it that way before the change? Maybe because it wasn't ever needed before? Git's been good with only sha-1 for 12 years. Think about the flip side of your question... what were the alternatives 12 years ago, or 5 years ago? And why would someone write code for alternatives that aren't expected to be used and maybe don't exist? In my experience, generalizing ahea…

SHA2 predates git by about four years.

Re: The beginning of Git supporting other hash algorithms

#49

Post by CydeWeys » Sun, Mar 19, 2017, 4:37 PM UTC

ufmace wrote: post #41

CydeWeys wrote: post #8

Earlier quoted context omitted.

I think backwards incompatibility would be acceptable. Add read support for the new format to git, but then don't have widespread repositories using the new format until some period of time later. By the time they do become commonplace, everyone should already be running a version of git supporting them. It's not exactly hard to upgrade git in most situations anyway, just a simple invocation of: $ sudo $PKG_MGR upgra…

Well, it's not hard to update the command-line git client on Unix-y systems with package management. The trouble will be with the hundreds/thousands of other programs that use Git in various ways and are essential to development workflows in various places. Github themselves, Microsoft and Jetbrains IDEs, etc.

There's two potentially mitigating factors at play here:

I suspect a lot of the tools you mentioned also already treat hashes as strings, not as 160 bit numeric types. The entire front-end JS for GitHub, for example, just uses strings. That's what I'd do if I were writing IDE integrations and such too.

Secondly, the new format will likely still be a 160-bit numeric type, just calculated using a different hash algorithm (e.g. it might be the first 160 bits of the SHA256 result). The tools you mentioned likely don't have to calculate said hashes, they just display them. The entire GitHub front-end, for instance, just displays whatever is given to it; commit hashes are input data to it, not output data.

Re: The beginning of Git supporting other hash algorithms

#50

Post by IncRnd » Sun, Mar 19, 2017, 4:37 PM UTC

rocqua wrote: post #31

amluto wrote: post #29

Earlier quoted context omitted.

> Technically MD5(128bits) and SHA1(160bits) lengths are sufficient for hashes, but they had cryptographic weaknesses Not true. A 128-bit hash gets collisions after ~2^64 tries. A big cluster can find targeted 128-bit collisions. To attack something like git, the entire attack can be done offline. The big MD5 X.509 break needed cryptanalysis to make it day I because the attack needed to happen in real time.

The threat models in which 64 bits of security (by birthday attack on 128bit hashes) is insufficient are really limited.

That is misleading, since a birthday attack is not required. The security strength of hashes is not measured by the length of the hashes.

md5 was first "broken" in 1995. As of 10 years ago, a collision attack took only a few seconds. Plus, there are a _number_ of other attacks on the hash.