Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The beginning of Git supporting other hash algorithms

github.com

Post reply on HN
Print view View original

21–30 of 128 posts

Re: The beginning of Git supporting other hash algorithms

#21

Post by d0m »

ebbv wrote: post #4
pyed wrote: post #3

Earlier quoted context omitted.

Probably newer versions will commit only using a new hash algorithm, while completely able to deal with the old one

Can it really be that simple though? If you are using a newer version of Git on your repo which is committing only with the newer hash and I try to clone your repo with an older version I will be unable to do so. I guess maybe that's acceptable though?

One straightforward way is to use the new sha only for new Git repos. Old repo could be migrated but it would require "re-commiting" everything to the new repo.

Re: The beginning of Git supporting other hash algorithms

#22

Post by dahart »

zoren wrote: post #10

Someone please remind me why the hash is not a type definition so the representation would only have to be changed in one place.

That's exactly what this change is. You mean why wasn't it that way before the change? Maybe because it wasn't ever needed before? Git's been good with only sha-1 for 12 years. Think about the flip side of your question... what were the alternatives 12 years ago, or 5 years ago? And why would someone write code for alternatives that aren't expected to be used and maybe don't exist?

In my experience, generalizing ahead of need more often than not causes problems, and I've watched over-engineering result in far more effort to fix when the need it was anticipating does arrive than just waiting until the need is there.

Re: The beginning of Git supporting other hash algorithms

#23

Post by bk2204 »

pwdisswordfish wrote: post #6

struct object_id was introduced in this commit, in 2015: https://git.kernel.org/pub/scm/git/git.git/commit/?id=5f7817... So this change doesn't do much for now. Good to see, though.

Yes, this is correct. The struct object_id changes don't actually change the hash. What they do, however, is allow us to remove a lot of the hard-coded instances of 20 and 40 (SHA-1 length in bytes and hex, respectively) in the codebase.

The remaining instances of those values become constants or variables (which I'm also doing as part of the series), and it then becomes much easier to add a new hash function, since we've enumerated all the places we need to update (and can do so with a simple sed one-liner).

The biggest impediment to adding a new hash function has been dealing with the hard-coded constants everywhere.

Re: The beginning of Git supporting other hash algorithms

#24

Post by ProblemFactory »

ebbv wrote: post #4
pyed wrote: post #3

Earlier quoted context omitted.

Probably newer versions will commit only using a new hash algorithm, while completely able to deal with the old one

Can it really be that simple though? If you are using a newer version of Git on your repo which is committing only with the newer hash and I try to clone your repo with an older version I will be unable to do so. I guess maybe that's acceptable though?

Yes, that is sort of Linus's plan: http://marc.info/?l=git&m=148798319024294&w=2

> You want to have a model that basically reads old data, but that very aggressively approaches "new data only" in order to avoid the situation where you have basically the exact same tree state, just _represented_ differently.

> That way everything "converges" towards the new format: the only way you can stay on the old format is if you only have old-format objects, and once you have a new-format object all your objects are going to be new format - except for the history.

As soon as there is one new-hash commit in a repo, all users of it will have to upgrade their git client - and that git client will (probably?) default to writing new-hash commits.

Re: The beginning of Git supporting other hash algorithms

#26

Post by chmod775 »

kozak wrote: post #25

Do they anticipate that one day we'll have to move from SHA256 to something else again? It's only matter of time. Hash function have lifecycle. Tre transition has to be done in a way that will also make the next transition more straightforward.

Reading even one changed line tells us that they replaced hardcoded char arrays for SHA1 with a generic struct that could be used as a container for any hash.

Some functions that previously operated on those char arrays have been changed to deal with the more generic struct instead.

Re: The beginning of Git supporting other hash algorithms

#27

Post by anilgulecha »

kozak wrote: post #25

Do they anticipate that one day we'll have to move from SHA256 to something else again? It's only matter of time. Hash function have lifecycle. Tre transition has to be done in a way that will also make the next transition more straightforward.

A note on 'lifecycle': that's not how it works -- the age of use/lifecycle is not a function of the bit-length in hash, or inevitability of the current standard being broken.

Technically MD5(128bits) and SHA1(160bits) lengths are sufficient for hashes, but they had cryptographic weaknesses -- the functions had cryptanalytic attacks, which reduced bruteforce from the complete keyspace to something of a much smaller magnitude. These weaknesses are what has lead to the deprecation of MD5 and SHA1.

It is definitely possible that new crypt-analytic attacks could be shown on SHA256/512, but none have so far been publicly provided. Hence the confidence in them.

Re: The beginning of Git supporting other hash algorithms

#28

Post by bk2204 »

sorenbs wrote: post #17
bk2204 wrote: post #9

I'm the person who's been working on this conversion for some time. This series of commits is actually the sixth, and there will be several more coming. (I just posted the seventh to the list, and I have two more mostly complete.) The current transition plan is being discussed here: https://public-inbox.org/git/CA+dhYEViN4-boZLN+5QJyE7RtX+q6a...

Out of curiosity: when did you start to take the first serious steps in this direction?

From the commit history, 2015 (commit 5f7817c85d4b5f65626c8f49249a6c91292b8513).

I proposed the idea of improved compile-time checking and maintainability, as there wasn't originally much interest in a new hash function, but the maintainability improvements were something people could go for.

I hadn't spent as much time working on it as I am now, so it moved slowly. Other people also helped by converting parts of the code that they were working on (like parts of the refs subsystem).

Re: The beginning of Git supporting other hash algorithms

#29

Post by amluto »

anilgulecha wrote: post #27
kozak wrote: post #25

Do they anticipate that one day we'll have to move from SHA256 to something else again? It's only matter of time. Hash function have lifecycle. Tre transition has to be done in a way that will also make the next transition more straightforward.

A note on 'lifecycle': that's not how it works -- the age of use/lifecycle is not a function of the bit-length in hash, or inevitability of the current standard being broken. Technically MD5(128bits) and SHA1(160bits) lengths are sufficient for hashes, but they had cryptographic weaknesses -- the functions had cryptanalytic attacks, which reduced bruteforce from the complete keyspace to something of a much smaller ma…

> Technically MD5(128bits) and SHA1(160bits) lengths are sufficient for hashes, but they had cryptographic weaknesses

Not true. A 128-bit hash gets collisions after ~2^64 tries. A big cluster can find targeted 128-bit collisions. To attack something like git, the entire attack can be done offline.

The big MD5 X.509 break needed cryptanalysis to make it day I because the attack needed to happen in real time.

Re: The beginning of Git supporting other hash algorithms

#30

Post by nuntius »

jlgaddis wrote: post #15
nuntius wrote: post #13

Earlier quoted context omitted.

Backwards compat requires that both old and new hashes work at the same time. A simple typedef is unlikely to handle all the semantics and space needed for such a change... It is often hard to generalize when N=1. Now that the N=1 use case is established and we are moving towards N=2, it is painfully obvious to all that a better abstraction is needed. Typedef or no, we would still need a full audit of the code to fin…

Perhaps you haven't read Linus' comments where he stated (more than a decade ago) that the usage of SHA1 here isn't for "security"? (Hint: that's why GPG signing commits is an option.)

I read those comments more than a decade ago. They seemed weak but tolerable then. They seem broken now. Git is supposed to guarantee that the code I see is the code the author saw, in a distributed and decentralized environment. This is Git's entire reason for existing.

A secure design is essential for trusting this functionality. My trust in Git has always been tempered by the weakness of SHA1.

A GPG signature is no stronger than its object ref.

Have you seen how many frameworks believe "auto-pull and compile deps by hash from github" is reasonable? They are assuming this isn't a massive attack vector. They are trying to build on a core feature that Git claims to have.

Recent events moved this from probably foolish to provably so.

Post reply on HN