Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linus on Git and SHA-1

plus.google.com

Post reply on HN
Print view View original

171–180 of 187 posts

Re: Linus on Git and SHA-1

#171

Post by Aldo_MX »

debatem1 wrote: post #44
cwbrandsma wrote: post #24

Earlier quoted context omitted.

What is the realistic worst case situation here? From what I understand, git uses SHA1 as a way of generating an id for a file, not for security. So two files might match up when they shouldn't? Is that it or is there more to this? Or is this a proverbial "since SHA1 is don't work over there, then it shouldn't be used anywhere under any circumstance" attack?

As Linus touches on in the article, SHA1 is used for signing in git. This is clearly a security function, and should not have depended on SHA1 for at least the last several years. Additionally, the defense offered against substituting binary blobs essentially comes down to "well, the kernel doesn't do that". Respectfully, other projects do. Those users' concerns were not taken seriously until lots of unrelated users…

> Additionally, the defense offered against substituting binary blobs essentially comes down to "well, the kernel doesn't do that".

Why should it matter? Git was designed for the needs of the kernel project since the very beginning[1].

It is us, mere mortals, who are at fault for picking a tool that was not designed with our needs in mind, just because that tool happened to be better suited for our needs than the alternatives it was designed to replace.

The sky isn't falling, and if you really believe so, well, there's Mercurial, although you may also feel uneasy since Mercurial also asks you to don't panic[2]; Maybe you should just build your own SCM?

[1] https://git-scm.com/book/en/v2/Getting-Started-A-Short-Histo...

[2] https://www.mercurial-scm.org/wiki/mpm/SHA1

Re: Linus on Git and SHA-1

#172

Post by joatmon-snoo »

azernik wrote: post #141
joatmon-snoo wrote: post #106

The actual mailing list discussion thread can be found here, and is infinitely more informative than any of the bull being spouted in this thread: http://public-inbox.org/git/20170226004657.zowlojdzqrrcalsm@...

My takeaway from all that - the git project is internally taking steps to mitigate vulnerabilities from this particular attack (making it harder to insert the arbitrary binary data necessary into git metadata), but a) is just throwing up their hands at the problem of projects that store binary blobs like image data in their repos, and b) is not taking this as a signal that more serious sha-1 attacks are on the horizo…

> a) is just throwing up their hands at the problem of projects that store binary blobs like image data in their repos, and b) is not taking this as a signal that more serious sha-1 attacks are on the horizon and they should speed up their hash-replacement efforts.

As @tytso points out, there is ongoing work to replace the use of SHA-1. Also, yes, SHAttered raises serious concerns about the safety of SHA-1, but that doesn't mean everyone has to immediately work on the switch (and honestly, it's questionable how much it would help - throwing more developers at a problem doesn't necessarily help). Should Stefan Baller drop the work he's doing on submodules because SHA-1 is more imminent now?

It's also worth noting Linus isn't really a core Git dev any more, he just submits patches occasionally. Junio Hamano is the primary maintainer.

Re: Linus on Git and SHA-1

#173

Post by joatmon-snoo »

azernik wrote: post #165
Chyzwar wrote: post #152

Earlier quoted context omitted.

SHA1 is only used for uniqueness. You still need to have write access to repo to perform attack. If you already have write access you do not need to make collision...

For this particular attack - what if you do not have write access, but have sufficient social capital to get a pull request merged with a benign-seeming file?

If we're just talking about this particular attack, the two files don't even resolve to the same SHA-1 in Git:

    $ sha1sum shattered*
    38762cf7f55934b34d179ae6a4c80cadccbb7f0a  shattered-1.pdf
    38762cf7f55934b34d179ae6a4c80cadccbb7f0a  shattered-2.pdf

    $ git hash-object shattered*
    ba9aaa145ccd24ef760cf31c74d8f7ca1a2e47b0
    b621eeccd5c7edac9b7dcba35a8d5afd075e24f2

Re: Linus on Git and SHA-1

#174

Post by x1798DE »

hueving wrote: post #110
x1798DE wrote: post #80

Earlier quoted context omitted.

This is presumably true also of small inputs, given that the large and small inputs are all mapping to the same space.

Well not necessarily true for small inputs. Hash a single byte for example. There is no collision in sha1 for that so you can build a 1:1 mapping of hashes back to input examples for that case. But yeah, as the input size approaches the output size, the probability of a collision existing gets to 1. The birthday paradox formula will give the probability of a collision (assumes random placement in output space) based…

> Well not necessarily true for small inputs. Hash a single byte for example. There is no collision in sha1 for that so you can build a 1:1 mapping of hashes back to input examples for that case.

I suppose you are saying that if you know that the input size is sufficiently small, you don't have to worry about collisions, which is true. I was interpreting "for small inputs" to mean that if you give a small input to a hash function (which can take inputs much larger than the space of the output), that you can still reconstruct the small inputs uniquely. Unless the hash function is deliberately designed to provide unique 1:1 mappings for small inputs, I would think that it's not true that you can uniquely reconstruct small inputs because they will likely map to the same value as a large input would (i.e. 'a' might hash to the same value as some 14821-byte string).

Re: Linus on Git and SHA-1

#175

Post by yuhong »

DannyBee wrote: post #169
ploxiln wrote: post #167

Earlier quoted context omitted.

Thus my qualification "well known". I was aware of monotone and Darcs ... but never used them, or knew anyone who did, or anyone who seriously considered it. I have no idea if Darcs is "cryptographically secure". But both of these were experimental and extremely slow at the time git was designed.

"Thus my qualification "well known"." So that, to you, means "i don't know anyone who used them"? Because that's a pretty small definition of "well known". For reference: Inside the VCS community, they were very well known. Graydon also went on to start Rust, of course. Outside of that, even to corporations I dealt with (i was working on SVN at the time), plenty knew it existed, a few evaluated it. " But both of thes…

Of note is that Monotone also used SHA-1 but it was released in 2003.

Re: Linus on Git and SHA-1

#176

Post by bascule »

Dylan16807 wrote: post #129
bascule wrote: post #114

Earlier quoted context omitted.

I am likewise perplexed why "cryptographic hash functions are unnecessary in the absence of an attacker" is such a difficult concept for you to grasp. It is not an "odd angle". It is literally the very purpose for which they were created in the first place: to defend against attacks (preimage, collision) If there are no attackers, the cryptography buys you nothing and merely makes the system slower. Again, to go back…

> I am likewise perplexed why "cryptographic hash functions are unnecessary in the absence of an attacker" is such a difficult concept for you to grasp. Because you can't seem to tell the difference between "unnecessary" and "shouldn't be done". If I build a shed then using larger screws on the door might be unnecessary but it only costs me .2% more and I know it won't fall over. Using a recent SHA function might be…

Clearly it's not, because it's both several times slower than the CRC family, and was known to have cryptographic flaws before git was even released. "The worst of both worlds" so to speak...

Re: Linus on Git and SHA-1

#177

Post by debatem1 »

Aldo_MX wrote: post #171
debatem1 wrote: post #44

Earlier quoted context omitted.

As Linus touches on in the article, SHA1 is used for signing in git. This is clearly a security function, and should not have depended on SHA1 for at least the last several years. Additionally, the defense offered against substituting binary blobs essentially comes down to "well, the kernel doesn't do that". Respectfully, other projects do. Those users' concerns were not taken seriously until lots of unrelated users…

> Additionally, the defense offered against substituting binary blobs essentially comes down to "well, the kernel doesn't do that". Why should it matter? Git was designed for the needs of the kernel project since the very beginning[1]. It is us, mere mortals, who are at fault for picking a tool that was not designed with our needs in mind, just because that tool happened to be better suited for our needs than the alt…

I'm not sure if you're being sarcastic here.

Git supports binary blobs. That's a choice that was made way back, and it is not the fault of the user for using that feature. If Linus didn't want people to use git for binary blobs he probably shouldn't have added support for it.

Once he did add support for binary blobs he had an obligation to take the security of that component and its users seriously. And as I said before, although the risk is not sky-is-falling bad, it is present. I think even Linus agrees with that now or he wouldn't be working on a path forward at all.

Re: Linus on Git and SHA-1

#178

Post by wolf550e »

runeks wrote: post #119
wolf550e wrote: post #61

Earlier quoted context omitted.

https://twitter.com/veorq/status/834872988445065218

I think the claim is interesting, and I certainly wouldn't reject it. But just relaying someone else's claim, without any substantiating argument, I find quite uninteresting.

Jean Philippe Aumasson[1] is an important currently active cryptographer who studied this particular problem closely. If he says he does not expect an algorithmic break of SHA2, I believe him that I don't need to plan to switch from SHA2 to SHA3 in an hurry.

I don't have the math to understand a more detailed explanation for his reasons to make such a statement. If you do have the math, ask him.

1 - https://131002.net/index.html

Re: Linus on Git and SHA-1

#179

Post by acqq »

joatmon-snoo wrote: post #173
azernik wrote: post #165

Earlier quoted context omitted.

For this particular attack - what if you do not have write access, but have sufficient social capital to get a pull request merged with a benign-seeming file?

If we're just talking about this particular attack, the two files don't even resolve to the same SHA-1 in Git: $ sha1sum shattered* 38762cf7f55934b34d179ae6a4c80cadccbb7f0a shattered-1.pdf 38762cf7f55934b34d179ae6a4c80cadccbb7f0a shattered-2.pdf $ git hash-object shattered* ba9aaa145ccd24ef760cf31c74d8f7ca1a2e47b0 b621eeccd5c7edac9b7dcba35a8d5afd075e24f2

As soon as somebody invests a few $100k there are going to be such files. Now it's known how much it takes. That much.

Luckily, there is also a known solution to detect that kind of files.

Re: Linus on Git and SHA-1

#180

Post by ploxiln »

DannyBee wrote: post #169
ploxiln wrote: post #167

Earlier quoted context omitted.

Thus my qualification "well known". I was aware of monotone and Darcs ... but never used them, or knew anyone who did, or anyone who seriously considered it. I have no idea if Darcs is "cryptographically secure". But both of these were experimental and extremely slow at the time git was designed.

"Thus my qualification "well known"." So that, to you, means "i don't know anyone who used them"? Because that's a pretty small definition of "well known". For reference: Inside the VCS community, they were very well known. Graydon also went on to start Rust, of course. Outside of that, even to corporations I dealt with (i was working on SVN at the time), plenty knew it existed, a few evaluated it. " But both of thes…

I didn't claim that Linus came up with the design from scratch, nor that git was less experimental than monotone in 2005. Just that no one used monotone, so git was not a reduction in safety/assurance compared to any in-use SCM. And as mentioned, it's goal was speed, not cryptographic assurance.

I also was surprised that git became so popular, I thought mercurial would win for "ease of use". But I've never been an advocate or influencer of popularity. I think that's mostly due to GitHub, which got it to a market-share tipping point.

Post reply on HN