Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linus' reply on Git and SHA-1 collision

marc.info

Post reply on HN
Print view View original

251–260 of 273 posts

Re: Linus' reply on Git and SHA-1 collision

#251

Post by sqeaky »

danmaz74 wrote: post #212
orblivion wrote: post #106

Earlier quoted context omitted.

I control a fleet of servers. I have a saltstack or ansible script. One of the steps in provisioning a new server is to pull library X from github.com. One day Egor Homakov finds a new hack and finds his way into access to the master branch for library X. As a prank, he force pushes a change to master. Being aware of such a possibility, instead of setting up my script to pull from master or even a specific tag, perha…

> Tomorrow, when preimage SHA-1 attacks are cheap, it will no longer save me. Well, Moore's law is petering out isn't it? :)

Depending on your interpretation. If you go by the relatively useless transistor/fixed price cpu then it is dead, but if you go by more practical measures like instructions per second in whole system/dollar the it is as fast as ever. GPUs are really fast and still advancing.

Re: Linus' reply on Git and SHA-1 collision

#252

Post by sqeaky »

dozzie wrote: post #202
amingilani wrote: post #183

Earlier quoted context omitted.

Almost all of them: Npm: https://docs.npmjs.com/cli/install Bundler: https://bundler.io/git.html Homebrew: https://github.com/Homebrew/brew

And which of these are "package systems supplied by your operating system"?

While I agree that the OS package should be used first and foremost, it just often doesn't have the required software. (Even after adding extra repos that it might support)

You are picking on the useless details. Those are all commonly used package managers in production environments. They often provide software that simply never gets packaged with the OS. They likely always will, because they have more focused design goals.

A better way to argue this would be point out that specific ways to use the package managers better. For example: bundlr supports saving all the required packages offline. This provides the opportunity to do a security review and save the packages locally/internally rather than always trusting the whatever is on the Internet.

Re: Linus' reply on Git and SHA-1 collision

#253

Post by sqeaky »

disiplus wrote: post #155
fryguy wrote: post #75

Earlier quoted context omitted.

That's not entirely true. You could fork a popular git repo, and then make some kind of patch for a bug in some seldomly changed file. Then force a collision in the new file with the benign change as well as your poisoned version. Then they could convince you to pull in the changes. Then they could reset their repository to the one with the poisoned version and anyone who pulls from them first would get the poisoned…

thats stretching it. if you could convince anybody to pull from you then why even bother to go to a great expense of creating a collision.

It creates plausible dependability.

Imagine the NSA publishing a crypto algorithm and contributes it to openSSL or some hypothetical crypto library using git. If they commit their new algorithm, everyone will be looking at that. They could do something devious like tinker with the way random numbers are generated elsewhere and reduce the possible keyspace of another algorithm to something very small and easy to brute force.

When this keyspace shortening is found out it would be hard or impossible to track back. No amount of inspecting the files that reportedly changed would reveal that the NSA did this.

Re: Linus' reply on Git and SHA-1 collision

#254

Post by dozzie »

sqeaky wrote: post #252
dozzie wrote: post #202

Earlier quoted context omitted.

And which of these are "package systems supplied by your operating system"?

While I agree that the OS package should be used first and foremost, it just often doesn't have the required software. (Even after adding extra repos that it might support) You are picking on the useless details. Those are all commonly used package managers in production environments. They often provide software that simply never gets packaged with the OS. They likely always will, because they have more focused desig…

> While I agree that the OS package should be used first and foremost, it just often doesn't have the required software. (Even after adding extra repos that it might support)

OS-supplied packages don't grow on magical trees. If you don't have the necessary software in official repositories (or if it's your software), you can package it yourself. Deployment then becomes a breeze, and you save yourself otherwise completely useless process of recompiling things over and over again.

> You are picking on the useless details.

Quite the contrary. Those details make important difference.

> Those are all commonly used package managers in production environments. They often provide software that simply never gets packaged with the OS.

Apart from Homebrew, which is for workstations (hardly anybody runs macOS servers), none of these "package managers used in production environments" provide you a complete way to rebuild your software. You can be fine for a while if you stay away from modules that are interfaces to C or C++ libraries and from tools from other languages (e.g. I have used Python's Sphinx to document Erlang daemons quite successfully), but once you hit that, deployment starts to be PITA, because you'll need to remember to install all the required libraries, -dev packages, compilers, and what not.

On the other hand, DEB or RPM with artifacts will just automatically pull the required libraries, and its build dependencies give a dedicated and standard place for the necessary build tools.

Your comment supports my opinion that today's programmers usually don't want to be bothered with learning things that have been working for sysadmins for twenty years already.

Re: Linus' reply on Git and SHA-1 collision

#255

Post by Xylakant »

dozzie wrote: post #254
sqeaky wrote: post #252

Earlier quoted context omitted.

While I agree that the OS package should be used first and foremost, it just often doesn't have the required software. (Even after adding extra repos that it might support) You are picking on the useless details. Those are all commonly used package managers in production environments. They often provide software that simply never gets packaged with the OS. They likely always will, because they have more focused desig…

> While I agree that the OS package should be used first and foremost, it just often doesn't have the required software. (Even after adding extra repos that it might support) OS-supplied packages don't grow on magical trees. If you don't have the necessary software in official repositories (or if it's your software), you can package it yourself. Deployment then becomes a breeze, and you save yourself otherwise comple…

Gentoos emerge and FreeBSD port are package managers that preferably build from source. At least emerge also accepts git sources, with tags and commit specifiers. I haven't used bsd in a while on a prod system, but I'd not be surprised if port gained the same functionality - after all whether you're pulling a tarball and use a checksum for integrity checking or hand the task off to git makes exactly no difference at all.

And I have yet to hear that port and emerge are incapable of doing dependency resolution. They're battle tested systems that work well in production environments.

Re: Linus' reply on Git and SHA-1 collision

#256

Post by dozzie »

Xylakant wrote: post #255
dozzie wrote: post #254

Earlier quoted context omitted.

> While I agree that the OS package should be used first and foremost, it just often doesn't have the required software. (Even after adding extra repos that it might support) OS-supplied packages don't grow on magical trees. If you don't have the necessary software in official repositories (or if it's your software), you can package it yourself. Deployment then becomes a breeze, and you save yourself otherwise comple…

Gentoos emerge and FreeBSD port are package managers that preferably build from source. At least emerge also accepts git sources, with tags and commit specifiers. I haven't used bsd in a while on a prod system, but I'd not be surprised if port gained the same functionality - after all whether you're pulling a tarball and use a checksum for integrity checking or hand the task off to git makes exactly no difference at…

First, portage and ports are OS-supplied mechanism for installing software. They are nothing like pip or gems or npm, which only can install things written in their respective languages of choice and fail miserably for modules touching any library external to them (unless you manually ensure the library's and compiler toolchain's presence, that is).

Second, ports and portage have support for and networks of mirror servers that keep copies of software available through these packaging systems. It's trivial to switch if one of the mirrors goes down. For pip, gems, or npm you need to plan ahead for the problems and deploy your own package cache, from what I know.

Third, I was using Gentoo with one of these "battle tested systems that work well in production" for several years. It was doable, but it wasn't pretty, could lead to breaking software after updating some random deep dependency (if it was recompiled with different flags), and generally required more work and attention than APT would, all that for very little gain (if any gain at all). Oh, and it ended up working with binary packages, after all, I just needed to compile them myself instead of having a half an hour downtime of production MySQL because it needed to get compiled (which could fail, leaving me with no working database installed).

Re: Linus' reply on Git and SHA-1 collision

#257

Post by budu3 »

Can someone with more expertise shed mpre light on this? What does he mean? What kind of things are they hiding in the Git commit. If git is opensource then how can they hide anything?

> Git has opaque data in some places (we hide things in commit objects intentionally...

Re: Linus' reply on Git and SHA-1 collision

#258

Post by andrewshadura »

Kubuxu wrote: post #204
almog wrote: post #4

Do you know if git objects' size header was designed to deal with a possible collision or does it serve another purpose as well? Just some context - git calculates an object's name by his content in the following way. Say we have a blob that represent a file who's content is 'Here be dragons', then the file name would be: printf "blob 17\0Here be dragons\!\n" | openssl sha1 # => a54eff8e0fa05c40cca0ab3851be5aa8058f20…

Also git compresses before hashing IIRC.

It doesn't.

Re: Linus' reply on Git and SHA-1 collision

#259

Post by jefurii »

Perseids wrote: post #182
notfed wrote: post #56

Pertinent facts for the worried: 1) Git doesn't rely on SHA-1 for security. It relies on HTTPS, and a web of trust. 2) Even if git did rely on SHA-1, there's no imminent threat. What happened today was a SHA-1 collision, not a preimage attack. If a collision costs 2^n, a preimage attack costs 2^(2n). 3) Even if someone managed to pull off a preimage attack, creating a "poisonous" version of one your git repository's…

I don't like the living-on-the-edge-attitude that Linus and others here promote regarding Sha1 in git. First, attacks only get faster over time. What costs millions today is likely to be achievable on commodity hardware in the coming years. Second, attacks only get more flexible over time. A contrived collision on MD5 in 2004 got perfected to a single block collision in 2010 [1]. Third, devising an update strategy an…

> Fourth, people use git in creative ways. Linus may think it is a cardinal sin to commit binary blobs in a git repository, but I can't imagine I'm the only one using git as a poor man's backup and file sharing solution.

git-annex (written by Joey Hess from the email) is a way to manage binary blobs using Git, but IIRC it uses SHA256.

Re: Linus' reply on Git and SHA-1 collision

#260

Post by sneak »

kyrias wrote: post #213
throwaway7767 wrote: post #190

Earlier quoted context omitted.

> 1) Git doesn't rely on SHA-1 for security. It relies on HTTPS, and a web of trust. Some security-focused developers sign git tags and/or commits, specifically to have things verifiable end-to-end and not having to trust HTTPS and all the middle men that entails. Would that not be a case where git relies on SHA1 for security? Someone could replace a tag or commit with a malicious version that verifies fine since it…

Regarding tags, https://github.com/cgwalters/git-evtag seems like a really interesting idea.

or, you know, this could just be fixed in git by using a proper hash function that allows you to trust your merkle tree.
Post reply on HN