Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dependency Confusion: How I Hacked Into Apple, Microsoft and Other Companies

medium.com

Post reply on HN
Print view View original

131–140 of 412 posts

Re: Dependency Confusion: How I Hacked Into Apple, Microsoft and Other Companies

#131

Post by golergka »

TeMPOraL wrote: post #126
ehnto wrote: post #117

Pulling packages down at build time seems ludicrous to me, I can understand it in a development environment, but I don't understand how "Pull packages from the public internet and put them into our production codebase" past any kind of robustness scrutiny. I guess it's a case of the ease of use proving too great, so convenient in fact that we just kind of swept the implications under the rug.

> I can understand it in a development environment I can't. It's incredibly wasteful time and resource-wise, and ties your development process to third-party providers (and your ISP), which fall over often enough in practice. It's a good practice to have a local cache of all the third-party dependencies you use, available to both developers and CI infrastructure.

> local cache

For a distributed company with developers from all over the globe the "local" here doesn't really make much sense. But from my experience with NPM, you download packages on your developer machine once you set up a project, and then only when something really messes up node_modules, which happens once in three months, on average.

You do re-download packages for every build in CI pipeline as you build a docker image from scratch though, and that's when NPM mirror is usually set up.

Re: Dependency Confusion: How I Hacked Into Apple, Microsoft and Other Companies

#132

Post by CaptArmchair »

syshum wrote: post #118
CaptArmchair wrote: post #31

Earlier quoted context omitted.

There are a lot of expensive things you can outsource. Responsibility isn't among those. Free software / open source propels engineering as you can share and leverage the results of collective efforts. However, at no point did the concept come with inherent guarantees about concerns such as security. esr defined 19 points for "good" open source software development in his seminal essay "The Cathedral and the Bazaar".…

>>There are a lot of expensive things you can outsource. Responsibility isn't among those. That is not true at all, the industry both Development and even more so in Operations has been outsourcing responsibility for a long time, they is why we have support contracts, SLA's and other very expensive services we pay many many times more than the cost of hardware for... To outsource responsibility... Network down -- Cal…

I disagree.

A support contract allows you to hold a sub-contractor accountable. But that's the extent of what an SLA does. What it doesn't do is diminish your responsibility towards anyone who relies on the services you provide yourself. These are distinct things.

Put more succinctly, if the network's down: that's still very much your problem. Especially if you over-promised 100% availability to the end users of your services. Your end users do not care about Cisco, EMC or Nimble. They don't have contracts with any of those. They have a contract with you and they can and will hold you accountable if you don't deliver on what you sold them.

Re: Dependency Confusion: How I Hacked Into Apple, Microsoft and Other Companies

#133

Post by 076ae80a-3c97-4 »

golergka wrote: post #131
TeMPOraL wrote: post #126

Earlier quoted context omitted.

> I can understand it in a development environment I can't. It's incredibly wasteful time and resource-wise, and ties your development process to third-party providers (and your ISP), which fall over often enough in practice. It's a good practice to have a local cache of all the third-party dependencies you use, available to both developers and CI infrastructure.

> local cache For a distributed company with developers from all over the globe the "local" here doesn't really make much sense. But from my experience with NPM, you download packages on your developer machine once you set up a project, and then only when something really messes up node_modules, which happens once in three months, on average. You do re-download packages for every build in CI pipeline as you build a d…

You're lucky. I feel like I have to delete node_modules at least once a week.

Re: Dependency Confusion: How I Hacked Into Apple, Microsoft and Other Companies

#134

Post by prepend »

keyle wrote: post #10

I'm flabbergasted by how silly this is. Bump the version and the package manager chooses yours online vs. the private one. Amazing. How silly and how expensive is this going to be as this blatant security issue is going ripple on for the next months to come.

This is why explicit pins are a good idea. Whenever you finish a project you should set the explicit versions in the lock and then tag it. The problem is with dependencies of your dependencies, but if they are public, then by their nature they won’t be using private packages that can be hijacked.

Re: Dependency Confusion: How I Hacked Into Apple, Microsoft and Other Companies

#135

Post by angry_octet »

I've been wishing npm/pypi/apt etc would improve for ages, but it seems like infrastructure improves one disaster at a time, software one hack at a time. I'm only annoyed I didn't do it myself.

The pypi maintainer is being ridiculous, it is much better to have this guy poke MSFT than have the Russians do it, he's doing them a favour.

Re: Dependency Confusion: How I Hacked Into Apple, Microsoft and Other Companies

#136

Post by konaraddi »

For npm enterprise, it looks like setting the scope (e.g. @acmecorp/internal-pkg) would mitigate the public and private confusion. For Verdaccio, an open source light weight npm registry, it first checks if a private package is available before searching the public npm registry (however, their best practices say to use a prefix for private packages https://verdaccio.org/docs/en/best )

Re: Dependency Confusion: How I Hacked Into Apple, Microsoft and Other Companies

#137

Post by alistairSH »

uncledave wrote: post #119
ehnto wrote: post #117

Pulling packages down at build time seems ludicrous to me, I can understand it in a development environment, but I don't understand how "Pull packages from the public internet and put them into our production codebase" past any kind of robustness scrutiny. I guess it's a case of the ease of use proving too great, so convenient in fact that we just kind of swept the implications under the rug.

Yes. The way we develop software quite frankly scares the shit out of me on a daily basis.

Even worse - if you consider yourself average, half the software out there was built on top of even weaker foundations.

The web really is held together by duct tape and bubble gum.

Re: Dependency Confusion: How I Hacked Into Apple, Microsoft and Other Companies

#138

Post by anchochilis »

golergka wrote: post #130
ehnto wrote: post #117

Pulling packages down at build time seems ludicrous to me, I can understand it in a development environment, but I don't understand how "Pull packages from the public internet and put them into our production codebase" past any kind of robustness scrutiny. I guess it's a case of the ease of use proving too great, so convenient in fact that we just kind of swept the implications under the rug.

Migrating from public NPM to a privately-hosted, your own mirror of NPM is not a very complicated process, and if you already have a CI pipeline in place, it can be implemented completely transparently to developers. But as many other things that an organisation has to change as it grows from a single-founder startup to a real company, it's something many people just forget to do until they face the consequences.

Mirrors are great for speed and protecting you from dependencies getting randomly deleted off the public repo, but I don't think they can protect from malicious packages. They'll just get pulled into the mirror.

At my last gig (Java), developers reviewed all third-party libraries + dependencies and manually uploaded them to a private Ivy server. I don't think that could work in the Node ecosystem, where every module seems to have 100+ dependencies.

EDIT:

There's a real security vs accessibility trade-off here. You can't be a productive web developer, according to modern standards, and review every single transitive dependency that gets pulled into your application. And it's very inefficient to have individual developers at different orgs separately reviewing the same libraries over and over again.

One would naturally turn to repository administrators to enforce stricter security standards. Maybe RubyGems could review all source code for every new version of a package and build it themselves instead of accepting uploads of pre-built artifacts. But these repositories are run by smallish groups of volunteers, and they don't have the resources to conduct those kinds of reviews. And no open-source developer wants to have to go through an App Store-like review process to upload their silly McWidget library.

Re: Dependency Confusion: How I Hacked Into Apple, Microsoft and Other Companies

#139

Post by Philip-J-Fry »

Aissen wrote: post #58
kasperni wrote: post #32

I'm surprised the reverse fully-qualified domain name (FQDN) model used by Java isn't more widely adopted. If you want to upload artifacts to the main repository (Maven Central) you first need to show ownership of a particular domain. For example, via a DNS TXT record (example [1]). Would make these kind of attacks a lot more difficult. [1] https://issues.sonatype.org/browse/OSSRH-61509

Or the URL-based model used by Go.

Using a URL isn't what makes Go's dependency management that good. It's just convenience that the import is a URL.

The key thing with Go is that all dependencies have a checksum (go.sum file) and that should be committed to the repo.

So even if the domain gets hijacked and a malicious package is served up, then the checksum will fail and it will refuse to build.

People should be using internal module proxies anyway for Go. You can just store the module files in a directory, a git repo or a web service and serve up an internal cache.

Re: Dependency Confusion: How I Hacked Into Apple, Microsoft and Other Companies

#140

Post by sp332 »

xurukefi wrote: post #51

I never understood why these package repositories don't include some (opt-in?) integrity checking option using digital signatures. If I download code that executes on my machine there should be at least the option to establish some level of trust. We have been doing that with linux distro package managers for decades. Seems like common sense to me.

The package integrity would be fine in this case. The packages downloaded from PyPI would be legitimately signed by PyPI, and the internal packages would be signed by the local package server. The issue is not knowing which source to use for each package, and you'd have the same issue with not knowing which certificate to use to check them.
Post reply on HN