Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Malicious software libraries found in PyPI posing as well known libraries

nbu.gov.sk

Post reply on HN
Print view View original

51–60 of 254 posts

Re: Malicious software libraries found in PyPI posing as well known libraries

#51

Post by simon3 »

cdnsteve wrote: post #42
raesene6 wrote: post #26

Earlier quoted context omitted.

Package managers are providing (in the most case) a free service, so it's hard to see a strong case for them providing more services here. The problem is one of scale. npm has over 500,000 packages, so no manual review will address their scale over the whole repository. Until the developer market shows that they'll pay for a more secure service (e.g. package signed, reviews done etc) I doubt much will change.

If there was a business/enterprise offering with extra security I'm sure they'd have a long list of people who would sign up and happily pay for it.

The conda package manager is free and generally feels like a professional package manager like yum or apt-get.

Re: Malicious software libraries found in PyPI posing as well known libraries

#52

Post by cdnsteve »

I'm all for security but this hit a nerve with me: "Success of the attack relies on negligence of the developer, or system administrator, who does not check the name of the package thoroughly."

Package managers need to do more. If they had an enterprise version that you could subscribe to monthly/annually invoice that you would get enterprises onboard, they are concerned about security and will pay. Developers like us will help encourage it. I'd rather not see some third-party "secure" package managers but make them part of PyPi and send funding to the Python foundation. They are seeking donations but that doesn't work well with businesses. Make it a monthly/yearly service.

Re: Malicious software libraries found in PyPI posing as well known libraries

#53

Post by kasabali »

Yet another attack vector that doesn't exist at all in Linux distributions but invented by language package managers, sadly.

They solved the issue 2 decades ago by heavily vetting packages before accepting them into repositories. Users are allowed to add and use packages from 3rd party repositories.

Maybe solution to this is creating curated repositories based on publicly open ones and using them by default (and requiring opt-in for using other repositories). Conda for Python and Stackage for Haskell seems like relevant solutions.

Re: Malicious software libraries found in PyPI posing as well known libraries

#55

Post by avsm »

chatmasta wrote: post #12

Package managers seem to be an increasingly popular attack vector. It's only luck that none of the attacks have been particularly malicious yet. Considering how many package manager downloads go to a server in a datacenter, a widely distributed malicious package could control a botnet with extremely high throughput, or wreak havoc on any databases it comes into contact with. It's only a matter of time before somethin…

This is why we are working on integrating TUF (The Update Framework) signing into the OCaml OPAM package manager. See https://github.com/hannesm/conex-paper/blob/master/paper.pdf for the talk from last year. There's one more iteration required on the implementation before we're happy with it, but we are aiming to get this live on the OPAMv2 package repository some time in 2018 for all the publicly available OCaml packages.

OPAMv2 also exposes sufficient hooks during the build process for using OS sandboxing during builds, and disconnecting network access/etc. It would be nice to factor this out to be more OS independent (e.g. for all the `unshare` tricks on Linux, or the sexp-format for sandboxing on OSX) in the future.

Re: Malicious software libraries found in PyPI posing as well known libraries

#56

Post by kumarvvr »

Whoa urlib & urllib3. Those are pretty popular packages, especially to newbies. Hundreds of websites that teach web-scraping use those libraries.

Wonder what is an effective form of protection against such attack vectors?

Do digitally signed certificates fit into this usage scenario??

Re: Malicious software libraries found in PyPI posing as well known libraries

#57

Post by raesene6 »

mbrock wrote: post #49
raesene6 wrote: post #38

Earlier quoted context omitted.

Credibility is an easier check but still tricky. Many of the package are uploaded by anonymous or pseudonymous authors, so there's no easy way to even tie that to an IRL identity, let alone check for credibility. I'd agree that a curated small package repository would be a better way to address the problem, but the market doesn't seem very interested in that as a solution.

I think it might just not have happened yet. The npm community can be pretty creative and enthusiastic! I don't think IRL identities are necessary for what I imagine. It's more like establishing a basic set of packages that have been around, have communities of committers, reverse dependencies, etc. Maybe we would even make a starting assumption that the transitive closure of dependencies originating with a set of hi…

So, apologies for being a bit cynical here, but I don't see this one being addressed any time soon.

it's been at least 5 years since npm started getting scrutiny relating to security weaknesses https://blog.andyet.com/2012/03/08/compromising-the-integrit...

and 4 years since Rubygems was compromised http://blog.rubygems.org/2013/01/31/data-verification.html

and yet, I don't see substantial movements relating to package security and trustability in these repo's. To be clear I'm not suggesting these two are any worse than others, they're just large repo's who have had incidents in the past.

The problem here (to my view) is that increasing the security of package repo's will slow down releases (additional checks take time) and cost money (additional security, hosting etc) and until there's a market demand for those service, they won't happen.

Re: Malicious software libraries found in PyPI posing as well known libraries

#58

Post by EGreg »

This is why I am not a huge fan of using package managers. I like to understand the code we put into our platform, and vet it. And not have it change under us automatically, after that, but review the changes manually before accepting it.

I felt a bit curmudgeonly but we have a responsibility at https://qbix.com/platform for all our apps being secure. I wanted to use repos for each package and manually git pull or hg pull them when they changed.

I was finally convinced by our developers to just use package managers with version pinning. Honestly it's really hard to avoid package managers, especially for all the newer functionality such as Payment Requests or Web Push. Luckily there is version pinning.

We want our clients to feel secure that we vetted ALL the code that went into the platform. So our package json (and composer.json) uses version pinning. We'd rather take a bug report and manually fix it than NO bug report and have a SHTF moment.

Re: Malicious software libraries found in PyPI posing as well known libraries

#59

Post by mikehollinger »

kasabali wrote: post #53

Yet another attack vector that doesn't exist at all in Linux distributions but invented by language package managers, sadly. They solved the issue 2 decades ago by heavily vetting packages before accepting them into repositories. Users are allowed to add and use packages from 3rd party repositories. Maybe solution to this is creating curated repositories based on publicly open ones and using them by default (and requ…

There's a certain amount of work (and therefore money) required to do this. That incremental difference is small for a well designed application, but - someone must actually vet and curate the contents of the repo. That tends to slow down execution, leading to scenarios where things like docker a year or two ago from the canonical "trusty" repo were hopelessly behind the "real" docker since docker was evolving so quickly and trusty was by design slowing down.

Each commit that went into trusty required a team to submit and a team to approve. That costs money. ;-)

Post reply on HN