Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Malicious software libraries found in PyPI posing as well known libraries

nbu.gov.sk

Post reply on HN
Print view View original

91–100 of 254 posts

Re: Malicious software libraries found in PyPI posing as well known libraries

#91

Post by zzzcpan »

ex_amazon_sde wrote: post #73
zzzcpan wrote: post #65

Earlier quoted context omitted.

I don't think there are any maintainers that verify upstream code, they only manage packages and updates. Which is actually safer to do without maintainers, completely automatically, as it will eliminate a huge attack surface introduced by a maintainer.

Debian Developers are responsible of the package quality. Most package only well-known software that they are familiar with or read the code. Some do more thorough security audits. Some security-sensitive packages are maintained by teams to share the workload.

They do a lot that could be and sometimes even is automated, but that doesn't help with security, only weakens it. Which is my point. It's better to automate package generation and trust fewer people, mainly the authors, not introduce maintainers into it.

Re: Malicious software libraries found in PyPI posing as well known libraries

#92

Post by Jach »

mbrock wrote: post #62
raesene6 wrote: post #57

Earlier quoted context omitted.

So, apologies for being a bit cynical here, but I don't see this one being addressed any time soon. it's been at least 5 years since npm started getting scrutiny relating to security weaknesses https://blog.andyet.com/2012/03/08/compromising-the-integrit... and 4 years since Rubygems was compromised http://blog.rubygems.org/2013/01/31/data-verification.html and yet, I don't see substantial movements relating to packa…

I'm skeptical too. But if I think like a sci-fi writer I can vaguely imagine ways for it to actually happen. That open source maintenance happens at all is pretty remarkable, so I think this thing, with an appropriate concept and some good tools (with emojis in their command line output), is at least vaguely plausible...

I don't see any reason it can't be done, either, in theory, and that's with the manual approach. Fancier ideas are viable too, but 500k is still relatively tiny and a manually tractable number. Incremental reviews starting today by many coordinating groups in the node community would take a while to complete, maybe a few years, but with some sensible ordering heuristics like e.g. the most downloaded first, or the most suspicious names first, some value could be produced quickly. But it won't happen, package vetting isn't really a value in these communities. (And that might not really be a bad thing, at least for now...)

Re: Malicious software libraries found in PyPI posing as well known libraries

#93

Post by mschuster91 »

raesene6 wrote: post #24
IgorPartola wrote: post #15

This to me is the nightmare scenario. Well one of the two, the other one being that a developer of an obscure library I use has their password to PyPI compromised and a bad actor uploads a backdoored version of the library. Fundamentally, the reason this is different from how thinks like Linux distos work is because Linux distros have maintainers who are in charge of making sure every new update to one of their packa…

A review system unfortunately isn't likely to be practicable with current development models. npm alone has over 500,000 packages ( http://www.modulecounts.com/ ) so even a one time review isn't going to happen. If people want a more trusted solution the likely outcome is that they'll need to use a smaller more static set of libraries and then either do the audits themselves, or outsource that to a 3rd party. Ofc wit…

> npm alone has over 500,000 packages (http://www.modulecounts.com/) so even a one time review isn't going to happen.

But at least the modules with the most downloads (webpack, react, or stuff like left-pad) could be vetted, and especially npm could implement a 2-or-more person model - basically, everyone with publish access can upload a new artifact, but to actually have it distributed to endusers, a second person would be required to sign off.

Re: Malicious software libraries found in PyPI posing as well known libraries

#94

Post by jessaustin »

raesene6 wrote: post #48
cdnsteve wrote: post #43

Earlier quoted context omitted.

I'm sure companies would pay for it. The service needs to be part of the main package service, not some third party.

Interesting if you think that npm/Rubygems/PyPI are leaving a load of money on the table, why do you think they haven't introduced those services so far...

ISTM we're just talking about running an alternate, more restrictive registry? npm etc. don't have to play any part in that. This service could be offered by anyone: IBM could do it.

Re: Malicious software libraries found in PyPI posing as well known libraries

#95

Post by raesene6 »

mschuster91 wrote: post #93
raesene6 wrote: post #24

Earlier quoted context omitted.

A review system unfortunately isn't likely to be practicable with current development models. npm alone has over 500,000 packages ( http://www.modulecounts.com/ ) so even a one time review isn't going to happen. If people want a more trusted solution the likely outcome is that they'll need to use a smaller more static set of libraries and then either do the audits themselves, or outsource that to a 3rd party. Ofc wit…

> npm alone has over 500,000 packages ( http://www.modulecounts.com/ ) so even a one time review isn't going to happen. But at least the modules with the most downloads (webpack, react, or stuff like left-pad) could be vetted, and especially npm could implement a 2-or-more person model - basically, everyone with publish access can upload a new artifact, but to actually have it distributed to endusers, a second person…

Indeed it's not impossible to do (although full code review would be expensive/tricky/slow).

The fact it hasn't been done despite the obvious risks indicates how much demand there is for this feature...

Re: Malicious software libraries found in PyPI posing as well known libraries

#96

Post by julianj »

rantanplan wrote: post #3

The regex they have for identifying fake/harmful packages is wrong. `pip list –format=legacy | egrep '^(acqusition|apidev-coop|bzip|crypt|django-server|pwd|setup-tools|telnet|urlib3|urllib) '` This incorrectly lists `urllib3` or the `cryptography` package for example, which are perfectly valid packages. [UPDATE] Read "tobltobs" comment below. I incorrectly removed a trailing space from the regex.

xml should be added to this list.

https://pypkg.com/pypi/xml/f/setup.py

Re: Malicious software libraries found in PyPI posing as well known libraries

#98

Post by gkya »

raesene6 wrote: post #36
gkya wrote: post #33

Earlier quoted context omitted.

Five hundred thousand packages, am I reading it right? I dont believe top ten OS package managers combined would reach that number. Either this is a typo or it's crazy.

Not a typo, http://www.modulecounts.com/ has the details. npm is adding 497/day at the moment.

This [1] is npm growth compared to anything else. God this can't be safe nor sane...

[1] https://imgur.com/a/enjvR

Re: Malicious software libraries found in PyPI posing as well known libraries

#99

Post by elcapitan »

Would it be possible to have a general package manager (like apt) as reusable base for the individual language specific package managers? I know that npm and pip and gem etc all do some additional stuff, but at the core they all do the same (pull packages from repo, do some postinstall, resolve dependencies, maybe in some cases even check if the package is legit). So we could implement and check that once and then just reuse it like we do with many other libraries for image processing etc.

Re: Malicious software libraries found in PyPI posing as well known libraries

#100

Post by IshKebab »

kasabali wrote: post #53

Yet another attack vector that doesn't exist at all in Linux distributions but invented by language package managers, sadly. They solved the issue 2 decades ago by heavily vetting packages before accepting them into repositories. Users are allowed to add and use packages from 3rd party repositories. Maybe solution to this is creating curated repositories based on publicly open ones and using them by default (and requ…

The Linux distribution approach to package management ("we'll package everything ourselves!") simply doesn't scale.
Post reply on HN