Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Malicious software libraries found in PyPI posing as well known libraries

nbu.gov.sk

Post reply on HN
Print view View original

231–240 of 254 posts

Re: Malicious software libraries found in PyPI posing as well known libraries

#231

Post by zzzeek »

danjoc wrote: post #229
zzzeek wrote: post #192

Earlier quoted context omitted.

> In contrast, Maven central requires signing. forgive my ignorance and my lack of 56 minutes to watch the entire youtube video, but who are the identities behind these signatures? The blog post you reference discusses the problem both of users signing their own packages (anyone can make a signature and any malicious package author can point people at a maliciously-owned signature as well) as well as having a central…

>who are the identities behind these signatures? https://maven.apache.org/guides/mini/guide-central-repositor... "we require you to provide PGP signatures for all your artifacts (all files except checksums), and distribute your public key to a key server like http://pgp.mit.edu." >anyone can make a signature The article flip flops on this. any hacker can do it it's too much burden for developers >any malicious packag…

> Do python packages even list licenses?

Of course they do and it goes into the package classifiers.

Re: Malicious software libraries found in PyPI posing as well known libraries

#232

Post by defined »

chupapuma wrote: post #155
defined wrote: post #140

Here's something that contributes to typosquatting: the lack of responsiveness by package management organizations to claims on orphaned or unmaintainable packages. People who upload packages often leave organizations, who are then stuck with a package they can't update because the password went with the person, and the email reset link points to a now-defunct email address. Petitioning the package management team is…

I have found the PyPI group of people to be very helpful in these cases. You also should probably, as an organization, have more than one owner of your packages. That way, unless two people leave, things aren't orphaned. We have gone as far to have a 'meta-user' that is on all packages. It is only ever used to recover a fully abandoned package.

I understand you are trying to be helpful, and of course you are right, but the fact is that sometimes things fall between the cracks, especially in, say, hard-pressed startups.

There are so many shoulds in the world that don't make it to dids, it reminds me of the joke about the salesman trying to sell farming improvement techniques and being turned down by the old farmer, who says, "Son, I don't farm half as good as I know how to already."

Unfortunately, I have not found the PyPI group as helpful as you have. Perhaps I have been looking in the wrong places.

Re: Malicious software libraries found in PyPI posing as well known libraries

#233

Post by sounds »

chupapuma wrote: post #152
edraferi wrote: post #123

Earlier quoted context omitted.

Man. You're right - that's a mess.

And just to be super clear, this is the code. Hard from scary or obfuscated: html = urllib_request.urlopen( "https://www.pytosquatting.org/pingback/pypi/{}/".format(package_name) ) raise Exception( "This is a bogus package that should not be installed\n\n" "Please read https://www.pytosquatting.org" )

"Hard from scary or obfuscated" Maybe you are typing on mobile?

"Far from scary or obfuscated" reads more clearly.

Re: Malicious software libraries found in PyPI posing as well known libraries

#234

Post by cookiecaper »

pmoriarty wrote: post #197
cookiecaper wrote: post #189

Earlier quoted context omitted.

It has to be constrained to something reasonable. You can't guarantee the software is safe, but you can guarantee it is published by someone who is who they say they are, similar to EV certificates for domains. You can also refuse to publish packages with intentionally-confusing names.

"you can guarantee it is published by someone who is who they say they are" Can you? Positively identifying people seems a pretty tricky and easily screwed up business. ID's can be forged, and a web of trust requires, well, trust. I guess such a service could say something like "we got this person's ID (and/or address)" or "here's this key's web of trust", and that would probably be a bit better than what we have tod…

EV certs have a complex verification process that can involve sending a physical representative from the company down to the place of business to confirm its presence/existence.

Bitcoin trading platforms have shown that compliance with AML/KYC regulations can be performed virtually by manual verification of a valid government ID, timestamped photo, handwritten note, and other mechanisms.

A company offering this service would go outside of the keyserver and verify the ID independently. It'd be much more of a "notarized packages" paradigm rather than just "published by 1337PyHax0r-88".

It is true that even extensive manual verification processes dependent on government-issued IDs can be faked, but there's a much higher bar involved.

Re: Malicious software libraries found in PyPI posing as well known libraries

#235

Post by cookiecaper »

xapata wrote: post #222
cookiecaper wrote: post #187

Earlier quoted context omitted.

Typosquatting and package signatures are separate issues. Package signing only prevents typosquatting insofar as either the user or some intermediate layer resolves the typo to the intended package. If someone was going to this effort, they'd probably go to the effort of double-checking the package name before installation anyway. PyPi needs moderators to sit in the middle and remove anything that is obviously malici…

> needs moderators Like Anaconda and Enthought? And countless internal departments? Or are you suggesting folks donate to the PSF and they hire a team?

I'm suggesting that whoever currently has admin rights on the PyPi packaging servers, and it's someone, take responsibility for this and physically remove the typosquatting libs from the lookup mechanism. "We need donations before we can do that" doesn't pass muster as far as I'm concerned; leaving this unaddressed is an existential issue for PyPI.

There are already privately-maintained repositories and that's great, but IMO it's not an excuse for PyPi to leave this vulnerability open.

Re: Malicious software libraries found in PyPI posing as well known libraries

#236

Post by xapata »

cookiecaper wrote: post #235
xapata wrote: post #222

Earlier quoted context omitted.

> needs moderators Like Anaconda and Enthought? And countless internal departments? Or are you suggesting folks donate to the PSF and they hire a team?

I'm suggesting that whoever currently has admin rights on the PyPi packaging servers, and it's someone, take responsibility for this and physically remove the typosquatting libs from the lookup mechanism. "We need donations before we can do that" doesn't pass muster as far as I'm concerned; leaving this unaddressed is an existential issue for PyPI. There are already privately-maintained repositories and that's great,…

Ah, so you're going to refuse to use PyPI until someone volunteers. I'm not certain that's an existential issue for PyPI. I plan to continue using it.

Re: Malicious software libraries found in PyPI posing as well known libraries

#237

Post by larrykwg »

takluyver wrote: post #215
eeZah7Ux wrote: post #107

Earlier quoted context omitted.

That's a feature. You want a set of vetted and curated packages you can trust, and you want to receive security updates on them.

But developers also want a way to get software without getting it blessed by Debian and waiting months/years for a distro release. That's why repositories like PyPI exist and are in very widespread use. Distro repositories are a great example of 'secure for ideal users'. They give you security if you can put up with a small selection of software and older versions. In practice, we end up working around distro reposit…

I don't understand why they even try, Debian stable seem to have an almost arbitrary selection of outdated ruby and python libraries, at this point that hardly seems worth the effort. Sure I get the idea, but it obviously doesn't work in practice. Their security methodology also seems heavily flawed, to backport security fixes to older versions is neither scalable nor particularly reliable. I sincerely doubt that Debian can provide adequate security to its almost 50,000 packages. If the security community would invest equal amount of resources that they invest in finding flaws in iOS or Chrome, nothing would be left of Debian but a pile of smoking ashes.

Re: Malicious software libraries found in PyPI posing as well known libraries

#238

Post by rplnt »

nyrikki wrote: post #154
drdaeman wrote: post #46

Earlier quoted context omitted.

Maybe you're right, but I see one possible downside that is quite important. I have encountered the case "the package has an important bugfix but is not yet published on PyPI" way more than once or twice. With the intermediate maintainers, that's going to get worse. I believe namespaces and signatures are the way to go. With a special privileged namespace for the curated widely known packages (e.g. SciPy or Django) -…

PyPI should also run a build to audit behavior which would be fairly easy to implement. A submitted package would just fail if it access the network or privileged files during compile unless unique needs are called out in an spec file. I do wish that `--user` was the default for pip. It is also a pity that trivial Debian bugs like this block adoption of non sudo pip installs weren't ignored. https://bugs.debian.org/c…

With this system, one could pull a Volkswagen.

Re: Malicious software libraries found in PyPI posing as well known libraries

#239

Post by ThiefMaster- »

kevin_thibedeau wrote: post #79

Python needs a way to run 2to3 during package installation that doesn't use setup.py (setup.cfg or wheels). As it stands now, you have the hassle of building a release four times if you want to support all combos of Py2, Py3, 32-bit, and 64-bit platforms. The absent support for 2 to 3 migration in the safer alternatives is why I stick with setup.py.

No, what you need to do is fix your package's code to work on Python 2 and 3 without running 2to3 on it. The only case where this doesn't work is if you have binary extensions - but then you need separate wheels in any case.

Re: Malicious software libraries found in PyPI posing as well known libraries

#240

Post by eeZah7Ux »

concede_pluto wrote: post #178
eeZah7Ux wrote: post #109

Earlier quoted context omitted.

The left-pad disaster has been predicted well in advance...

There's nothing especially awful about left-pad being its own package, the disaster was because a huge number of developers were betting on npm to somehow be highly available (despite being donated by its admins at no cost and with no committed SLA) rather than vendoring their deps.

Vendoring thousands of tiny libs is even worse. Trusting many lesser known, tiny libs is more risky than few, big well known ones.

Also, they are not vetted and there are much more opportunities for an attacker to sneak in a backdoored lib on the edge of the dependency graph.

Finally, due to vendoring there's no way to receive timely drop-in security fixes for all dependencies from a trusted source.

Post reply on HN