Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Malicious software libraries found in PyPI posing as well known libraries

nbu.gov.sk

Post reply on HN
Print view View original

151–160 of 254 posts

Re: Malicious software libraries found in PyPI posing as well known libraries

#151

Post by hodgesrm »

hannob wrote: post #84

Ok, here's some ugly backstory on this: This problem has been known for a while, yet both the pypi devs and the python security team decided to ignore it. Last year someone wrote his thesis describing python typosquatting and standard library name squatting: http://incolumitas.com/2016/06/08/typosquatting-package-mana... However after that the packages used in this thesis - the most successful one being urllib2 - wer…

This is a scary attack. One partial mitigation is to use a firewall (e.g., Amazon VPC network ACLs) to restrict outbound network traffic to a small number of known addresses like well-known repos. I can't think of a good reason why code in any well-behaved application should be allowed to make random outbound network calls.

I think it's also on app developers to rethink the culture of randomly grabbing packages to build applications quickly. This is already a security problem even with approved repos. Having a rat's nest of packages makes it hard to upgrade quickly when those repos post updates to address vulnerabilities.

Edit: Removed confusing statement about return connections

Re: Malicious software libraries found in PyPI posing as well known libraries

#152

Post by chupapuma »

edraferi wrote: post #123
hannob wrote: post #119

Earlier quoted context omitted.

We're not the authors of those packages. But we own many others. 1. We're not obfuscating pingbacks. 2./3. We're raising an exception with an explanation and a link. Just look at the code of one of our packages: https://pypi.python.org/pypi/codecs The research in 2016 was done by someone else. The kinda crazy thing is: Some of the package names he used were made available again after that instead of being blocked...…

Man. You're right - that's a mess.

And just to be super clear, this is the code. Hard from scary or obfuscated:

  html = urllib_request.urlopen(
  "https://www.pytosquatting.org/pingback/pypi/{}/".format(package_name)
  )
  raise Exception(
    "This is a bogus package that should not be installed\n\n"
    "Please read https://www.pytosquatting.org"
  )

Re: Malicious software libraries found in PyPI posing as well known libraries

#154

Post by nyrikki »

drdaeman wrote: post #46
Sir_Cmpwn wrote: post #29

I think a more Linux-like approach to package repos is better - a curated package repository run by volunteers in maintainership roles. Then you have a human being verifying the upstream and keeping malware out, and get more consistency across packages as a bonus. If you want your package added it's as simple as sending an email and provides a new avenue for people to contribute to the success of the ecosystem as pac…

Maybe you're right, but I see one possible downside that is quite important. I have encountered the case "the package has an important bugfix but is not yet published on PyPI" way more than once or twice. With the intermediate maintainers, that's going to get worse. I believe namespaces and signatures are the way to go. With a special privileged namespace for the curated widely known packages (e.g. SciPy or Django) -…

PyPI should also run a build to audit behavior which would be fairly easy to implement. A submitted package would just fail if it access the network or privileged files during compile unless unique needs are called out in an spec file.

I do wish that `--user` was the default for pip.

It is also a pity that trivial Debian bugs like this block adoption of non sudo pip installs weren't ignored.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=839155

Although Debian/Ubuntu default to --user on pip people resort to sudo because the current standard user bin directory isn't in the default path due to a regression.

I may start a project to create a apparmor/selinux wrapper for pip to audit and restrict access to sensitive resources. I actually have a fairly heavyweight version in place on my build pipeline to detect new dependencies. I add the files/network resources that a build accesses outside of the testing stage to the build artifacts. But it wouldn't be cross platform enough for Windows/Mac.

Re: Malicious software libraries found in PyPI posing as well known libraries

#155

Post by chupapuma »

defined wrote: post #140

Here's something that contributes to typosquatting: the lack of responsiveness by package management organizations to claims on orphaned or unmaintainable packages. People who upload packages often leave organizations, who are then stuck with a package they can't update because the password went with the person, and the email reset link points to a now-defunct email address. Petitioning the package management team is…

I have found the PyPI group of people to be very helpful in these cases. You also should probably, as an organization, have more than one owner of your packages. That way, unless two people leave, things aren't orphaned. We have gone as far to have a 'meta-user' that is on all packages. It is only ever used to recover a fully abandoned package.

Re: Malicious software libraries found in PyPI posing as well known libraries

#156

Post by teilo »

I think we need a system to prevent this instead of the wild-west that PyPi has become. For example: Developer signatures that are checked against a community rating. If someone does `pip install` pip would look up the developer signature of the package and check a community rating that would verify this is a developer who has offered legit packages in the past. It's not foolproof, but it would go a long way towards solving this.

Re: Malicious software libraries found in PyPI posing as well known libraries

#158

Post by geekamongus »

Why is there no indication of any of this on the python.org website or any of their social media accounts?

I checked:

https://pypi.python.org/pypi

https://www.python.org/blogs/

http://planetpython.org/

https://pypi.python.org/security

https://twitter.com/pythoninsider

https://plus.google.com/+Python

https://www.facebook.com/pythonlang?fref=ts

https://twitter.com/ThePSF

Re: Malicious software libraries found in PyPI posing as well known libraries

#159

Post by tekromancr »

ams6110 wrote: post #148
raesene6 wrote: post #28

Earlier quoted context omitted.

Another fun fact to consider is that with many package formats, you can execute arbitrary code at install time so if a malicious package can get into a repository, it's very likely to start compromising systems quickly. Whilst a package manager repo. compromise would be the biggest bang in terms of attack, compromising the credentials of the developers of popualar libraries would be an easier attack (and indeed is al…

Doubly so since when installed on a server very often it will be done as "sudo pip install ....." (/s/pip/other-package-manager/ as needed)

I almost never see this. Even on systems that are only running a single python project, I only ever see folks use virtualenv. The only time I ever see things installed with sudo is when the package is being installed in a docker container.

Re: Malicious software libraries found in PyPI posing as well known libraries

#160

Post by striking »

tekromancr wrote: post #159
ams6110 wrote: post #148

Earlier quoted context omitted.

Doubly so since when installed on a server very often it will be done as "sudo pip install ....." (/s/pip/other-package-manager/ as needed)

I almost never see this. Even on systems that are only running a single python project, I only ever see folks use virtualenv. The only time I ever see things installed with sudo is when the package is being installed in a docker container.

This used to be super common, though. I see it all the time in legacy apps.
Post reply on HN