Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Malicious software libraries found in PyPI posing as well known libraries

nbu.gov.sk

Post reply on HN
Print view View original

111–120 of 254 posts

Re: Malicious software libraries found in PyPI posing as well known libraries

#111

Post by eeZah7Ux »

raesene6 wrote: post #24
IgorPartola wrote: post #15

This to me is the nightmare scenario. Well one of the two, the other one being that a developer of an obscure library I use has their password to PyPI compromised and a bad actor uploads a backdoored version of the library. Fundamentally, the reason this is different from how thinks like Linux distos work is because Linux distros have maintainers who are in charge of making sure every new update to one of their packa…

A review system unfortunately isn't likely to be practicable with current development models. npm alone has over 500,000 packages ( http://www.modulecounts.com/ ) so even a one time review isn't going to happen. If people want a more trusted solution the likely outcome is that they'll need to use a smaller more static set of libraries and then either do the audits themselves, or outsource that to a 3rd party. Ofc wit…

> they'll need to use a smaller more static set of libraries

That's what stable Linux distributions do.

Re: Malicious software libraries found in PyPI posing as well known libraries

#112

Post by peterwwillis »

zzzcpan wrote: post #91
ex_amazon_sde wrote: post #73

Earlier quoted context omitted.

Debian Developers are responsible of the package quality. Most package only well-known software that they are familiar with or read the code. Some do more thorough security audits. Some security-sensitive packages are maintained by teams to share the workload.

They do a lot that could be and sometimes even is automated, but that doesn't help with security, only weakens it. Which is my point. It's better to automate package generation and trust fewer people, mainly the authors, not introduce maintainers into it.

Authors don't generate distro packages because there's too many distros and each distro needs to make changes that have nothing to do with code. Maintainers are necessary and a package signed by the author is a non-starter.

However, source code can be signed and then used to make a package signed by a distro.

Re: Malicious software libraries found in PyPI posing as well known libraries

#113

Post by eeZah7Ux »

elcapitan wrote: post #99

Would it be possible to have a general package manager (like apt) as reusable base for the individual language specific package managers? I know that npm and pip and gem etc all do some additional stuff, but at the core they all do the same (pull packages from repo, do some postinstall, resolve dependencies, maybe in some cases even check if the package is legit). So we could implement and check that once and then ju…

+1 to this. There's no need to reinvent the wheel a million times. At least having a shared standard on how to do packaging.

Re: Malicious software libraries found in PyPI posing as well known libraries

#114

Post by peterwwillis »

kumarvvr wrote: post #88
zapt02 wrote: post #69

Earlier quoted context omitted.

> Do digitally signed certificates fit into this usage scenario?? No, because either the package author would have to sign them, in which case you have to choose to trust each package author, or the repository would sign them, in which case there would be no improvement for this current issue, since the repo would sign the fake packages as well.

Any way in which blockchain technology can be used? Like, the transaction becomes the act of the author uploading the code and the repo and user verify the transaction in some form?

Nope. You can't solve phishing with technological means. You have to curate either a whitelist or blacklist.

The best way to handle this is whitelists of trusted package maintainers and/or code authors.

Re: Malicious software libraries found in PyPI posing as well known libraries

#115

Post by edraferi »

xnyhps wrote: post #60
sdiepend wrote: post #5

Related to this? http://incolumitas.com/2016/06/08/typosquatting-package-mana...

That one is even more malicious, it uploads contents of your ~/.bash_history and system profile. But at least it notifies you afterwards...

Yes, but they do filter bash_history client side, only transmitting pip-related commands. They did this to find additional common typos. The relevant code:

  def get_command_history():
    if os.name == 'nt':
      # handle windows
      # http://serverfault.com/questions/95404/
      #is-there-a-global-persistent-cmd-history
      # apparently, there is no history in windows :(
      return ''
  
    elif os.name == 'posix':
      # handle linux and mac
      cmd = 'cat {}/.bash_history | grep -E "pip[23]? install"'
      return os.popen(cmd.format(os.path.expanduser('~'))).read()

Re: Malicious software libraries found in PyPI posing as well known libraries

#117

Post by IgorPartola »

IshKebab wrote: post #101
IgorPartola wrote: post #15

This to me is the nightmare scenario. Well one of the two, the other one being that a developer of an obscure library I use has their password to PyPI compromised and a bad actor uploads a backdoored version of the library. Fundamentally, the reason this is different from how thinks like Linux distos work is because Linux distros have maintainers who are in charge of making sure every new update to one of their packa…

> Fundamentally, the reason this is different from how thinks like Linux distos work is because Linux distros have maintainers who are in charge of making sure every new update to one of their packages is legit. How is that different?

Because the person who pushes the code to the public repo is not the same person who makes sure it isn't malicious. You have a review process. Nothin is stopping me right now from creating a PyPI package called Django2.0 and having some poor souls download it. Or creating a tiny but useful utility, having it become popular, then introducing an update with a backdoor.

Re: Malicious software libraries found in PyPI posing as well known libraries

#118

Post by kumarvvr »

peterwwillis wrote: post #114
kumarvvr wrote: post #88

Earlier quoted context omitted.

Any way in which blockchain technology can be used? Like, the transaction becomes the act of the author uploading the code and the repo and user verify the transaction in some form?

Nope. You can't solve phishing with technological means. You have to curate either a whitelist or blacklist. The best way to handle this is whitelists of trusted package maintainers and/or code authors.

Ohh ok. Thanks. Great to know.

Re: Malicious software libraries found in PyPI posing as well known libraries

#119

Post by hannob »

edraferi wrote: post #103
hannob wrote: post #84

Ok, here's some ugly backstory on this: This problem has been known for a while, yet both the pypi devs and the python security team decided to ignore it. Last year someone wrote his thesis describing python typosquatting and standard library name squatting: http://incolumitas.com/2016/06/08/typosquatting-package-mana... However after that the packages used in this thesis - the most successful one being urllib2 - wer…

I appreciate the proactive approach. Is your project the author of the packages identified by NBU? If so: (1) Why is the tracking pingback obfuscated? (2) Why does the code include a cheeky hello instead of a link to https://www.pytosquatting.org/ ? (3) Why is there not a visible warning when installing one of these packages? ================= edit: Reading through the linked blog post [0], it appears these researche…

We're not the authors of those packages. But we own many others.

1. We're not obfuscating pingbacks.

2./3. We're raising an exception with an explanation and a link.

Just look at the code of one of our packages: https://pypi.python.org/pypi/codecs

The research in 2016 was done by someone else. The kinda crazy thing is: Some of the package names he used were made available again after that instead of being blocked... And now we own them.

Re: Malicious software libraries found in PyPI posing as well known libraries

#120

Post by daenney »

ex_amazon_sde wrote: post #78
disconnected wrote: post #61

Earlier quoted context omitted.

> Yet another attack vector that doesn't exist at all in Linux distributions but invented by language package managers, sadly. https://www.schneier.com/blog/archives/2008/05/random_number... A.K.A., the Debian openssl Fiasco. Just one example of distros fucking up the packages from upstream and causing major havoc.

You are cherry-picking one example involving a library that had a plethora of vulnerabilities from upstream. Contrast it with reviewing and maintaining 50k+ packages, managing thousands of CVEs every year, sometimes even writing security patches before upstream. Also the project pioneered reproducible builds and implemented build hardening for most packages.

None of that, nor the good that they have done, negates the fact that distributions can fuck up too. And regardless of whether OpenSSL had a plethora of bugs from upstream, this one wasn't one of them. It was only in Debian, because of changes the project had made. Just because it's packaged in Debian by Debian maintainers doesn't mean you're immune to these kind of issues. They're arguably less likely but you'd need to do a comprehensive study of all packages in the repo to get to some usable statistic.
Post reply on HN