Malicious software libraries found in PyPI posing as well known libraries • Hacker News Board

Re: Malicious software libraries found in PyPI posing as well known libraries

#161

Post by zokier » Fri, Sep 15, 2017, 4:31 PM UTC

hodgesrm wrote: post #151

hannob wrote: post #84

Ok, here's some ugly backstory on this: This problem has been known for a while, yet both the pypi devs and the python security team decided to ignore it. Last year someone wrote his thesis describing python typosquatting and standard library name squatting: http://incolumitas.com/2016/06/08/typosquatting-package-mana... However after that the packages used in this thesis - the most successful one being urllib2 - wer…

This is a scary attack. One partial mitigation is to use a firewall (e.g., Amazon VPC network ACLs) to restrict outbound network traffic to a small number of known addresses like well-known repos. I can't think of a good reason why code in any well-behaved application should be allowed to make random outbound network calls. I think it's also on app developers to rethink the culture of randomly grabbing packages to bu…

> One partial mitigation is to use a firewall (e.g., Amazon VPC network ACLs) to restrict outbound network traffic to a small number of known addresses like well-known repos.

That breaks down very quickly with the combination of public CDNs and TLS. I suppose you could do SNI based firewalling, but that is bit ugly, and afaik you can't do that easily with common firewalls (like netfilter).

Re: Malicious software libraries found in PyPI posing as well known libraries

#162

Post by vosper » Fri, Sep 15, 2017, 4:35 PM UTC

raesene6 wrote: post #28

chatmasta wrote: post #12

Package managers seem to be an increasingly popular attack vector. It's only luck that none of the attacks have been particularly malicious yet. Considering how many package manager downloads go to a server in a datacenter, a widely distributed malicious package could control a botnet with extremely high throughput, or wreak havoc on any databases it comes into contact with. It's only a matter of time before somethin…

Another fun fact to consider is that with many package formats, you can execute arbitrary code at install time so if a malicious package can get into a repository, it's very likely to start compromising systems quickly. Whilst a package manager repo. compromise would be the biggest bang in terms of attack, compromising the credentials of the developers of popualar libraries would be an easier attack (and indeed is al…

With Python you can execute arbitrary code at import time... you don't even have to get to the install process. I've seen packages on Pypi that try to "sudo apt-get install ..." when the setup.py file is imported.

Also, even without sudo there's absolutely nothing stopping you (for example) downloading a cryptocurrency miner, or DDOS tool, or something, and starting it up to run in the background.

Re: Malicious software libraries found in PyPI posing as well known libraries

#163

Post by zokier » Fri, Sep 15, 2017, 4:36 PM UTC

kasabali wrote: post #53

Yet another attack vector that doesn't exist at all in Linux distributions but invented by language package managers, sadly. They solved the issue 2 decades ago by heavily vetting packages before accepting them into repositories. Users are allowed to add and use packages from 3rd party repositories. Maybe solution to this is creating curated repositories based on publicly open ones and using them by default (and requ…

> Yet another attack vector that doesn't exist at all in Linux distributions but invented by language package managers, sadly

Not really, PPAs (and equivalents like copr or obs or aur etc etc) are mostly vulnerable to similar problems. People do want to install upstream software for various reasons, blaming language package managers for the reduced security of that is imho disingenuous.

Re: Malicious software libraries found in PyPI posing as well known libraries

#164

Post by zokier » Fri, Sep 15, 2017, 4:38 PM UTC

teilo wrote: post #156

I think we need a system to prevent this instead of the wild-west that PyPi has become. For example: Developer signatures that are checked against a community rating. If someone does `pip install` pip would look up the developer signature of the package and check a community rating that would verify this is a developer who has offered legit packages in the past. It's not foolproof, but it would go a long way towards…

You could add the ability for well known members to vet newbie developers, maybe by signing their key. And now you have re-invented web of trust.

Re: Malicious software libraries found in PyPI posing as well known libraries

#165

Post by fovc » Fri, Sep 15, 2017, 4:42 PM UTC

raesene6 wrote: post #30

This isn't, in any way, a new problem. I did a presentation on this topic for OWASP AppSecEU 2015 ( https://www.youtube.com/watch?v=Wn190b4EJWk&list=PLpr-xdpM8w... ) and when doing the research for that I encountered cases of repo. attacks and compromise. IME the problem will continue unless the customers (e.g. companies making use of the libraries hosted) are willing to pay more for a service with higher levels of a…

If someone here wants to build a business around this, count me in for NPM (high willingness to pay) or PyPi (lower WTP).

Here's an idea: make it similar to Kickstarter, where customers can commit a certain amount of funds towards a specific package. If the package doesn't "tilt" in a certain amount of time money goes back. Otherwise you vet a point release and add it to your repo. you could offer subscriptions to keep packages updated or handle each update as its own project (with presumably lower costs if a recent release has been audited). Handling dependencies is key as an exercise for the reader

Re: Malicious software libraries found in PyPI posing as well known libraries

#166

Post by zokier » Fri, Sep 15, 2017, 4:45 PM UTC

singularity2001 wrote: post #40

"Success of the attack relies on negligence of the developer" How about package manager managers accept their enourmous responsabilty? urllib vs urllib2, one is a virus? Sorry but that is not "negligence of the developer"

Managing supply chain is one the basic principles of good engineering. Not properly vetting your sources is negligence. The problem of course is that computers are really good at amplifying work, including mistakes. So small mistake, like a typo, could have catastrophic impact, like injecting malware that can take over the whole system.

Re: Malicious software libraries found in PyPI posing as well known libraries

#167

Post by philsnow » Fri, Sep 15, 2017, 4:48 PM UTC

hodgesrm wrote: post #151

hannob wrote: post #84

Ok, here's some ugly backstory on this: This problem has been known for a while, yet both the pypi devs and the python security team decided to ignore it. Last year someone wrote his thesis describing python typosquatting and standard library name squatting: http://incolumitas.com/2016/06/08/typosquatting-package-mana... However after that the packages used in this thesis - the most successful one being urllib2 - wer…

This is a scary attack. One partial mitigation is to use a firewall (e.g., Amazon VPC network ACLs) to restrict outbound network traffic to a small number of known addresses like well-known repos. I can't think of a good reason why code in any well-behaved application should be allowed to make random outbound network calls. I think it's also on app developers to rethink the culture of randomly grabbing packages to bu…

This is probably a good mitigation anyway, but it's not enough: a malicious package could include an (obfuscated) API key that allows it to exfil whatever data it wants to an s3 bucket or similar.

Unless you're super gung-ho about using Service Control Policies _and_ your service doesn't otherwise need to access s3, then there isn't really a way to block this.

Re: Malicious software libraries found in PyPI posing as well known libraries

#168

Post by wongarsu » Fri, Sep 15, 2017, 4:50 PM UTC

cdnsteve wrote: post #66

raesene6 wrote: post #48

Earlier quoted context omitted.

Interesting if you think that npm/Rubygems/PyPI are leaving a load of money on the table, why do you think they haven't introduced those services so far...

Because their mission isn't to generate income like a traditional business. But if the income went back to the foundations, like Python Foundation, I think that would make sense.

But income can be also used to help finance their main mission. Obviously they seem to operate fine without strong reasons to expand revenue streams, but I feel like they ignore an opportunity to create improvements for just about everyone.

Re: Malicious software libraries found in PyPI posing as well known libraries

#169

Post by jsty » Fri, Sep 15, 2017, 4:59 PM UTC

hodgesrm wrote: post #151

hannob wrote: post #84

Ok, here's some ugly backstory on this: This problem has been known for a while, yet both the pypi devs and the python security team decided to ignore it. Last year someone wrote his thesis describing python typosquatting and standard library name squatting: http://incolumitas.com/2016/06/08/typosquatting-package-mana... However after that the packages used in this thesis - the most successful one being urllib2 - wer…

This is a scary attack. One partial mitigation is to use a firewall (e.g., Amazon VPC network ACLs) to restrict outbound network traffic to a small number of known addresses like well-known repos. I can't think of a good reason why code in any well-behaved application should be allowed to make random outbound network calls. I think it's also on app developers to rethink the culture of randomly grabbing packages to bu…

> I can't think of a good reason why code in any well-behaved application should be allowed to make random outbound network calls

If your application implements webhooks, then there's a valid use case? I'm in complete agreement however that you should default to deny and open up as required.

Re: Malicious software libraries found in PyPI posing as well known libraries

#170

Post by wongarsu » Fri, Sep 15, 2017, 5:01 PM UTC

teilo wrote: post #156

I think we need a system to prevent this instead of the wild-west that PyPi has become. For example: Developer signatures that are checked against a community rating. If someone does `pip install` pip would look up the developer signature of the package and check a community rating that would verify this is a developer who has offered legit packages in the past. It's not foolproof, but it would go a long way towards…

That sounds easy to defeat. Make some mundane, but legit packages (maybe on of those "$X but without the pointless complexity"-packages), gain trust, once trust is reached start uploading typo-squatting packages.

Knowing today's internet, programmers from cheap-labour nations (India & Co.) would soon start offering "trusted PyPi accounts" for sale on hacker forums.