Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Devs unknowingly use “malicious” modules snuck into official Python repository

arstechnica.com

Post reply on HN
Print view View original

51–60 of 119 posts

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#51

Post by seanp2k2 »

zizek23 wrote: post #43
jacalata wrote: post #41

Earlier quoted context omitted.

> Users should simply boycott such user hostile developers and languages I await with interest your newsletter to "the internet" on how to boycott JavaScript.

There are a lot of constructive ways to begin to solve this but I guess you prefer the camel in sand approach. Every single Ruby post has commentators complaining about dependency hell and steering clear of Ruby apps. This was not the case even a couple of years ago. This is effectively a user boycott which Ruby may not deserve but has brought on itself by letting the 'break everything crowd' run amok. They have move…

What has changed in the past few years in the Ruby world that has changed dep mgmt? Bundler has been the thing to use for quite a while now, and it used a version lock for all dependencies. It took Node a little while to figure that one out, and shrinkwrap seemingly led to yarn then back into NPM improvements. Ruby is comparatively not that bad, excepting libs with e.g. C dependencies which Node has as well.

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#52

Post by dom0 »

sametmax wrote: post #38
swiley wrote: post #2

One nice thing about languages like C is that a lot of programmers just avoid dependencies because dealing with them kind of sucks. That's one solution to this problem.

Yeah this way you reinvent a less unit tested, battle field tested and documented wheel. It's like saying the good thing with not having a cellphone is that you avoid a lot of fight with your girlfriend cause you can't talk as much.

> It's like saying the good thing with not having a cellphone is that you avoid a lot of fight with your girlfriend cause you can't talk as much.

I can't quite see what's wrong with that...

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#53

Post by gjjrfcbugxbhf »

viraptor wrote: post #22
zizek23 wrote: post #17

The casual culture of pulling in hundred of dependencies, and mushrooming language specific package managers is ridiculously insecure and has to go. There is no way for anyone to know what all this code is doing, there is little way to verify updates and its simply untenable. If some developers like this sort of unsafe practice it should be strictly limited to their machines and in no way make it across in any form a…

You mean that every time you write a new project you write your own request parser, your own server, your own web framework, your own database access libraries, etc. from scratch? Packages taken over include misspelling of urllib. What exactly do you propose as an alternative here? I'm all for limiting bloat, but your rant here seems completely inappropriate for the issue bring described.

> Packages taken over include misspelling of urllib. What exactly do you propose as an alternative here?

1. Similar way of identifying who is responsible for a package to Debian style repo - would allow those affected to identify a named person if malicious code added.

2. Fuzzy matching of package names (so taking the library name urllib also takes urilib, urlib, urllib2 and other names with a short document distance).

3. Built in mechanism to allow third party reviews within the repositories infrastructure.

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#54

Post by seanp2k2 »

onion2k wrote: post #36
xroche wrote: post #26

Earlier quoted context omitted.

> You mean that every time you write a new project you write your own request parser, your own server, your own web framework, your own database access libraries, etc. from scratch? No, but downloading random code is just insane. As a comparison, Debian has ~3000 packages, and every single package had an identified maintainer, with its own GPG key, validated in face-to-face meeting with an ID card by three people, an…

Debian has ~3000 packages, and every single package had an identified maintainer... As of yesterday npmjs had 516,132 packages, which was an increase of 373 since the day before. Debian's 3000 packages is a couple of weeks of npmjs activity. Even if you stripped out the unnecessary, abandoned, or duplicate packages you're still looking at a something significantly different to Debian. For what it's worth I think a we…

The culture around what should be in a package is vastly different. Node has left-pad. Debian has stuff like Apache. Sure, those are radical examples, but the barrier to entry for Debian is pretty high (as is the standard for quality) vs npm where anyone can put whatever out there.

How do things like left-pad even come to be widespread dependencies? Does the node development process involve a lot of "gee I wonder if someone made a package for (simple thing I need to do with a few lines of code)"?

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#55

Post by camus2 »

Well pip has the same problem as NPM : no namespaces by default.

But NPM is worse with all its dependencies of dependencies. Composer (PHP) got both namespace and dependencies right: flat dependencies, it's up to the developer to resolve conflicts, not to the package manager to create insane dependency trees.

It leads to more stable packages and make spotting fakes easier.

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#56

Post by coldtea »

viraptor wrote: post #22
zizek23 wrote: post #17

The casual culture of pulling in hundred of dependencies, and mushrooming language specific package managers is ridiculously insecure and has to go. There is no way for anyone to know what all this code is doing, there is little way to verify updates and its simply untenable. If some developers like this sort of unsafe practice it should be strictly limited to their machines and in no way make it across in any form a…

You mean that every time you write a new project you write your own request parser, your own server, your own web framework, your own database access libraries, etc. from scratch? Packages taken over include misspelling of urllib. What exactly do you propose as an alternative here? I'm all for limiting bloat, but your rant here seems completely inappropriate for the issue bring described.

>You mean that every time you write a new project you write your own request parser, your own server, your own web framework, your own database access libraries, etc. from scratch?

No, they mean that we need a better method than trusting some random popular GitHub / PyPy / npm whatever storage and delivery mechanism.

Strong core language libraries ("batteries included"), that come with your distribution of the language/compiler and are well maintained, used by millions, and signed would be a good start to at least have 80% of dependencies being actually dependable.

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#57

Post by coldtea »

onion2k wrote: post #36
xroche wrote: post #26

Earlier quoted context omitted.

> You mean that every time you write a new project you write your own request parser, your own server, your own web framework, your own database access libraries, etc. from scratch? No, but downloading random code is just insane. As a comparison, Debian has ~3000 packages, and every single package had an identified maintainer, with its own GPG key, validated in face-to-face meeting with an ID card by three people, an…

Debian has ~3000 packages, and every single package had an identified maintainer... As of yesterday npmjs had 516,132 packages, which was an increase of 373 since the day before. Debian's 3000 packages is a couple of weeks of npmjs activity. Even if you stripped out the unnecessary, abandoned, or duplicate packages you're still looking at a something significantly different to Debian. For what it's worth I think a we…

>As of yesterday npmjs had 516,132 packages, which was an increase of 373 since the day before. Debian's 3000 packages is a couple of weeks of npmjs activity.

Only in volume. In code quality and man hours they're an order of magnitude ahead of all of npm.

(Just consider something like Linux the kernel, GCC, Apache, OpenJDK, core utils, Python, Postgres, etc, compared to the trite that even the best npm packages are).

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#58

Post by hannob »

sametmax wrote: post #37
userbinator wrote: post #18

The IP address it phones home to, 121.42.217.44, is located in China and visiting it with HTTP just displays this interesting message: Hi bro :) Welcome Here! Leave Messages via HTTP Log Please :) On 2017-09-16: Happy to see somebody find it ! :) Just curious about how long it would take for people to find those 'bad' packages As you see, that's just a toy script, no harm, hope you enjoy it ! It looks like someone (s…

It may be https://pytosquatting.org/ . In that case it was indeed a harmless and quite useful PoC.

No. (Co-operator of pytosquatting here)

While our code was similar in nature, it was non-obfuscated and we always threw an exception telling the user that he installed something he shouldn't.

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#59

Post by skrause »

hannob wrote: post #58
sametmax wrote: post #37

Earlier quoted context omitted.

It may be https://pytosquatting.org/ . In that case it was indeed a harmless and quite useful PoC.

No. (Co-operator of pytosquatting here) While our code was similar in nature, it was non-obfuscated and we always threw an exception telling the user that he installed something he shouldn't.

But you still call an URL first, e.g. https://www.pytosquatting.org/pingback/pypi/urllib2/ for "urllib2" before raising the exception. Though it is not an IP in China and only returns the text "Pingback from package urllib2 in repository pypi".

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#60

Post by jlgaddis »

xroche wrote: post #26
viraptor wrote: post #22

Earlier quoted context omitted.

You mean that every time you write a new project you write your own request parser, your own server, your own web framework, your own database access libraries, etc. from scratch? Packages taken over include misspelling of urllib. What exactly do you propose as an alternative here? I'm all for limiting bloat, but your rant here seems completely inappropriate for the issue bring described.

> You mean that every time you write a new project you write your own request parser, your own server, your own web framework, your own database access libraries, etc. from scratch? No, but downloading random code is just insane. As a comparison, Debian has ~3000 packages, and every single package had an identified maintainer, with its own GPG key, validated in face-to-face meeting with an ID card by three people, an…

> ... Debian has ~3000 packages ...

It's irrelevant to your point but Debian 9 included >51,000 packages [0].

[0]: https://www.debian.org/News/2017/20170617

Post reply on HN