Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Devs unknowingly use “malicious” modules snuck into official Python repository

arstechnica.com

Post reply on HN
Print view View original

101–110 of 119 posts

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#101

Post by flachsechs »

ageofwant wrote: post #76
zizek23 wrote: post #17

The casual culture of pulling in hundred of dependencies, and mushrooming language specific package managers is ridiculously insecure and has to go. There is no way for anyone to know what all this code is doing, there is little way to verify updates and its simply untenable. If some developers like this sort of unsafe practice it should be strictly limited to their machines and in no way make it across in any form a…

Bulshit. The benefits of this "casual culture of pulling in hundred of dependencies" vastly outweigh the harm. I'll be assembling another $6000 job that will take me maybe 12 hours to complete while you write your compiler, from scratch, in your own assembler, made for your own cpu, that you youre gonna cook up from a bucket of sand you collected yourself, from a sandpit you trust.

i agree that the dependencies are probably unavoidable and probably a net good but... well... just don't forget the system and network security best practices while you quickly put together that project, especially if it's in a public cloud.

btw you should be making $6000 + $1000 or so a month for at least a couple of years doing that, not $6000 a single time. the total lifetime value of setting up an application should be above $30k, hopefully way above, but at the entry level where you are that's a nice healthy number to shoot for.

we bill our total customer base over $6000 a day for this kind of work and it all starts with 'assembling a job' but certainly doesn't end there, or ever (hopefully). good luck buddy!

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#102

Post by Stranger43 »

namelost wrote: post #64
zizek23 wrote: post #23

Earlier quoted context omitted.

A package manager must at a minimum address code security, dependency hell and deployment. A json parser that wgets urls into a folder is not a package manager. These things must be thought of upfront and cannot be left vague and open to adhoc practices that leak to end users and deployment creating insecurity. For instance a package should be a package, it should not be a simple class or function that in turn pulls…

Who says? npm is proof that package managers need to do none of these things to be successful, and indeed the traditional design of a package manager as a constraint solver is self-defeating, because it puts a perverse pressure on libraries to have as few dependencies as possible and to avoid using the package manager to resolve their dependencies . The reason npm has teething problems is because it's one of the firs…

npm is today about 15 steps behind where CPAN was doing the hight of Perl, have anyone actually tired CPAN on something new say an aarch64 or even good old armv7 CPU, or a excotic new uclib/busybox based linux distro like alpine?

How confident are anybody that a random CPAN package's original maintainer is still actively maintaining anything given the average perl hackers age? and how confident would you be that the NPM repos are going to suffer under less bitrot then CPAN in say 10 years from now?

How confident are you that all major versions of your own code will be either removed from any active package manager or production install or patched for all known security flaws?

And for last the big one how confident are you that all of the fixed version dependencies you add can you make any guarantee that they wont be a big open security hole sitting waiting for anyone still stupid enough to just npm install code 3 years from now?

NPM benefits from still being just before/around peak hype where most of the people who commited to npm's repo's are still early in their careers, at one point it will face the point where most original package submitters have abandoned the task of maintaining them, even if node itself remains around to an greater extend then Perl still does today?

In the linux package management world it have significance when a project is packaged and put into a core repo, as most Linux distributors promise to help fix abandoned code included with the core distribution, nobody in the world of gem, yarn, pip or npm makes any such guarantee and nobody screens new package maintainers before granting them a "name" with commit access to even the degree of Debian or Fedora, which are both fairly open communities.

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#103

Post by Ajedi32 »

chii wrote: post #100
Ajedi32 wrote: post #88

Earlier quoted context omitted.

If by "transitive dependencies problem" you mean a case where package A depends on package B which depends on package C and you want to use A in your project, then it solves that by simply pulling in all three of those packages. Or did you mean the case where you want package D as well, and it depends on a conflicting version of package C? In that case, it solves the problem by pulling in both versions of package C a…

I do mean the latter - A and D both using different versions of C. Running both C side-by-side cannot work if C is incompatible (that is, even if both versions are api compatible, but each assumes it's the sole C being loaded - and therefore, do some static/singleton crap that might get clobbered when loaded again).

> Running both C side-by-side cannot work if C is incompatible

I've honestly never encountered an NPM package which couldn't be run side-by-side with another version of itself. This is due to the nature of how Node's module system (CommonJS) works; packages are isolated from each other and only share resources with each other via explicit exports and imports.

I suppose a conflict might be possible if the package was using native extensions or connecting to some external service or something, but for the most part NPM's module system makes conflicts very unlikely.

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#104

Post by [deleted] »

seanp2k2 wrote: post #54
onion2k wrote: post #36

Earlier quoted context omitted.

Debian has ~3000 packages, and every single package had an identified maintainer... As of yesterday npmjs had 516,132 packages, which was an increase of 373 since the day before. Debian's 3000 packages is a couple of weeks of npmjs activity. Even if you stripped out the unnecessary, abandoned, or duplicate packages you're still looking at a something significantly different to Debian. For what it's worth I think a we…

The culture around what should be in a package is vastly different. Node has left-pad. Debian has stuff like Apache. Sure, those are radical examples, but the barrier to entry for Debian is pretty high (as is the standard for quality) vs npm where anyone can put whatever out there. How do things like left-pad even come to be widespread dependencies? Does the node development process involve a lot of "gee I wonder if…

[deleted]

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#105

Post by pmilot »

chii wrote: post #100
Ajedi32 wrote: post #88

Earlier quoted context omitted.

If by "transitive dependencies problem" you mean a case where package A depends on package B which depends on package C and you want to use A in your project, then it solves that by simply pulling in all three of those packages. Or did you mean the case where you want package D as well, and it depends on a conflicting version of package C? In that case, it solves the problem by pulling in both versions of package C a…

I do mean the latter - A and D both using different versions of C. Running both C side-by-side cannot work if C is incompatible (that is, even if both versions are api compatible, but each assumes it's the sole C being loaded - and therefore, do some static/singleton crap that might get clobbered when loaded again).

The nature of the Javascript language makes it possible to sandbox an entire library, ensure that two versions of the same library can run side-by-side without conflicts (because they are both in a different sandbox).

The way sandboxing works is actually not specified by the package manager (npm) nor the language itself (Javascript); each consumer of npm packages can roll their own sandboxing mechanism (webpack, browserify, nodejs, etc.).

There isn't even a common specification for the way packages should export public symbols. You have a choice of CommonJS, AMD, Ecmascript 2015, etc.

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#106

Post by watwut »

thehardsphere wrote: post #82
chii wrote: post #74

Earlier quoted context omitted.

Java ecosystem has just as many libraries, and yet you don't see this problem. Why is that? Maven Central is just as available to add to (last i checked, all you need is a public GPG key registered to the MIT public gpg server).

It's not just as available to add to; you can only add to Central through one of three approved hosts unless your project has a special exception. And those hosts all involve an actual review of your artifact, its signature, and your POM. I've never done it, but it at least sounds like there's a process where you need to convince 3 or more people that adding your package is a good idea and will not hurt security. Tha…

There is only one person and you are not convincing the person much about security nor have to go through any kind of difficult vetting. Just formal stuff about packege naming, properly filled pom.xml and such.

However, you have to sign everything with pgp including updates and that is verified. You also have to own the domain with same path as your packages - meaning name space is larger and name clashes less likely. They actually check this and won't release unless you host project. Which explains why java open source tend to use packages like com.github.my_account.my_project

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#107

Post by buckminster »

userbinator wrote: post #18

The IP address it phones home to, 121.42.217.44, is located in China and visiting it with HTTP just displays this interesting message: Hi bro :) Welcome Here! Leave Messages via HTTP Log Please :) On 2017-09-16: Happy to see somebody find it ! :) Just curious about how long it would take for people to find those 'bad' packages As you see, that's just a toy script, no harm, hope you enjoy it ! It looks like someone (s…

What would you expect the page to say if they were malicious?

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#108

Post by Stranger43 »

ageofwant wrote: post #76
zizek23 wrote: post #17

The casual culture of pulling in hundred of dependencies, and mushrooming language specific package managers is ridiculously insecure and has to go. There is no way for anyone to know what all this code is doing, there is little way to verify updates and its simply untenable. If some developers like this sort of unsafe practice it should be strictly limited to their machines and in no way make it across in any form a…

Bulshit. The benefits of this "casual culture of pulling in hundred of dependencies" vastly outweigh the harm. I'll be assembling another $6000 job that will take me maybe 12 hours to complete while you write your compiler, from scratch, in your own assembler, made for your own cpu, that you youre gonna cook up from a bucket of sand you collected yourself, from a sandpit you trust.

the problem is that your $6000,- one off will cause your client to face either completely trivial to exploit production systems or exponentially growing operating costs, because lets face it you are going to provide zero hours of post deployment support for that task.

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#109

Post by xamuel »

namelost wrote: post #64
zizek23 wrote: post #23

Earlier quoted context omitted.

A package manager must at a minimum address code security, dependency hell and deployment. A json parser that wgets urls into a folder is not a package manager. These things must be thought of upfront and cannot be left vague and open to adhoc practices that leak to end users and deployment creating insecurity. For instance a package should be a package, it should not be a simple class or function that in turn pulls…

Who says? npm is proof that package managers need to do none of these things to be successful, and indeed the traditional design of a package manager as a constraint solver is self-defeating, because it puts a perverse pressure on libraries to have as few dependencies as possible and to avoid using the package manager to resolve their dependencies . The reason npm has teething problems is because it's one of the firs…

Who says cancer cells are bad? Look how efficiently they proliferate!

Re: Devs unknowingly use “malicious” modules snuck into official Python repository

#110

Post by Can_Not »

seanp2k2 wrote: post #46
Waterluvian wrote: post #12

Earlier quoted context omitted.

I love Python for this. Big standard library. I hate JavaScript for this... So many weird legacy issues that generally get resolved with libraries. Though ES6 went a long long way.

ES6 does not fix the left-pad issue; there is a propensity to use a package to get one small function vs using a library of common helper functions (which is somewhat what e.g. ES6 helps with...less need for underscore or similar). You pretty much don't need jquery now, but so many things depend on it for convenience / backward compatibility / because it's so insanely battle-tested that you know that it will work or…

> there is a propensity to use a package to get one small function vs using a library of common helper functions

Lodash functions are available in both forms.

Post reply on HN