Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dozens of malicious PyPI packages discovered targeting developers

blog.phylum.io

Post reply on HN
Print view View original

291–300 of 334 posts

Re: Dozens of malicious PyPI packages discovered targeting developers

#291

Post by bigDinosaur »

fabioz wrote: post #86
sigg3 wrote: post #7

It's using base64 encoded strings to deliver the initial stage. Can this be avoided/flagged more easily if by adding a scan of statements featuring base64 or import?

The way I'd go about this is probably starting a VM, installing the package and seeing what in the filesystem is affected by it rather than trying to do static analysis (which becomes a cat and mouse game as detection heuristics improve so do the stealth heuristics). The attack surface area is too big when random python code is executed, which is the case for `setup.py`, but even if there wasn't code executed there,…

Unless you can hide the fact that it's running in a VM, I don't see why code couldn't act normally if it thought it was being analysed like this. Or what about some kind of payload that executes after a long delay, and would become visible for long running programs but not short tests? and so on.

Re: Dozens of malicious PyPI packages discovered targeting developers

#292

Post by pjmlp »

erustemi wrote: post #166

I think a proper way to solve this issue, not specific to python but languages running in a VM in general, would be to have some sort of language support where you specifically define what access rights/ system resources you allow for any given dependency. Example of defining project dependencies: { "apollo-client": { "version": "...", "access": ["fetch"] // only fetch allowed }, "stringutils": { "version": "...", "a…

Java and .NET had it, and in both cases they eventually dropped it, because many security exploits were cause by developers not really understanding how to use them.

In the end they became yet another attack vector, and now everyone should use OS security services instead.

Re: Dozens of malicious PyPI packages discovered targeting developers

#293

Post by jasfi »

louislang wrote: post #28
coffeeblack wrote: post #3

I started to develop only inside VMs, with a full Desktop, IDE, browser etc. inside the virtual machine. There have been to many contaminations of major package repos lately. Only one typo in an import statement up the dependency chain and you’d be compromised.

Full disclosure, I am a co-founder at Phylum. We are actively working on a solution that will fully sandbox package installations for npm, yarn, poetry and others. It's rolled up as part of our core CLI [1], but is totally open source [2]: [1] https://github.com/phylum-dev/cli [2] https://github.com/phylum-dev/birdcage

I think this is a very good idea. It should actually be built into the OS. I know of BSD jails, but not sure what else there is for Linux/Windows/MacOS.

Re: Dozens of malicious PyPI packages discovered targeting developers

#294

Post by lionkor »

erustemi wrote: post #166

I think a proper way to solve this issue, not specific to python but languages running in a VM in general, would be to have some sort of language support where you specifically define what access rights/ system resources you allow for any given dependency. Example of defining project dependencies: { "apollo-client": { "version": "...", "access": ["fetch"] // only fetch allowed }, "stringutils": { "version": "...", "a…

Sort of like BSD's pledge() and similar APIs[1]

[1]: https://man.openbsd.org/pledge.2

Re: Dozens of malicious PyPI packages discovered targeting developers

#295

Post by freeqaz »

djbusby wrote: post #249

Would it be possible to make a more trusted package mirror? Somehow validating packages before inclusion? IIRC mirrors for NPM, Packagist and others is not impossible, can be done for PyPY and others too? Maybe it's a stop-gap before all the fancy permissions feature build out (which seems hard)

It is possible to set your registry in NPM via the "npmrc" file. That will let you hit the specified HTTP server whenever you run commands like "npm install".

I know this is also possible for Python because we did it at Uber. I don't remember the specific details anymore though.

In either case though, a lot of people have written proxies for this use case (I helped write one for NPM at Uber). Companies like Bytesafe and Artifactory also exist in this space.

We're working on something similar that's on GitHub here: https://github.com/lunasec-io/lunasec

Proxy support isn't built out yet but the data is all there already.

Re: Dozens of malicious PyPI packages discovered targeting developers

#297

Post by jart »

dilawar wrote: post #252
rollcat wrote: post #210

Earlier quoted context omitted.

Check out OpenBSD's pledge(2): https://man.openbsd.org/pledge.2 It does exactly that (although on a per-process basis). I don't think this kind of permission system can be retrofitted into an existing language without direct OS support, and probably not at the library level (you'd need something like per-page permissions which would get hairy real fast).

I think @jart has been porting it to Linux https://justine.lol/pledge/ .

Indeed! If your dependencies are able to be command line programs that are shell scripted together, then you can in fact have an access policy on a per-dependency basis, using the pledge.com program linked on my website. So shell scripters rejoice.

But it gets better. If you build Python in the Cosmopolitan Libc repository:

    git clone https://github.com/jart/cosmopolitan
    cd cosmopolitan
    build/bootstrap/make.com -j8 o//third_party/python/python.com
Then you can use cosmo.pledge() directly from Python.

    $ o//third_party/python/python.com
    Python 3.6.14+ (Actually Portable Python) [GCC 9.2.0] on cosmo
    Type "help", "copyright", "credits" or "license" for more information.
    >>: import cosmo, socket
    >>: cosmo.pledge('stdio rpath wpath tty', None)
    >>: print('hi')
    hi
    >>: socket.socket()
    Traceback (most recent call last):
      File "", line 1, in 
      File "/zip/.python/socket.py", line 144, in __init__
        _socket.socket.__init__(self, family, type, proto, fileno)
    PermissionError: [Errno 1] EPERM/1/Operation not permitted
Since we didn't put "inet" in the pledge, you can now be certain your PyPi deps aren't spying on you or uploading your bitcoin wallet to the cloud. You can even use os.fork() to rapidly put each dependency in its own process, then call cosmo.pledge() afterwards to grant each component of your app its own maximally restrictive policy.

Cosmopolitan Python also ports OpenBSD's unveil() system call to Linux too. For example, to disallow all file system access, just call cosmo.unveil(None, None). You need a very recent version of Linux though. For instance, I use unveil() in production on GCE but I had to apt install linux-image-5.18.0-0.deb11.4-cloud-amd64 in order for Landlock LSM to be available to use unveil().

Re: Dozens of malicious PyPI packages discovered targeting developers

#298

Post by kevin_thibedeau »

coredog64 wrote: post #212
dheera wrote: post #202

Earlier quoted context omitted.

pip throws your dependencies in some lib directory either on your system (default if you use sudo), in your home directory (default if you don't use sudo), or inside your virtualenv's lib directory. npm pulls dependencies into node_modules as a subdirectory of your own project as default. Python really should consider doing something similar. Dependencies shouldn't live outside your project folder. We are no longer i…

As of Python 3, pip install into the system Python lib directory is strongly discouraged. ISTR that even using pip to update pip results in a warning. That’s not to say that there’s not still some libs out there that haven’t updated docs to get with the times.

More distros should adopt the Debian practice of installing into dist-packages and leaving site-packages as a /usr/local equivalent for pip to use on it's own.

Re: Dozens of malicious PyPI packages discovered targeting developers

#299

Post by CoolCold »

paulmd wrote: post #277
rollcat wrote: post #218

Earlier quoted context omitted.

> I'd try to keep the permission sets as small and simple as possible though. You've described OpenBSD in general. I recommend a deeper dive - it's fantastically refreshing, how simple yet functional an OS can be.

Same for FreeBSD. Incredibly code-stable and well-documented by Linux standards. I often use the FreeBSD Handbook as an example of first-party documentation done right, and that's only possible because of deliberately limited "churn for churn's sake". The kinds of regular code rot and attrition that Linux suffers just does not take place on BSD systems because if you contribute something new you're expected to make y…

And with all that beauty, FreeBSD is not something I would [nowdays] look into as base OS for hosting my services and products. It means something, probably something about humans.

Re: Dozens of malicious PyPI packages discovered targeting developers

#300

Post by uhhyeahdude »

juancampa wrote: post #260
SahAssar wrote: post #205

Earlier quoted context omitted.

Yes, it applies to the whole process. It's incredibly hard to sandbox dependencies individually since you don't know how your code or other dependencies interact with it. If you want you can run dependencies in a worker process and sandbox that tighter, but that is quite a bit of work.

This is exactly what I've done for Membrane[0]. It's capabilities based, even to get the time (and thus introduce non-determinism) you need a capability. Dependencies run as separate processes and everything is orthogonally persistent. It's a typescript/javascript system for personal automation built entirely within VSCode. Stay tuned, I'll be posting a video this week. [0] https://membrane.io

Hi there! I just wanted to let you know that I read the blog posts, and membrane sounds extremely cool. Ambitious, though. If you don’t mind a small bit of feedback: it would be encouraging to potential users or testers to see some semi-regular posts related to development. It would also be great to see how membrane might work to build a tool using current APIs. I know there is video forthcoming, and perhaps this will be addressed. I didn’t look at the GitHub, where I probably could glean some additional info. But from my perspective, a development blog builds trust and anticipation. It’s also a great way to check your assumptions (or have them checked, rather).

Good luck with the project. I hope it delivers, because I’d love to use it. Signed up for the mailing list.

Post reply on HN