271–280 of 334 posts
Earlier quoted context omitted.
What if the package is signed by a key available at google.com/pypi/key ? Actually it should be not just a key but a whole TLS certificate, with references to a CA, activity dates, etc.
How would you know where you’d find that key?
I think a proper way to solve this issue, not specific to python but languages running in a VM in general, would be to have some sort of language support where you specifically define what access rights/ system resources you allow for any given dependency. Example of defining project dependencies: { "apollo-client": { "version": "...", "access": ["fetch"] // only fetch allowed }, "stringutils": { "version": "...", "a…
Is there a way to check if you've been compromised by these PyPl packages? Does PyPl have a mechanism to let people know that they've downloaded a compromised package?
Earlier quoted context omitted.
Have you seen how much space a virtualenv uses? It can easily be >1 GB. For every project, this adds up. (Not to mention the bandwidth, which is not always plentiful).
4TB hard drives are $300 these days.
This type of stuff is one reason I like vendoring all my deps in golang. You have to be very explicit about updating dependencies, which can be a big hassle, but you're required to do a git commit of all the changes, which gives you a good time to actually browse through the diffs. If you update dependencies incrementally, it's not even that big a job. Of course, this doesn't guarantee I won't miss any malicious code…
It also blows up the size of your git checkouts pretty fast though. I don't think you really gain much either; vendoring was useful before modules, but now we have modules and go.sum I don't really see the advantage. If you have "github.com/foo/bar" specified at version 1.0.4 the go.sum will ensure you have EXACTLY that version or it will issue an error in case of any tomfoolery.
Going on a trip somewhere without an Internet connection? Checkout the repo on your laptop and go. Without vendoring: oh shoot, I forgot to download the deps, I guess I’m going to be forced into a work-life balance. With vendoring: no additional step needed after checking out the repo. The repo has everything you need to work.
Another case: repo of your dependency is removed, or force-pushed to overwriting history. You’ve lost the ability to build your project, and need to either find another source for your dependency, or rewrite it. With vendoring: everything still works, you don’t even notice the dep repo went under.
Generally, with vendoring your code is in just one place instead of being a distributed being which crumbles when any part of it gets sick.
Moreover, relying on checksums to me seems a bit overcomplicated. It’s like going to a pub and giving each drink from a stranger to a chemist for verification to make sure they didn’t slip any pills, when you could just carry your own drink around and cover the top with your hand.
I started to develop only inside VMs, with a full Desktop, IDE, browser etc. inside the virtual machine. There have been to many contaminations of major package repos lately. Only one typo in an import statement up the dependency chain and you’d be compromised.
With modern virtio interfaces for network, disk and graphics practically giving near metal performance, there's no reason to not utilize VMs for development.
Earlier quoted context omitted.
I like this. I'd try to keep the permission sets as small, limited, and simple as possible though.
> I'd try to keep the permission sets as small and simple as possible though. You've described OpenBSD in general. I recommend a deeper dive - it's fantastically refreshing, how simple yet functional an OS can be.
I often use the FreeBSD Handbook as an example of first-party documentation done right, and that's only possible because of deliberately limited "churn for churn's sake". The kinds of regular code rot and attrition that Linux suffers just does not take place on BSD systems because if you contribute something new you're expected to make your thing mesh with the ecosystem, they don't tolerate the "I'm gonna churn everyone's environments because my version is 10% better than the existing one" that tends to take place on Linux.
As an example, the amount of init systems that Ubuntu has gone through over the last 20 years is completely insane by BSD standards. They've gone from sysvinit to upstart to systemd. You don't have people ripping out the graphics and audio subsystems and doing total rewrites, either. It's nuts that a lot of Linux people don't even realize that it doesn't have to be that way - that's not "just how maintenance is supposed to work", that's a bunch of incredibly bad behavior on the parts of distros and Poettering specifically that has become highly normalized and overlooked. Maintenance shouldn't be breaking things like that. "we never break userland" doesn't have to be a suicide pact either - BSD has actually maintained a much more stable ABI than the linux kernel without crystallizing a bunch of shitty bug-compatibility stuff like Linux is doing either. It's all insanely stable and competent and professional by linux standards.
If I was developing a product for true long-term support, with the minimum possible engineering effort devoted to fighting churn, BSD would be at the top of the list.
Earlier quoted context omitted.
It also blows up the size of your git checkouts pretty fast though. I don't think you really gain much either; vendoring was useful before modules, but now we have modules and go.sum I don't really see the advantage. If you have "github.com/foo/bar" specified at version 1.0.4 the go.sum will ensure you have EXACTLY that version or it will issue an error in case of any tomfoolery.
Vendoring also means your builds don’t need an Internet connection. Going on a trip somewhere without an Internet connection? Checkout the repo on your laptop and go. Without vendoring: oh shoot, I forgot to download the deps, I guess I’m going to be forced into a work-life balance. With vendoring: no additional step needed after checking out the repo. The repo has everything you need to work. Another case: repo of y…
> Another case: repo of your dependency is removed, or force-pushed to overwriting history. You’ve lost the ability to build your project, and need to either find another source for your dependency, or rewrite it.
The GOPROXY (https://proxy.golang.org/) still contains that removed repo, and since everything is summed people can't just force overwrite it. Plus, you still have it in the module cache locally.
You can of course always come up with "but what if...?" scenarios where any of the above fails, and all sort of things can happen, but they're also not especially likely to happen. So the question isn't "is it useful in some scenario?" but rather "is it worth the extra effort?"
> Moreover, relying on checksums to me seems a bit overcomplicated.
It's built-in, so no extra complications needed.
In a previous HN discussion on the topic of rogue Python packages, readers had suggested bubblewrap and firejail for sandboxing. They limit the access a script and its packages have to your filesystem and network. I think that's the better approach - just assume all packages are malicious by default. Can't rely on scanners because of the large number of packages and attacks.
Another options is "nsjail". I landed on it having considered at firejail, apparmour, selinux, bubblewrap.
I'm very frustrated with firejail since I can't for example block execution in my home directory, with the exception of one subdirectory.
It just can't be done.