Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PyPI Blog: Project Quarantine

blog.pypi.org

Post reply on HN
Print view View original

31–40 of 62 posts

Re: PyPI Blog: Project Quarantine

#31

Post by woodruffw »

xgstation wrote: post #29
woodruffw wrote: post #18

Earlier quoted context omitted.

It's not good, but it should also not be baffling: it's the exact same thing other ecosystems do (npm with install hooks/scripts, Rust with build.rs, Ruby with gemspecs, etc).

I know other ecosystems do the same and those are baffling too, especially for the newer created languages like rust, which is why https://internals.rust-lang.org/t/pre-rfc-sandboxed-determin... exists

Sandboxing is a great idea. But the fact that this is a near-universal feature of language packaging reveals a preference that's going to be hard to counter: users do want effectively-arbitrary system access at build time, because that's the paradigm that's supported by the million-and-one different ways in which a build environment can be valid.

Re: PyPI Blog: Project Quarantine

#32

Post by woodruffw »

IshKebab wrote: post #30
woodruffw wrote: post #17

Earlier quoted context omitted.

Who is "they"? PyPI is an index; it doesn't control your installing client. (This is a larger issue - or feature, depending on your perspective - with Python packaging. But it's important to understand that PyPI itself can't force `pip` or any other client to pick any particular resolution order between indices.)

> Who is "they"? The PyPI and Pip developers of course.

Those are largely disjoint sets, and the post in question is about PyPI.

Re: PyPI Blog: Project Quarantine

#33

Post by f1shy »

woodruffw wrote: post #18
xgstation wrote: post #11

the fact that `pip install` just runs whatever is in `setup.py` is still mind baffling, even if the author weren't mallicious the `setup.py` can still do harm (say delete a file by mistake), there really needs to be an official way of sandbox its running.

It's not good, but it should also not be baffling: it's the exact same thing other ecosystems do (npm with install hooks/scripts, Rust with build.rs, Ruby with gemspecs, etc).

Notably also common lisp (quicklisp)

Re: PyPI Blog: Project Quarantine

#34

Post by spencerchubb »

davidshepherd7 wrote: post #12
spencerchubb wrote: post #6

Earlier quoted context omitted.

companies that actually care about security have a more secure solution and don't allow devs to use pypi

Could you give some examples of more secure solutions?

jfrog is the one my company uses

Re: PyPI Blog: Project Quarantine

#35

Post by spencerchubb »

cjalmeida wrote: post #7
spencerchubb wrote: post #6

Earlier quoted context omitted.

companies that actually care about security have a more secure solution and don't allow devs to use pypi

You’d be surprised by the amount of companies handling critical infrastructure that are OK with using PyPI directly

really depends on the company. my company cares a lot about security because it's a huge fortune 50 company with sensitive data and a lot of reputation could be lost with a security scandal

Re: PyPI Blog: Project Quarantine

#36

Post by zbentley »

spencerchubb wrote: post #34
davidshepherd7 wrote: post #12

Earlier quoted context omitted.

Could you give some examples of more secure solutions?

jfrog is the one my company uses

How do you decide what externally available packages to store/cache in artifactory?

I’m curious, as I also deal with this tension. What (human and automated) processes do you have for the following scenarios?

1. Application developer wants to test (locally or in a development environment) and then use a net new third party package in their application at runtime.

2. Application developer wants to bump the version used of an existing application dependency.

3. Application developer wants to experiment with a large list of several third party dependencies in their application CI system (e.g. build tools) or a pre-production environment. The experimentation may or may not yield a smaller set of packages that they want to permanently incorporate into the application or CI system.

How, if at all, do you go about giving developers access via jfrog to the packages they need for those scenarios? Is it as simple as “you can pull anything you want, so long as X-ray scans it”, or is there some other process needed to get a package mirrored for developer use?

Re: PyPI Blog: Project Quarantine

#37

Post by LtWorf »

woodruffw wrote: post #25
LtWorf wrote: post #23

Earlier quoted context omitted.

For all intents and purposes "pip" is the official client. It is referenced in the official documentation https://docs.python.org/3/installing/index.html

The fact that pip is the official client isn’t in dispute. The point was that pip and PyPI are different entities, per a larger pattern of devolved ownership/control/standards-over-tools in Python packaging. PyPI has little to no say over how pip and other tools choose to handle resolutions across multiple indices.

The PSF has a saying in which is the default installer and how pypi is run.

Re: PyPI Blog: Project Quarantine

#38

Post by pyuser583 »

LtWorf wrote: post #37
woodruffw wrote: post #25

Earlier quoted context omitted.

The fact that pip is the official client isn’t in dispute. The point was that pip and PyPI are different entities, per a larger pattern of devolved ownership/control/standards-over-tools in Python packaging. PyPI has little to no say over how pip and other tools choose to handle resolutions across multiple indices.

The PSF has a saying in which is the default installer and how pypi is run.

PSF has little control over anything. The Python ecosystem is consensus-based.

Re: PyPI Blog: Project Quarantine

#39

Post by NeutralCrane »

LtWorf wrote: post #21
openrisk wrote: post #20

Its always an interesting dynamic: assuming a high trust society pays dividends - Python would be nowhere close the success it has been without PyPI. But then success attracts trust abusers and forces raising the fences (which comes with higher costs, both direct and indirect). Direct costs in the people and infrastructure that must be dedicated to the task. Indirect costs in the frictions generated by complicating w…

They won't pay anything unless they are forced to do so. Basic capitalism brings to externalise costs to society

Perhaps, but can you explain how an alternative to capitalism wouldn’t result in people no paying for a service they don’t have to pay for?

Re: PyPI Blog: Project Quarantine

#40

Post by CableNinja »

zbentley wrote: post #36
spencerchubb wrote: post #34

Earlier quoted context omitted.

jfrog is the one my company uses

How do you decide what externally available packages to store/cache in artifactory? I’m curious, as I also deal with this tension. What (human and automated) processes do you have for the following scenarios? 1. Application developer wants to test (locally or in a development environment) and then use a net new third party package in their application at runtime. 2. Application developer wants to bump the version use…

Where i am, every package repo - docker, pypi, rpm, deb, npm, and more - all go through artifactory and are scanned. Packages are autopulled into artifactory when a user requests the package and scanned by xray. Artifactory has a remote pull through process that downloads once from the remote, and then never again unless you nuke the content. Vulnerable packages must have exceptions made in order to get used. Sadly, we put the burden of allowances on the person requesting the package, but it at least makes them stop and think before they approve it. Granting access to new external repos is easy, and we make requesting them painfree, just making sure that we enable xray. Artifactory also supports local repos so users can upload their packages and pull them down later.
Post reply on HN