Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PYX: The next step in Python packaging

astral.sh

Post reply on HN
Print view View original

281–290 of 479 posts

Re: PYX: The next step in Python packaging

#281

Post by alisonatwork »

zanie wrote: post #262
alisonatwork wrote: post #218

Earlier quoted context omitted.

I recently did the same at work, just converted all our pip stuff to use uv pip but otherwise no changes to the venv/requirements.txt workflow and everything just got much faster - it's a no-brainer. But the increased resource usage is real. Now around 10% of our builds get OOM killed because the build container isn't provisioned big enough to handle uv's excessive memory usage. I've considered reducing the number of…

Please open an issue with some details about the memory usage. We're happy to investigate and feedback on how it's working in production is always helpful. (I work on uv)

Last time I looked into this I found this unresolved issue, which is pretty much the same thing: https://github.com/astral-sh/uv/issues/7004

We run on-prem k8s and do the pip install stage in a 2CPU/4GB Gitlab runner, which feels like it should be sufficient for the uv:python3.12-bookworm image. We have about 100 deps that aside from numpy/pandas/pyarrow are pretty lightweight. No GPU stuff. I tried 2CPU/8GB runners but it still OOMed occasionally so didn't seem worth using up those resources for the normal case. I don't know enough about the uv internals to understand why it's so expensive, but it feels counter-intuitive because the whole venv is "only" around 500MB.

Re: PYX: The next step in Python packaging

#282

Post by hulitu »

Eduard wrote: post #224
bastawhiz wrote: post #204

Earlier quoted context omitted.

Well I started with pip because it's what I was told to use. But it was slow and had footguns. And then I started using virtualenv, but that only solved part of the problem. So I switched to conda, which sometimes worked but wrecked my shell profile and often leads to things mysteriously using the wrong version of a package. So someone told me to use pipenv, which was great until it was abandoned and picked up by som…

I started with "sudo apt install python" a long time ago and this installed python2. This was during the decades-long transition from python2 to python3, so half the programs didn't work so I installed python3 via "sudo apt install python3". Of course now I had to switch between python2 and python3 depending on the program I wanted to run, that's why Debian/Ubuntu had "sudo update-alternatives --config python" for ma…

Your comment shows the sad state of software quality those days. Rust is the same, move fast and break things. And lately also Mesa started to suffer from the same disease. You basically need, those days, the same build env like the one on the developer's machine or the build will fail.

Re: PYX: The next step in Python packaging

#283

Post by joshdavham »

megamix wrote: post #266

No Thanks. For majority of my use case pip is just fine. I’m not here to chase time just to live life

This is a low quality comment.

From the guidelines [1]

> Please don't post shallow dismissals, especially of other people's work.

[1] https://news.ycombinator.com/newsguidelines.html

Re: PYX: The next step in Python packaging

#284

Post by aidenn0 »

dakiol wrote: post #39

I'm brushing up with Python for a new job, and boy what a ride. Not because of the language itself but the tooling around packages. I'm coming from Go and TS/JS and while these two ecosystems have their own pros and cons, at least they are more or less straightforward to get onboarded (there are 1 or 2 tools you need to know about). In Python there are dozens of tools/concepts related to packaging: pip, easy_install,…

I don't know; I was looking at TS tutorials the other day and there seemed to be at least half a dozen "bundlers" with different tutorails suggesting different ones to use. It took me a while to figure out I could just directly invoke "tsc" to generate javascript from typescript.

Re: PYX: The next step in Python packaging

#285

Post by nurettin »

nickserv wrote: post #60
nurettin wrote: post #53

Earlier quoted context omitted.

Have you tried JRuby? It might be a bit too large for your droplet, but it has the java versions of most gems and you can produce cross-platform jars using warbler.

The speed of Ruby with the memory management of Java, what's not to love? Also, now you have two problems.

For me, it solves a very specific problem.

In-house app that uses jdbc, is easy to develop and needs to be cross-platform (windows, linux, aix, as400). The speed picks up as it runs, usually handling 3000-5000 eps over UDP on decade old hardware.

Re: PYX: The next step in Python packaging

#286

Post by imp0cat »

zzzeek wrote: post #239
typpilol wrote: post #228

Earlier quoted context omitted.

Installing packages it the most joy you've ever gotten outta tech? Not a project you built, or something you're proud of? Installing packages?

I know it's like everyone's lost their mind right ?

Nope. You just haven't wrestled with python packages for long enough to appreciate the change.

Re: PYX: The next step in Python packaging

#287

Post by brettpro »

Welp, I guess it's time to start pulling all the uv deps out of our builds and enjoy the extra 5 minutes of calm per deploy. I'm not gonna do another VC-fueled supply chain poisoning switcheroo under duress of someone else's time crunch to start churning profit.

Re: PYX: The next step in Python packaging

#288

Post by joshvm »

integralid wrote: post #280
bastawhiz wrote: post #204

Earlier quoted context omitted.

Well I started with pip because it's what I was told to use. But it was slow and had footguns. And then I started using virtualenv, but that only solved part of the problem. So I switched to conda, which sometimes worked but wrecked my shell profile and often leads to things mysteriously using the wrong version of a package. So someone told me to use pipenv, which was great until it was abandoned and picked up by som…

What's wrong with just using virtualenv. I never used anything else, and I never felt the need to. Maybe it's not as shipping l shiny as the other tools, but it just works.

Nothing is inherently wrong with virtualenv. All these tools make virtual environments and offer some way to manage them. But virtualenv doesn't solve the problem of dependency management.

Re: PYX: The next step in Python packaging

#289

Post by frollogaston »

rs186 wrote: post #252
bastawhiz wrote: post #214

Earlier quoted context omitted.

I'd otherwise agree but this problem seems unique to Python. I don't have problems like this with npm or composer or rubygems. Or at least very infrequently. It's almost every time I need to update dependencies or install on a new machine that the Python ecosystem decides I'm not worthy.

I think pip made some poor design choices very early, but pip stuck around for a long time and people kept using it. Of course things got out of control, then people kept inventing new package management until uv comes along. I don't know enough about Python to understand how people could live with that for so long.

Every big Python repo has a Dockerfile, which is much less common in JS.
Post reply on HN