Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Self-contained highly-portable Python distributions

gregoryszorc.com

Post reply on HN
Print view View original

21–30 of 45 posts

Re: Self-contained highly-portable Python distributions

#21

Post by rented_mule »

0cf8612b2e1e wrote: post #19

I keep meaning to explore compiling Python +libs to WASM for running in a desktop environment. It is already Python, so I am willing to accommodate enormous performance losses. I just want to package up the code into something more straightforward than PyInstaller.

JavaScript has module bundlers that will merge all your code modules into one module for deployment. It's a little surprising nobody (that I'm aware of) has done this for Python and PEP 723 dependency specifications. Then the single binary for uv is all that's required to run the bundle using this type of approach, at least on unix-ish machines (not sure if there's a trick to make this work on Windows): https://docs.astral.sh/uv/guides/scripts/#using-a-shebang-to...

Re: Self-contained highly-portable Python distributions

#22

Post by 0cf8612b2e1e »

rented_mule wrote: post #21
0cf8612b2e1e wrote: post #19

I keep meaning to explore compiling Python +libs to WASM for running in a desktop environment. It is already Python, so I am willing to accommodate enormous performance losses. I just want to package up the code into something more straightforward than PyInstaller.

JavaScript has module bundlers that will merge all your code modules into one module for deployment. It's a little surprising nobody (that I'm aware of) has done this for Python and PEP 723 dependency specifications. Then the single binary for uv is all that's required to run the bundle using this type of approach, at least on unix-ish machines (not sure if there's a trick to make this work on Windows): https://docs.…

That’s not a bad idea. uvx feels like a temporary hack on which I can’t rely for a permanent deployment, but it is all probably fine. I would want to at least materialize a real environment so I know that it is not getting randomly deleted as cache/temp gets cleared.

Unfortunately, it’s still not my ideal use case of distributing executables to Windows machines. I really want a single artifact users could freely share without any pre-configuration. I guess I could have a batch/vbs script which would bootstrap uv plus the rest of the machinery, but that is more moving parts than I would like.

Re: Self-contained highly-portable Python distributions

#24

Post by ameliaquining »

charliermarsh wrote: post #6

These are the Python distributions we use in uv ( https://github.com/astral-sh/uv ), i.e., when you install Python with uv, you're installing from python-build-standalone. (Same goes for many of the other tools that can install Python for you, like pipx, Hatch, Poetry, Bazel, etc.) Most of our engineering time on the project over the past ~year and a half has been split into three buckets: 1. Keeping up with upstream…

> When you download Python from http://python.org (on Linux or macOS), what you're actually downloading is an installer that builds Python from source on your machine.

> The net effect is that on Linux and macOS, you can't "download a Python binary" from... anywhere. Other than the python-build-standalone project.

Are you sure this is right about macOS? I just had a look inside the macOS installer from python.org and there are definitely some binaries in there.

Re: Self-contained highly-portable Python distributions

#25

Post by fragmede »

0cf8612b2e1e wrote: post #22
rented_mule wrote: post #21

Earlier quoted context omitted.

JavaScript has module bundlers that will merge all your code modules into one module for deployment. It's a little surprising nobody (that I'm aware of) has done this for Python and PEP 723 dependency specifications. Then the single binary for uv is all that's required to run the bundle using this type of approach, at least on unix-ish machines (not sure if there's a trick to make this work on Windows): https://docs.…

That’s not a bad idea. uvx feels like a temporary hack on which I can’t rely for a permanent deployment, but it is all probably fine. I would want to at least materialize a real environment so I know that it is not getting randomly deleted as cache/temp gets cleared. Unfortunately, it’s still not my ideal use case of distributing executables to Windows machines. I really want a single artifact users could freely shar…

pyexe and pyinstaller let you just ship an exe to run

Re: Self-contained highly-portable Python distributions

#26

Post by chrisweekly »

brewmarche wrote: post #14
charliermarsh wrote: post #6

These are the Python distributions we use in uv ( https://github.com/astral-sh/uv ), i.e., when you install Python with uv, you're installing from python-build-standalone. (Same goes for many of the other tools that can install Python for you, like pipx, Hatch, Poetry, Bazel, etc.) Most of our engineering time on the project over the past ~year and a half has been split into three buckets: 1. Keeping up with upstream…

I install these Python builds into distroless containers with uv python install and it just works. (If you try this with Google’s images, use gcr.io/distroless/cc, libgcc/libstdc++ is needed for some extensions, like numpy)

Pretty sure a similar approach should work using smolvms from smolmachines.com

Re: Self-contained highly-portable Python distributions

#27

Post by js2 »

ameliaquining wrote: post #24
charliermarsh wrote: post #6

These are the Python distributions we use in uv ( https://github.com/astral-sh/uv ), i.e., when you install Python with uv, you're installing from python-build-standalone. (Same goes for many of the other tools that can install Python for you, like pipx, Hatch, Poetry, Bazel, etc.) Most of our engineering time on the project over the past ~year and a half has been split into three buckets: 1. Keeping up with upstream…

> When you download Python from http://python.org (on Linux or macOS), what you're actually downloading is an installer that builds Python from source on your machine. > The net effect is that on Linux and macOS, you can't "download a Python binary" from... anywhere. Other than the python-build-standalone project. Are you sure this is right about macOS? I just had a look inside the macOS installer from python.org and…

It's definitely wrong for macOS. The python.org macOS download has been an installer package that installs Python to /Applications as long as I can remember.

(That said, I've rarely used it, preferring either the macOS/Xcode-provided Python or these self-contained Pythons which I use via uv.)

Re: Self-contained highly-portable Python distributions

#28

Post by jdxcode »

charliermarsh wrote: post #6

These are the Python distributions we use in uv ( https://github.com/astral-sh/uv ), i.e., when you install Python with uv, you're installing from python-build-standalone. (Same goes for many of the other tools that can install Python for you, like pipx, Hatch, Poetry, Bazel, etc.) Most of our engineering time on the project over the past ~year and a half has been split into three buckets: 1. Keeping up with upstream…

mise uses these as well and adopting them has been a massive improvement in the user experience for my python users. y'all have been doing a fantastic job maintaining them.

i'm also doing the same for ruby—maintaining the binaries myself which is becoming the default in mise next week: https://github.com/jdx/ruby

i'm a firm believer that pre-compiled, portable binaries are the best way to distribute runtimes and clis.

Re: Self-contained highly-portable Python distributions

#29

Post by jdxcode »

ameliaquining wrote: post #24
charliermarsh wrote: post #6

These are the Python distributions we use in uv ( https://github.com/astral-sh/uv ), i.e., when you install Python with uv, you're installing from python-build-standalone. (Same goes for many of the other tools that can install Python for you, like pipx, Hatch, Poetry, Bazel, etc.) Most of our engineering time on the project over the past ~year and a half has been split into three buckets: 1. Keeping up with upstream…

> When you download Python from http://python.org (on Linux or macOS), what you're actually downloading is an installer that builds Python from source on your machine. > The net effect is that on Linux and macOS, you can't "download a Python binary" from... anywhere. Other than the python-build-standalone project. Are you sure this is right about macOS? I just had a look inside the macOS installer from python.org and…

i think that sentence is taken out of context a bit, the problem with those pkg binaries is that they're not portable—which the post describes

EDIT: actually no, I think charlie is wrong on that point, he said the installers compile python but that only happens with python-build-based installers like pyenv. still, the fact those binaries lack portability is the real problem.

Re: Self-contained highly-portable Python distributions

#30

Post by hack1312 »

ameliaquining wrote: post #24
charliermarsh wrote: post #6

These are the Python distributions we use in uv ( https://github.com/astral-sh/uv ), i.e., when you install Python with uv, you're installing from python-build-standalone. (Same goes for many of the other tools that can install Python for you, like pipx, Hatch, Poetry, Bazel, etc.) Most of our engineering time on the project over the past ~year and a half has been split into three buckets: 1. Keeping up with upstream…

> When you download Python from http://python.org (on Linux or macOS), what you're actually downloading is an installer that builds Python from source on your machine. > The net effect is that on Linux and macOS, you can't "download a Python binary" from... anywhere. Other than the python-build-standalone project. Are you sure this is right about macOS? I just had a look inside the macOS installer from python.org and…

It’s incorrect for Linux too, distributions build Python binaries for their repos.
Post reply on HN