Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

WebAssembly: Adding Python support to WASM language runtimes

wasmlabs.dev

Post reply on HN
Print view View original

21–30 of 88 posts

Re: WebAssembly: Adding Python support to WASM language runtimes

#21

Post by seddonm1 »

robertlagrant wrote: post #12

The non-Docker version seems to require an external site-packages, unless I missed it. Is it possible to produce a single wasm binary with all dependencies compiled in?

I have been following and playing with this repository: https://github.com/singlestore-labs/python-wasi/

It builds a single Python WASM module with all dependencies included (they use VFS) and a Dockerfile to make the process easy (and actually worked first go). It does produce large files though: wasi-python3.11.wasm 110MB

Re: WebAssembly: Adding Python support to WASM language runtimes

#22

Post by mjw1007 »

claytonjy wrote: post #20
mjw1007 wrote: post #18

Earlier quoted context omitted.

I've come across this notion that nowadays machine learning provides (in some sense) the biggest group of Python users a few times recently. What reason is there to suppose this is true? It seems surprising to me.

It's really hard to do much ML in anything _except_ python. Virtually everyone improving the ML ecosystems of other language got their start in Python and are knowingly competing with Python (e.g. R, Julia). If you want to get started in ML today, python is the obvious easiest path forward. So, most ML users are python users. I don't know how that group compares to non-ML python users, but I have a feeling there isn'…

I found a recentish (2021) survey at [1] which suggests that in 2021 ML was some way behind web development, sysadmin stuff, and data analysis among Python users (and didn't seem to be on the way up the list).

[1] https://lp.jetbrains.com/python-developers-survey-2021/#Gene...

Re: WebAssembly: Adding Python support to WASM language runtimes

#23

Post by ElectricalUnion »

angelmm wrote: post #13
c120 wrote: post #4

So for someone who has python installed locally, what's the point? Is it just the sandbox or is there anything else I'm missing?

You get an extra layer of isolation, even at your development environment level. I remember a NodeJs CVE that was caused by a poisoned dependency. It was affecting people when downloading it from npm. There’s still a gap here to cover, but the benefits may be worth :)

I don't see how this would in any way prevent you from being affected by a equivalent poisoned pypi dependency; after all your secrets/credentials are inside the sandbox anyways or your code can't work.

Re: WebAssembly: Adding Python support to WASM language runtimes

#24

Post by claytonjy »

mjw1007 wrote: post #22
claytonjy wrote: post #20

Earlier quoted context omitted.

It's really hard to do much ML in anything _except_ python. Virtually everyone improving the ML ecosystems of other language got their start in Python and are knowingly competing with Python (e.g. R, Julia). If you want to get started in ML today, python is the obvious easiest path forward. So, most ML users are python users. I don't know how that group compares to non-ML python users, but I have a feeling there isn'…

I found a recentish (2021) survey at [1] which suggests that in 2021 ML was some way behind web development, sysadmin stuff, and data analysis among Python users (and didn't seem to be on the way up the list). [1] https://lp.jetbrains.com/python-developers-survey-2021/#Gene...

Great source; looks like I've quite underestimated the python-web-dev crowd's size.

I'm curious what the longer-term trends look like; not much change between consecutive years.

Data analysis is basically a pre-requisite for ML, so the combined "data stuff" usage is quite a lot bigger than web dev usage!

Re: WebAssembly: Adding Python support to WASM language runtimes

#25

Post by simonw »

callahad wrote: post #15
simonw wrote: post #3

This looks very promising! The thing I most want to solve right now is this: I want to write a regular Python application that can safely execute untrusted Python code in a WASM sandbox as part of its execution. I want to do this so I can let end users customize my web applications in weird and interesting ways by pasting their own Python code into a textarea - think features like "run this Python code to transform m…

The startup I'm working at is basically trying to do exactly that as a service, but a one-off thing for a regular Python application shouldn't be as hard to figure out as it is. Can you link to the Mastodon thread (darn lack of search!) and we can continue there?

Here's the Mastodon conversation: https://fedi.simonwillison.net/@simon/109682777068881522

(I'm so close to building my own search engine just against my own content there.)

Re: WebAssembly: Adding Python support to WASM language runtimes

#26

Post by simonw »

mritchie712 wrote: post #16
simonw wrote: post #3

This looks very promising! The thing I most want to solve right now is this: I want to write a regular Python application that can safely execute untrusted Python code in a WASM sandbox as part of its execution. I want to do this so I can let end users customize my web applications in weird and interesting ways by pasting their own Python code into a textarea - think features like "run this Python code to transform m…

Have you tried to do it with pyodide? What issues did you hit using that?

Pyodide isn't currently supported outside of browsers, though that might change: https://github.com/pyodide/pyodide/issues/869

Either way, I couldn't figure out how to do the above sequence of steps with any of the available Python WASM runtimes - they're all very under-documented at the moment, sadly. I tried all three of these:

- https://github.com/wasmerio/wasmer-python

- https://github.com/bytecodealliance/wasmtime-py

- https://github.com/wasm3/pywasm3

Re: WebAssembly: Adding Python support to WASM language runtimes

#28

Post by phickey »

simonw wrote: post #3

This looks very promising! The thing I most want to solve right now is this: I want to write a regular Python application that can safely execute untrusted Python code in a WASM sandbox as part of its execution. I want to do this so I can let end users customize my web applications in weird and interesting ways by pasting their own Python code into a textarea - think features like "run this Python code to transform m…

Wasmtime's `wasmtime-py` embedding in python has support for Wasm Components: https://github.com/bytecodealliance/wasmtime-py#components (disclosure, I helped create it)

The remaining piece of the puzzle would be to create a wit-bindgen guest generator https://github.com/bytecodealliance/wit-bindgen#guests for this build of the python interpreter. You could then seamlessly call back and forth between the host and guest pythons, without even knowing that wasmtime is under the hood.

Re: WebAssembly: Adding Python support to WASM language runtimes

#29

Post by _visgean »

mjw1007 wrote: post #18
brrrrrm wrote: post #5

the issue right now with Python support in WASM (at least for machine learning, the main driver of the language) is that Python is largely a wrapper language and none the utilities that make it so powerful (numpy, PyTorch, JAX) work particularly well in wasm, since it's so limited performance-wise (no FMA, no GPU support). I'm excited for pairing wasm with WebGPU, which will likely unblock these projects from buildin…

I've come across this notion that nowadays machine learning provides (in some sense) the biggest group of Python users a few times recently. What reason is there to suppose this is true? It seems surprising to me.

> What reason is there to suppose this is true? It seems surprising to me.

One reason is its just super easy for input output operations. ML is all about data and getting the data to the right place is really easy in python compared to some other languages..

Re: WebAssembly: Adding Python support to WASM language runtimes

#30

Post by amelius »

AshleysBrain wrote: post #27

How does this handle garbage collection? AFAIK the WebAssembly GC proposal is still in development. Does it implement GC in WASM code?

Perhaps it just uses Python's built-in garbage collector that just increases/decreases the data segment size as needed by calling sbrk()?
Post reply on HN