The Python scientific stack, compiled to WebAssembly • Hacker News Board

Re: The Python scientific stack, compiled to WebAssembly

#41

Post by mdboom » Fri, Aug 24, 2018, 6:40 PM UTC

m_mueller wrote: post #5

xtrapolate wrote: post #4

Earlier quoted context omitted.

> "Part of an effort to make a Jupytr notebook-like environment without relying on server side code." So there won't be any server side code, you'd just be downloading the entire Python stack and dependencies every time you want to use this through your browser. This time it will be WASM though, instead of x86/x64/pyc. That we can, doesn't mean that we should.

I guess we need an in-browser package management like npm?

The plan is to build something based on conda.

Re: The Python scientific stack, compiled to WebAssembly

#42

Post by mdboom » Fri, Aug 24, 2018, 6:41 PM UTC

X6S1x6Okd1st wrote: post #40

Huh I am getting ``` TypeError: window[s] is undefined ``` with firefox.

File a bug over at https://github.com/iodide-project/pyodide/

I'd love to get to the bottom of that.

Re: The Python scientific stack, compiled to WebAssembly

#43

Post by mdboom » Fri, Aug 24, 2018, 6:43 PM UTC

Scarblac wrote: post #12

nerdponx wrote: post #11

What happens to all of the C code in Numpy, and the calls to external libraries like BLAS?

In particular, does this mean Javascript has access to Numpy? Sounds unbelievable, but would be extremely cool.

Yep. You can "import" python objects over to the Javascript side and start using them from there. It's not as pleasant as working with Numpy in Python, of course, due to lack of operator overloading, of course.

Re: The Python scientific stack, compiled to WebAssembly

#44

Post by mdboom » Fri, Aug 24, 2018, 6:44 PM UTC

gh02t wrote: post #25

mjs2600 wrote: post #16

Earlier quoted context omitted.

C can compile to WASM, but I have no clue about external tools like BLAS. I'd assume that things like Numpy are able to work without tools like BLAS, but you might also be able to compile the essential ones to WASM as well.

NumPy can work without libraries like BLAS or LAPACK, but it will be sloooooooooow. There are C versions of those libraries however, maybe they are using those?

Yes. It uses the C implementations of BLAS and LAPACK that ship in the Numpy source tree.

Re: The Python scientific stack, compiled to WebAssembly

#45

Post by mdboom » Fri, Aug 24, 2018, 6:46 PM UTC

krona wrote: post #3

Coming soon [...] Plotting using D3 from Python Wouldn't Bokeh make more sense? I'd be very interested in a serverless Bokeh that didn't force me to ditch Python, for example.

Bokeh could probably be done, but most of Bokeh's client/server complexity becomes unnecessary in this execution model.

More recently, pyodide has grown full matplotlib support. See, for example, https://iodide.io/pyodide-demo/matplotlib-sideload.html?side...

Re: The Python scientific stack, compiled to WebAssembly

#46

Post by travisoneill1 » Fri, Aug 24, 2018, 6:51 PM UTC

lx3459683 wrote: post #34

I benchmarked this recently. It comes in at 4-10x slower than cpython for random typical use cases on my machine, which is not too shabby at all. The author also states that numpy is slower than it should be due to lack of support for BLAS at this time. The real impressive feat to me isn't hosting jupyter in the browser. It's access to a reasonably fast implementation of numpy in the browser which smokes native JS co…

Did you use js typed arrays (e.g. Uint32Array) to benchmark large array operations or just Array?

Re: The Python scientific stack, compiled to WebAssembly

#47

Post by TAForObvReasons » Fri, Aug 24, 2018, 7:27 PM UTC

makmanalp wrote: post #35

lx3459683 wrote: post #34

I benchmarked this recently. It comes in at 4-10x slower than cpython for random typical use cases on my machine, which is not too shabby at all. The author also states that numpy is slower than it should be due to lack of support for BLAS at this time. The real impressive feat to me isn't hosting jupyter in the browser. It's access to a reasonably fast implementation of numpy in the browser which smokes native JS co…

I can't believe I'm saying this but it would be neat to have a browser native BLAS, or absent that perhaps cross-compiled to WASM from a RUST implementation or something! Makes me think of all the demos like the tensorflow playground (dunno if all that happens in the browser or not).

People are working on the pure JS version of BLAS: https://github.com/jacobbogers/blasjs . Hopefully that work leads to best practices and a standard

Re: The Python scientific stack, compiled to WebAssembly

#48

Post by sciurus » Fri, Aug 24, 2018, 8:00 PM UTC

zitterbewegung wrote: post #13

xtrapolate wrote: post #4

Earlier quoted context omitted.

> "Part of an effort to make a Jupytr notebook-like environment without relying on server side code." So there won't be any server side code, you'd just be downloading the entire Python stack and dependencies every time you want to use this through your browser. This time it will be WASM though, instead of x86/x64/pyc. That we can, doesn't mean that we should.

It would work on a Chromebook and a iPad also. And yes I could pay for a Jupiter notebook hosted on some instance but I dont want to.

Maybe, but it might require more RAM and CPU horsepower than they provide in order to be usable.

Re: The Python scientific stack, compiled to WebAssembly

#49

Post by supermdguy » Fri, Aug 24, 2018, 8:19 PM UTC

nategri wrote: post #38

quiq wrote: post #37

Earlier quoted context omitted.

Do they have Docker? Jupyter maintains a few docker images with various parts of the scientific stack.

How and why would they have docker and not python. Docker's like 10 rungs up the skill ladder.

If they're working with a different stack that includes docker?

Re: The Python scientific stack, compiled to WebAssembly

#50

Post by existencebox » Fri, Aug 24, 2018, 9:48 PM UTC

nsriv wrote: post #23

alexeiz wrote: post #19

This is great. Many times I wanted to showcase to colleagues who don't use Python how things would work in Python, but they don't have either Python or Jupyter installed. So I had to run Jupyter server in my own environment and give them access to it. This is quite an inconvenience as well as a security risk. Having a completely server-less Python/Jupyter environment should make this kind of showcasing much easier.

You could also use a Jupyter notebook uploaded to colab.research.google.com

I'll chime in to mention notebooks.azure.com as well.

(Typical disclaimer, I'm a dev on the azure notebooks team; we just try to solve exactly the scenario GP was asking so I would be remiss to not throw our hat in the ring)