11–20 of 92 posts
It seems this is doing round trips to the server to do the calculations? I often think that with the current state of Javascript in the browser, we could build an awesome, super fast, Jupyter style notebook software that runs completely in the browser. With the modules implemented as native Javascript modules which are dynamically loaded. Is anybody working on this? I have built a rough version of this idea for mysel…
It seems this is doing round trips to the server to do the calculations? I often think that with the current state of Javascript in the browser, we could build an awesome, super fast, Jupyter style notebook software that runs completely in the browser. With the modules implemented as native Javascript modules which are dynamically loaded. Is anybody working on this? I have built a rough version of this idea for mysel…
Here is an example instance: https://notebook.basthon.fr/
The thing is, creating a whole stack in pure JS would be very hard, since the current scientific stacks uses a lot of fortran, assembly and C with python to bind them all. Or julia. It's millions of man hours we are talking about.
So compiling Python into WASM is probably the best deal for such an app.
For the regular web, it would be a deal breaker: you don't want to load 15 mo of runtime before being able to interact with a web page. But for such a scientific app, it's not a problem. Besides, were you to write it entirely in JS, the size would be huge as well.
I really like Jupyter notebooks to build a simple concept and then move to .py files. But what I observe, especially at the entry level or junior level jobs in data science is that people spend huge amount of its work on jupyter, which did not focus on how to plan flow properly. What I meant is that there is very short path from usefullness to overkill.
That's because they are not programmers. One should not expect them to be experts in 2 fields. They do their work with the tool provided, and if we want a better output, we need to provide better tooling or accept what comes out. I want my physicists to spend their mental effort on physics, not on software architecture.
It seems this is doing round trips to the server to do the calculations? I often think that with the current state of Javascript in the browser, we could build an awesome, super fast, Jupyter style notebook software that runs completely in the browser. With the modules implemented as native Javascript modules which are dynamically loaded. Is anybody working on this? I have built a rough version of this idea for mysel…
Checkout https://github.com/jupyterlite/jupyterlite , WASM'd Jupyter in the browser with Pyodide
It's a very promising technology IMO. Not necessarily because it is faster (which is only 0.3x-2x according to my findings) but because it is a nice, simple compilation target.
"Problem: package xeus-cling-0.12.0-h5a79028_0 requires xtl >=0.7.0,sigh
I don't know when it started, but it seems to be a recent trend to add dependencies for anything and to package everything on demand. It's probably for security or something. But I do miss the days when people would link a static binary that "just works" even without internet and that'll keep working a week later, because it includes all of its dependencies as opposed to downloading and updating 500 packages on-demand.
It seems this is doing round trips to the server to do the calculations? I often think that with the current state of Javascript in the browser, we could build an awesome, super fast, Jupyter style notebook software that runs completely in the browser. With the modules implemented as native Javascript modules which are dynamically loaded. Is anybody working on this? I have built a rough version of this idea for mysel…
Only you know your data volumes but for the scenarios in which I've reached for Jupyter they've often involved very large amounts of data and calculating on the server is what I wanted and needed. Agreed that for some things, it would be great to be able to explicitly offload to the browser.
I have not yet tried to load a lot of data into it. Would be interesting to see when the load time starts to outweight the benefits of instant calculations. Maybe at something like 10 million datasets? Hard to say.
I really like Jupyter notebooks to build a simple concept and then move to .py files. But what I observe, especially at the entry level or junior level jobs in data science is that people spend huge amount of its work on jupyter, which did not focus on how to plan flow properly. What I meant is that there is very short path from usefullness to overkill.
That's because they are not programmers. One should not expect them to be experts in 2 fields. They do their work with the tool provided, and if we want a better output, we need to provide better tooling or accept what comes out. I want my physicists to spend their mental effort on physics, not on software architecture.
Beginners not being aware of some best practices doesn't automatically make them not-programmers.
I work as a data scientist and I see it as part of software development. It's just a different domain - some people do front-end, some do mobile or embedded, I do data science.