I wrote the C++ part of this codebase, would be happy to answer any questions about it here.
Can you say how old is this codebase and what was its original business case?
Perspective: Streaming pivot visualization via WebAssembly
101–110 of 141 posts
Re: Perspective: Streaming pivot visualization via WebAssembly
#102I wrote the C++ part of this codebase, would be happy to answer any questions about it here.
Can you say how old is this codebase and what was its original business case?
Codebase started in 2013. Original business case was to allow business users to create filtered/aggregated views on top of streaming data.
Re: Perspective: Streaming pivot visualization via WebAssembly
#103Earlier quoted context omitted.
There are some attempts on that direction. https://www.hanselman.com/blog/NETAndWebAssemblyIsThisTheFut...
Sure. I've played with Blazor. It's very raw at this point and just a side project. I'd like to see MS Research or even a full blown R&D team at MS go full force at this.
Microsoft just announced that they will start investing more into it, depending how the community takes it.
https://blogs.msdn.microsoft.com/webdev/2018/02/06/blazor-ex...
Re: Perspective: Streaming pivot visualization via WebAssembly
#104Earlier quoted context omitted.
Even outside of C/C++, Emscripten seems to be required in some fashion for just about every "compile X to webasm". Which i'm not that happy about as it's quite heavy and takes a long time to compile, which makes it harder for newcomers to try it out because it requires a fairly large time investment to even compile hello world.
Rust’s wasm32-unknown-unknown target does not use emscripten at all. The tool cargo-web will handle the js module shims for you.
cargo-web uses emscripten under the hood.
Re: Perspective: Streaming pivot visualization via WebAssembly
#105I wrote the C++ part of this codebase, would be happy to answer any questions about it here.
What's on the roadmap?
Re: Perspective: Streaming pivot visualization via WebAssembly
#106Earlier quoted context omitted.
I see it's possible to embed it in jupyterlab, it seems to do a lot: * grid format * graph format * actual pivoting of data Is that right? It's using webassembly so in effect to do the pivot you must have created functions that exist in pandas? Has anyone ported pandas to webassembly? Is the grid editable? What are you using to create your output, html or canvas or something? I realise I could look this stuff up but…
I believe you'd need to port Python to WebAssembly to make much use of pandas - unaware of any projects attempting the former. The grid is just a plugin for the excellent [Hypergrid]( https://github.com/openfin/fin-hypergrid ), which is editable, but you'd need to manually push those edits back to the engine for now.
WebAssembly code can talk to the rest of your JS (and therefore asm.js) code, right?
So PyPy.js runs Python in the browser via asm.js:
https://github.com/pypyjs/pypyjs
Livebook uses PyPy.js to make a Jupyter-like notebook that runs entirely in the browser, and includes pandas:
https://livebook.inkandswitch.com
https://github.com/inkandswitch/livebook
And random plug for Observable in case you haven't seen it, a Jupyter-like Javascript environment, completely in the browser:
Re: Perspective: Streaming pivot visualization via WebAssembly
#107First, let me say this looks awesome! I’m curious if your considered React.js for the presentation layer (i.e. the js/html5 parts) - on the surface it seems like it would be a good fit - and if so, what you saw as the benefits or drawbacks of it vs the approach you went with.
We chose to go with a Web Components based interface for compatibility across frameworks, but alot of where we go in the future will be determined by the expertise of the developers we hire to work on it, and the community if there is interest there.
First of all, great work everyone involved! I know first-hand how hard it is to push something like this past compliance ;)
Going with WC is a sound decision since its trivial to wrap it into ember, react, or any other framework-du-jour... especially so since more and more of them are adopting WC patterns and design practices.
I might try to wrap this into react and will see if bosses permit opensourcing it!
Re: Perspective: Streaming pivot visualization via WebAssembly
#108Earlier quoted context omitted.
Does it support: virtualization (showing only a view of full dataset that is updated from server on scroll) Pivoting (multiple levels/hierarchies on rows and on columns) Apart from features what part did you consider was the biggest hurdle? What browsers did you target?
It supports virtualization, and it is utilized by both plugins (Hypergrid and Highcharts). N levels of pivoting on both axes. Should work on all browsers - if not, please open an issue!
By virtualization I meant also not loading entire dataset to memory, but being able to work only on a constrained view of the data. For example, database has 1TB dataset, I only view 100 MB, further columns/sections are loaded when I scroll.
Re: Perspective: Streaming pivot visualization via WebAssembly
#109Re: Perspective: Streaming pivot visualization via WebAssembly
#110Earlier quoted context omitted.
Rust’s wasm32-unknown-unknown target does not use emscripten at all. The tool cargo-web will handle the js module shims for you.
cargo-web uses emscripten under the hood.
No, he is right. cargo-web supports three backends, of which only two include emscripten and the third using a native rust backend.
https://github.com/koute/cargo-web#features
Rust can be compiled to wasm without emscripten using wasm32-unknown-unknown.