Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Perspective: Streaming pivot visualization via WebAssembly

github.com

Post reply on HN
Print view View original

131–140 of 141 posts

Re: Perspective: Streaming pivot visualization via WebAssembly

#131

Post by dman »

lmeyerov wrote: post #130
BenGosub wrote: post #98

Earlier quoted context omitted.

bringing Apache Arrow in the browser alongside wasm is exciting to say the least! Amazing capabilities are coming to browsers...

We (Graphistry) recently contributed a native JS reader/writer into the Apache Arrow project, so may help both teams! We did it as legwork for our beyond-native efforts (GPU cloud streaming) and taming our JS datatypes, so similar needs here I'm guessing! Funny enough: was in NYC today talking with banking teams about related tech. Too bad we didn't know about this effort, would have loved to meet!

Drop me a note the next time you are in NY, would love to meet up.

Re: Perspective: Streaming pivot visualization via WebAssembly

#133

Post by robmcm »

bassman9000 wrote: post #128
mi100hael wrote: post #18

Really cool to see non-software companies putting cool projects like this out as open source

JPMorgan IT department must be larger than the majority of SW companies out there

"With 40,000 technologists across 14 technology hubs around the world, there are endless opportunities to create what’s next"

[https://careers.jpmorgan.com/careers/US/en/divisions/technol...]

Re: Perspective: Streaming pivot visualization via WebAssembly

#134

Post by robmcm »

mi100hael wrote: post #67
ashtonian wrote: post #25

Earlier quoted context omitted.

the stock market is digital, they are a software company.

J.P.Morgan is not "the stock market" and financial companies are not software companies. They are not selling software. They are selling financial services.

SAA(F)S?

Re: Perspective: Streaming pivot visualization via WebAssembly

#135

Post by texodus »

lmeyerov wrote: post #130
BenGosub wrote: post #98

Earlier quoted context omitted.

bringing Apache Arrow in the browser alongside wasm is exciting to say the least! Amazing capabilities are coming to browsers...

We (Graphistry) recently contributed a native JS reader/writer into the Apache Arrow project, so may help both teams! We did it as legwork for our beyond-native efforts (GPU cloud streaming) and taming our JS datatypes, so similar needs here I'm guessing! Funny enough: was in NYC today talking with banking teams about related tech. Too bad we didn't know about this effort, would have loved to meet!

Yes, this is the library we use.

We have met before actually, you did a demo at JPM in midtown several years back. Graphistry has come a very long way since then - impressive work!

Re: Perspective: Streaming pivot visualization via WebAssembly

#136

Post by texodus »

polskibus wrote: post #108
texodus wrote: post #55

Earlier quoted context omitted.

It supports virtualization, and it is utilized by both plugins (Hypergrid and Highcharts). N levels of pivoting on both axes. Should work on all browsers - if not, please open an issue!

By virtualization I meant also not loading entire dataset to memory, but being able to work only on a constrained view of the data. For example, database has 1TB dataset, I only view 100 MB, further columns/sections are loaded when I scroll.

It is virtual in the sense that it does not realize the entire dataset in the JS side of the WebAssembly bridge - this is a performance optimization but entirely in-memory. While you can run the engine in node.js and use it to efficiently stream updates to a symmetric engine in the browser, we do not currently implement server-virtualized views.

Though, we have quite alot of experience doing this in the past - and the design of Perspective is very much a reaction to what we learned, at lesat in regards to the typical financial dataset which is much smaller than 1TB.

Re: Perspective: Streaming pivot visualization via WebAssembly

#137

Post by lmeyerov »

texodus wrote: post #135
lmeyerov wrote: post #130

Earlier quoted context omitted.

We (Graphistry) recently contributed a native JS reader/writer into the Apache Arrow project, so may help both teams! We did it as legwork for our beyond-native efforts (GPU cloud streaming) and taming our JS datatypes, so similar needs here I'm guessing! Funny enough: was in NYC today talking with banking teams about related tech. Too bad we didn't know about this effort, would have loved to meet!

Yes, this is the library we use. We have met before actually, you did a demo at JPM in midtown several years back. Graphistry has come a very long way since then - impressive work!

Ah, small world. That was probably when we first started on clientserver GPU streaming. Looking forward to digging into the Perspective source!

Re: Perspective: Streaming pivot visualization via WebAssembly

#138

Post by a_humean »

bringtheaction wrote: post #127
a_humean wrote: post #110

Earlier quoted context omitted.

No, he is right. cargo-web supports three backends, of which only two include emscripten and the third using a native rust backend. https://github.com/koute/cargo-web#features Rust can be compiled to wasm without emscripten using wasm32-unknown-unknown.

Does that mean I can install cargo-web without emscripten or do I still need it even though I am not going to use it just because the two other targets use it? In other words, is emscripten an optional or a hard dependency?

Its an optional dependency. You can build without it by just using a nightly version of rust with the wasm32-unknown-unknown backend.

An example of using cargo-web with the stdweb rust project: https://github.com/koute/stdweb#running-the-examples

Re: Perspective: Streaming pivot visualization via WebAssembly

#139

Post by fitzoh »

dman wrote: post #2

I wrote the C++ part of this codebase, would be happy to answer any questions about it here.

Just stalked you on linkedin with a similar question, but I was wondering if you were involved with the actual process of getting the library open sourced, and how that works at a large security focused company like JPMorgan Chase. (former Chase employee)

Re: Perspective: Streaming pivot visualization via WebAssembly

#140

Post by btown »

lotyrin wrote: post #125
jnordwick wrote: post #122

Earlier quoted context omitted.

I think it is more correct to say that a large subunit of JPM works as a software company -- for its internal use. All modern financial companies are heavily tech oriented, even those you wouldn't normally think of that way. The best tech firms I've ever been involved with financial firms from front desk trading on back. One of the best tech teams I've ever met with has been one of Bloomberg's teams dealing with thei…

Sure yes, in modern business environments, everyone's business interacts at least in some way with computing, digital products like media and software, and the internet. Even if you had a remote industrial site cutoff from the wider internet, there would be some automation to be done and some code to do it. Every organization should have a CTO, if nothing else than to make procurement and outsourcing decisions. Organ…

You would be pleasantly surprised, then, that this reality has already permeated the modern financial industry. For instance, see http://www.businessinsider.com/goldman-sachs-wants-to-become... . While there are still “relationship bankers” in expensive suits, most acknowledge that software is key to giving them an edge over the competition, and that building a culture around software only helps them.
Post reply on HN