Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fastplotlib: GPU-accelerated, fast, and interactive plotting library

medium.com

Post reply on HN
Print view View original

111–120 of 190 posts

Re: Fastplotlib: GPU-accelerated, fast, and interactive plotting library

#111

Post by rossant »

kushalkolar wrote: post #98
rossant wrote: post #93

Earlier quoted context omitted.

I've been looking into this issue with Datoviz [1] following a user request. It turns out there may be a way to achieve it using Vulkan [2] (which Datoviz is based on) and CuPy's UnownedMemory [3]. I wrote a simple proof of concept using only Vulkan and CuPy. I'm now working on a way for users to wrap a Datoviz GPU buffer as a CuPy array that directly references the Datoviz-managed GPU memory. This should, in princip…

This looks cools thanks! Makes me wonder if there's any way to do that with WGPU if WGPU is interfacing with Vulkan, probably not easy if possible I"m guessing. WGPU has security protections since it's designed for the browser so I'm guessing it's impossible.

Indeed, it doesn't seem to be possible at the moment, see e.g. https://github.com/gfx-rs/wgpu/issues/4067

Re: Fastplotlib: GPU-accelerated, fast, and interactive plotting library

#113

Post by mturmon »

wtallis wrote: post #31
macleginn wrote: post #30

Earlier quoted context omitted.

Eigendecomposition of the covariance matrix, essentially PCA, is probably the first non-trivial step in the analysis of any dataset. The idea in the comment above seems to be that it's more useful to combine some basic knowledge of statistics with simpler visualisation techniques, rather than to quickly generate thousands of shallower plots. Being able to generate thousands of plot is useful, of course, but I would a…

> Eigendecomposition of the covariance matrix, essentially PCA, is probably the first non-trivial step in the analysis of any dataset For a sufficiently narrow definition of "dataset", perhaps. I don't think it's the obvious step one when you want to start understanding a time series dataset, for example. (Fourier transform would be a more likely step two, after step one of actually look at some of your data .)

I agree, but: the technique of “singular spectrum analysis” is pretty much PCA applied to a covariance matrix resulting from time-lagging the original time series. (https://en.wikipedia.org/wiki/Singular_spectrum_analysis)

So this is not unheard of for time series analysis.

Re: Fastplotlib: GPU-accelerated, fast, and interactive plotting library

#114

Post by vibranium »

I’m often working with a windows desktop and a remote Linux box on which I have my data & code. I’d like to plot “locally” on my desktop workstation from the remote host. This usually either means using X11 (slow) or some sort of web-based library like plotly. Does fastplotlib offer any easy solution here?

Re: Fastplotlib: GPU-accelerated, fast, and interactive plotting library

#115

Post by jampekka »

paddy_m wrote: post #88
jampekka wrote: post #20

Earlier quoted context omitted.

At least I usually do prefer to do the EDA plotting by writing and editing code. This is a lot more flexible. It's relatively rare to need other interactivity than zooming and panning. The differing approaches probably can be seen in some API choices, although the fastplotlib API is a lot more ergonomic than many others. Having to index the figure or prefixing plots with add_ are minor things, and probably preferable…

Writing and editting code is a lot more flexible, but it gets repetitive, and I have written the same stuff so many times. It's all adhoc, and it fixes the problem at the time, then it gets thrown away with the notebook only to be written again soon. As an example, I frequently want to run analytics on a dataframe. More complex summary stats. So you write a couple of functions, and have two for loops, iterating over…

Usually I write scripts that use function memoization cache (to disk) for expensive operations. Recently I've also used Marimo sometimes, which has great support for modules (no reloading hacks), can memoize to disk and has deterministic state.

Re: Fastplotlib: GPU-accelerated, fast, and interactive plotting library

#116

Post by kushalkolar »

vibranium wrote: post #114

I’m often working with a windows desktop and a remote Linux box on which I have my data & code. I’d like to plot “locally” on my desktop workstation from the remote host. This usually either means using X11 (slow) or some sort of web-based library like plotly. Does fastplotlib offer any easy solution here?

This is exactly why we use jupyter-rfb, I often have large datasets on a remote cluster computer and we perform remote rendering.

see: https://fastplotlib.org/ver/dev/user_guide/faq.html#what-fra...

Re: Fastplotlib: GPU-accelerated, fast, and interactive plotting library

#117

Post by lagrange77 »

kushalkolar wrote: post #72
lagrange77 wrote: post #66

Nice, i'd be interested to know which method for drawing lines (which is hard [0]) it uses. [0] https://mattdesl.svbtle.com/drawing-lines-is-hard

Almar made blog posts about the line shader he wrote! https://almarklein.org/triangletricks.html https://almarklein.org/line_rendering.html A big shader refactor was done in this PR: https://github.com/pygfx/pygfx/pull/628

Thank you!

Re: Fastplotlib: GPU-accelerated, fast, and interactive plotting library

#118

Post by enriquto »

That would be preposterous if it wasn't so hilariously false:

> These days, having a GPU is practically a prerequisite to doing science, and visualization is no exception.

It becomes really funny when they go on to this, as if it was a big deal:

> Depicted below is an example of plotting 3 million points

Anybody who has ever used C or fortran knows that a modern CPU can easily churn through "3 million points" at more than 30 frames per second, using just one thread. It's not a particularly impressive feat, three million points is the size of a mid-resolution picture, and you can zoom-in and out those trivially in real-time using a CPU (and you could do that 20 years ago, as well). Maybe the stated slowness of fastplotlib comes from the unholy mix of rust and python?

Now, besides this rant, I think that fastplotlib is fantastic and, as an (unwilling) user of Python for data science, it's a godsend. It's just that the hype of that website sits wrong in me. All the demos show things that could be done much easier and just as fast when I was a teenager. The big feat, and a really big one at that, is that you can access this sort of performance from python. I love it, in a way, because it makes my life easier now; but it feels like a self-inflicted problem was solved in a very roundabout way.

Re: Fastplotlib: GPU-accelerated, fast, and interactive plotting library

#120

Post by enriquto »

clewis7 wrote: post #34
theLiminator wrote: post #11

Do you have any numbers for the rough number of datapoints that can be handled? I'm curious if this enables plotting many millions of datapoints in a scatterplot for example.

Yes! The number of data points can range in the millions. Quite honestly, the quality of your GPU would be the limiting factor here. I will say, however, that for most use cases, an integrated GPU is sufficient. For reference, we have plotted upwards of 3 million points on a mid-range integrated GPU from 2017. I will work on adding somewhere in our docs some metrics for this kind of thing (I think it could be helpful…

>I will work on adding somewhere in our docs some metrics for this kind of thing (I think it could be helpful for many).

Certainly! A comparison of performance with specialized tools for large point clouds would be very interesting (like cloudcompare and potree).

Post reply on HN