31–40 of 73 posts
Is there anything similar for network graphs?
I added edge bundling (probably the slowest thing in datashader!) but I know there's examples of flight path rendering in the video I linked in another comment.
I've used datashader for plotting NGS (Next Generation Sequencing) enrichments. At the time I had to hack together the ability to use the polygon select tool on the data, but it worked and blew my mind. Very elegant solution to a difficult problem (overplotting).
Earlier quoted context omitted.
> thinking up fancy names for reinventing the wheel They're not claiming to have reinvented the wheel, they're just explaining what it is. > 'Turning data into images' isn't exactly a new concept. No, but doing so on large data accurately (the last word is important that you cut off) is not something I know I can easily achieve in a different python library faster. I'd like to know if I could.
> but doing so on large data accurately What do you mean by 'large' or 'accurate' ? Where would accuracy be lost in any approach?
We renamed from Abstract Rendering to Datashader for affordances of human cognition.
This is a great paper from Gordon Kindlmann and Carlos Scheidegger talk about how to gauge the accuracy of a visualization, as part of an effort to come up with an algebraic process for visual design: https://vis.cs.ucdavis.edu/vis2014papers/TVCG/papers/2181_20...
Using their metrics around "confusers" and "hallucinators", Datashader came out as one of the few things that doesn't suffer from such intrinsic limitations.
Looks like a really cool project. One thing that I would be interested in see would be using Datashader as a dynamic visualisation library - for example, generative art projects. Probably not the main interest of data visualisation practitioners but hey, if you've got a sweet pipeline to render all those points, why not?
Is there anything similar for network graphs?
Earlier quoted context omitted.
My problem isn't that isn't novel or new, it is that it is presented as novel. Visualization is great (and the visualization here looks good), but saying things like 'datashader: turn data into images' when that is literally what rendering is, is a nonsense way to approach a valid topic. Saying 'visualize big data and billions of points' when the buzzwords are just there to sugar coat an accumulation buffer, gets int…
I suspect marketing Datashader as an "accumulation buffer" wouldn't have the same effect on its target audience (data visualisation developers) as presenting it simply as a way to "Turn data into images". I'm also curious (as a fledgling graphics programmer) - what leads you to believe that Datashader uses an accumulation buffer internally? I would think that they use some magic to draw all the points in a single dra…
This is basically visualization 101. 'Turning data into images' isn't exactly a new concept. Sometimes I think the next generation of programmers has gotten too good at thinking up fancy names for reinventing the wheel and hasn't spent enough time looking at what has been done in the past 50 years so they can build off of it. Also there are links to 'visualizing big data' and 'visualizing billions of points'. These t…
I actually have a background in 3D computer graphics, and it's precisely because of my detailed knowledge of raytracing, rasterization, OpenGL, BMRT, photon maps, computational radiometry, BDRFs, computational geometry, and statistical sampling, etc... that when I came to the field of data science & specifically the problem of visualizing large datasets, I realized the total lack of tooling in this space.
The field of information visualization lags behind general "computer-generated imagery" by decades. When I first presented my ideas around Abstract Rendering (which became Datashader) to my DARPA collaborators, even to famous visualization people like Bill Cleveland or Jeff Heer, it was clear that I was thinking about the problem in an entirely different way. I recall our DARPA PM asking Hanspeter Pfister how he would visualize a million points, and he said, "I wouldn't. I'd subsample, or aggregate the data."
Datashader eats a million points for breakfast.
Since you're clearly a computer graphics guy, the way to think about this problem is not one of naive rendering, but rather one of dynamically generating correct primitives & aesthetics at every image scale, so that the viewer has the most accurate understanding of what's actually in the dataset. So it's not just a particle cloud, nor is it nurbs with a normal & texture map; rather, it's a bunch of abstract values from which a data scientist may want to synthesize any combination of geometry and textures.
I chose the name "datashader" for a very specific and intentional reason: we are dynamically invoking a shader - usually a bunch of Python code for mathematical transformation - at every point, within a sampling volume (typically a square, but it doesn't have to be). One can imagine drawing a map of the rivers of the US, with the shading based on some function of all industrial plants in its watershed. Both the domain of integration and the function to evaluate are dynamic for each point in the view frustum.
Earlier quoted context omitted.
My problem isn't that isn't novel or new, it is that it is presented as novel. Visualization is great (and the visualization here looks good), but saying things like 'datashader: turn data into images' when that is literally what rendering is, is a nonsense way to approach a valid topic. Saying 'visualize big data and billions of points' when the buzzwords are just there to sugar coat an accumulation buffer, gets int…
“Turn data into images” is a much better phrase than “rendering.” Anyone who is not intentionally being pedantic for dramatic effect would understand the purpose of the product based on reading that phrase, whereas “Datashader: a renderer” is less clear and could refer to products with an entirely different scope/purpose. Their product page is well-written and accurate. It sounds like you want them to purposely descr…
Earlier quoted context omitted.
My problem isn't that isn't novel or new, it is that it is presented as novel. Visualization is great (and the visualization here looks good), but saying things like 'datashader: turn data into images' when that is literally what rendering is, is a nonsense way to approach a valid topic. Saying 'visualize big data and billions of points' when the buzzwords are just there to sugar coat an accumulation buffer, gets int…
I suspect marketing Datashader as an "accumulation buffer" wouldn't have the same effect on its target audience (data visualisation developers) as presenting it simply as a way to "Turn data into images". I'm also curious (as a fledgling graphics programmer) - what leads you to believe that Datashader uses an accumulation buffer internally? I would think that they use some magic to draw all the points in a single dra…
The pipeline is also built in such a way that it permits front-end JS viewers like Bokeh to drive a very dynamic experience.
Earlier quoted context omitted.
What are those fractals on the 3rd picture? They remind me of Lissajous curves used in oscilloscopes.
I think they are "symmetric icons" : http://www.alunw.freeuk.com/symiconroom.html Can't remember where I've seen them before - possibly in a book on Symmetry.