Datashader: turns even the largest data into images, accurately • Hacker News Board

Re: Datashader: turns even the largest data into images, accurately

#71

Post by BubRoss » Tue, Feb 12, 2019, 7:06 PM UTC

jbednar wrote: post #67

BubRoss wrote: post #63

Earlier quoted context omitted.

If I said I was an expert in 'big data visualization with billions of points' and had written my own 'out of core' rendering library that I dubbed 'data shader', complete with a paper where I coined the term 'Abstract Rendering' or 'AR' for short, then you found out that I was just reading points from disk and drawing them with opengl's draw points function, what would you think? The term 'out of core' rendering come…

Sigh. Datashader is not a paper, it's an actual usable piece of software, so it should be compared to other tools and libraries for rendering data. Unlike nearly ever other 2D plotting library available for Python, it can operate in core or out of core, so it's entirely appropriate to advertise that fact (why hide it?). Unlike OpenGL's point drawing functions and nearly every other 2D plotting library available for P…

> Datashader is not a paper

https://www.semanticscholar.org/paper/Abstract-rendering%3A-...

You keep defending the project as a whole while not confronting the fact that they are touting rendering breakthroughs, while I have given a lot of explanation of why there are no rendering breakthroughs and the actual rendering, no matter where it is done and no matter how much data is used, is trivial. I'm not sure what can help you focus in on the point I'm making here, I haven't strayed from it. This isn't about the workflow or the language used or anything else. It is about false claims and buzzwords to make people think that it is solving rendering problems that have never existed like 'accuracy' and 'big data' ( in the context of these visualizations ).

Re: Datashader: turns even the largest data into images, accurately

#72

Post by CyberDildonics » Wed, Feb 13, 2019, 6:37 AM UTC

pwang wrote: post #66

CyberDildonics wrote: post #45

Earlier quoted context omitted.

There are a lot of red flags in the abstract of that paper alone. > Rendering techniques are currently a major limiter since they tend to be builtaround central processing with all of the geometric data present. This is completely untrue - OpenGL and virtually all real time rendering is done using z-buffer techniques that were originally used because they don't need all the geometry present. These techniques date bac…

> It looks like that paper is about the transformations of visualizations for higher dimensional data, not rendering accuracy, so these two things are being conflated even though they are completely separate concepts. Actually, no. The paper may not have been explicitly clear about this, but the ENTIRE point of a "data visualization" system is to transform potentially high-dimensional datasets, with a large number of…

> You seem to be interpreting this narrowly

I'm not sure why you would coin a term 'Abstract Rendering' and talk about 'out of core rendering' then turn around and say that transforming high dimensional data sets is part of rendering. Rendering is well defined and very established, coming up with transformations and calling that part of rendering is nonsense. You made this mess yourself by trying to stretch the truth.

Re: Datashader: turns even the largest data into images, accurately

#73

Post by eggie » Wed, Feb 13, 2019, 11:43 AM UTC

BubRoss wrote: post #71

jbednar wrote: post #67

Earlier quoted context omitted.

Sigh. Datashader is not a paper, it's an actual usable piece of software, so it should be compared to other tools and libraries for rendering data. Unlike nearly ever other 2D plotting library available for Python, it can operate in core or out of core, so it's entirely appropriate to advertise that fact (why hide it?). Unlike OpenGL's point drawing functions and nearly every other 2D plotting library available for P…

> Datashader is not a paper https://www.semanticscholar.org/paper/Abstract-rendering%3A-... You keep defending the project as a whole while not confronting the fact that they are touting rendering breakthroughs, while I have given a lot of explanation of why there are no rendering breakthroughs and the actual rendering, no matter where it is done and no matter how much data is used, is trivial. I'm not sure what can…

They are touting it specifically in the context of the visualization of very large datasets.

The fact that their software exists is itself a breakthrough. It enabled me to do things that other equivalent tools (such as in statistical packages) could not allow. I would have been reduced to directly implementing my rendering pipelines, and I would also have had to make many of the same design decisions they made, such as doing things out of core.