Datashader: turns even the largest data into images, accurately
1–10 of 73 posts
Re: Datashader: turns even the largest data into images, accurately
#2This is basically visualization 101. 'Turning data into images' isn't exactly a new concept. Sometimes I think the next generation of programmers has gotten too good at thinking up fancy names for reinventing the wheel and hasn't spent enough time looking at what has been done in the past 50 years so they can build off of it.
Also there are links to 'visualizing big data' and 'visualizing billions of points'. These too have been problems solved a long time ago with a lot less resources. The book advanced renderman are is a good place to start, and of course, opengl uses basic z-buffer rendering.
Unless you need sorted transparency (and accumulating points for brightness definitely does not), then you can rasterize as much as you want in any order, without holding anything in memory except for the image buffer. It doesn't matter how many point/particles you have, you don't have to do anything to make it scale.
Re: Datashader: turns even the largest data into images, accurately
#3This is basically visualization 101. 'Turning data into images' isn't exactly a new concept. Sometimes I think the next generation of programmers has gotten too good at thinking up fancy names for reinventing the wheel and hasn't spent enough time looking at what has been done in the past 50 years so they can build off of it. Also there are links to 'visualizing big data' and 'visualizing billions of points'. These t…
What are those fractals on the 3rd picture? They remind me of Lissajous curves used in oscilloscopes.
Re: Datashader: turns even the largest data into images, accurately
#4This is basically visualization 101. 'Turning data into images' isn't exactly a new concept. Sometimes I think the next generation of programmers has gotten too good at thinking up fancy names for reinventing the wheel and hasn't spent enough time looking at what has been done in the past 50 years so they can build off of it. Also there are links to 'visualizing big data' and 'visualizing billions of points'. These t…
And this is how you push people out of our field.
How about instead of starting with an insult (I can't believe you didn't already know this) you instead congratulate them on putting together a full working library, with pretty, easy-to-grasp examples then offer up some research links that they could use to further refine and improve their system. It's our job to teach people, you can't expect everyone to suddenly know everything.
To the Datashader team: I apologize for the above comment. Good job in building and launching a tool for others to use, and great choices for examples!
Re: Datashader: turns even the largest data into images, accurately
#5This is basically visualization 101. 'Turning data into images' isn't exactly a new concept. Sometimes I think the next generation of programmers has gotten too good at thinking up fancy names for reinventing the wheel and hasn't spent enough time looking at what has been done in the past 50 years so they can build off of it. Also there are links to 'visualizing big data' and 'visualizing billions of points'. These t…
It doesn't matter if something has been done before if the new way hooks into new systems. Tools do not exist in a vacuum. All tools are apart of a system of tools that should always be considered when evaluating any component piece.
Re: Datashader: turns even the largest data into images, accurately
#6This is basically visualization 101. 'Turning data into images' isn't exactly a new concept. Sometimes I think the next generation of programmers has gotten too good at thinking up fancy names for reinventing the wheel and hasn't spent enough time looking at what has been done in the past 50 years so they can build off of it. Also there are links to 'visualizing big data' and 'visualizing billions of points'. These t…
And this is how you push people out of our field. How about instead of starting with an insult (I can't believe you didn't already know this) you instead congratulate them on putting together a full working library, with pretty, easy-to-grasp examples then offer up some research links that they could use to further refine and improve their system. It's our job to teach people, you can't expect everyone to suddenly kn…
As a newcomer to the field, parent post is far more welcoming than the bevy of trash Ninja flavor-of-the-week bullshit that makes it seem impossible to catch up.
I only actually got my bearings and self-confidence as a programmer when I realized that most of the people pushing blogs with subscriptions about "cutting-edge" tech were literally snake-oil salesmen and shovel merchants.
That coding wasn't actually different from anything else I had learned in my life, and that there were some fundamentals I could latch onto, and grow from there upwards. All this nonsense about the field experiencing a revolution that upends all existing knowledge year-after-year is far more mentally taxing.
Re: Datashader: turns even the largest data into images, accurately
#7This is basically visualization 101. 'Turning data into images' isn't exactly a new concept. Sometimes I think the next generation of programmers has gotten too good at thinking up fancy names for reinventing the wheel and hasn't spent enough time looking at what has been done in the past 50 years so they can build off of it. Also there are links to 'visualizing big data' and 'visualizing billions of points'. These t…
And this is how you push people out of our field. How about instead of starting with an insult (I can't believe you didn't already know this) you instead congratulate them on putting together a full working library, with pretty, easy-to-grasp examples then offer up some research links that they could use to further refine and improve their system. It's our job to teach people, you can't expect everyone to suddenly kn…
“Dropbox is just rsync on a cron job” and “OP’s project is not completely novel” are unfortunately common tropes on HN...
Like you said. Starting with “Great job launching. What are the benefits of using Dropbox over, say just rsync + cron?” would go a long way towards improving the environment around here.
Re: Datashader: turns even the largest data into images, accurately
#8This is basically visualization 101. 'Turning data into images' isn't exactly a new concept. Sometimes I think the next generation of programmers has gotten too good at thinking up fancy names for reinventing the wheel and hasn't spent enough time looking at what has been done in the past 50 years so they can build off of it. Also there are links to 'visualizing big data' and 'visualizing billions of points'. These t…
What are those fractals on the 3rd picture? They remind me of Lissajous curves used in oscilloscopes.
I think they are "symmetric icons" :
http://www.alunw.freeuk.com/symiconroom.html
Can't remember where I've seen them before - possibly in a book on Symmetry.
Re: Datashader: turns even the largest data into images, accurately
#9This is basically visualization 101. 'Turning data into images' isn't exactly a new concept. Sometimes I think the next generation of programmers has gotten too good at thinking up fancy names for reinventing the wheel and hasn't spent enough time looking at what has been done in the past 50 years so they can build off of it. Also there are links to 'visualizing big data' and 'visualizing billions of points'. These t…
And this is how you push people out of our field. How about instead of starting with an insult (I can't believe you didn't already know this) you instead congratulate them on putting together a full working library, with pretty, easy-to-grasp examples then offer up some research links that they could use to further refine and improve their system. It's our job to teach people, you can't expect everyone to suddenly kn…
My problem isn't that isn't novel or new, it is that it is presented as novel. Visualization is great (and the visualization here looks good), but saying things like 'datashader: turn data into images' when that is literally what rendering is, is a nonsense way to approach a valid topic.
Saying 'visualize big data and billions of points' when the buzzwords are just there to sugar coat an accumulation buffer, gets into a territory of reinventing the wheel but naming it 'the flattened infinite curvature hypersphere'.
So, before you get too self righteous at least realize that it is the delivery and lack of context and precedent, not the actual work that is the problem.
Re: Datashader: turns even the largest data into images, accurately
#10This is basically visualization 101. 'Turning data into images' isn't exactly a new concept. Sometimes I think the next generation of programmers has gotten too good at thinking up fancy names for reinventing the wheel and hasn't spent enough time looking at what has been done in the past 50 years so they can build off of it. Also there are links to 'visualizing big data' and 'visualizing billions of points'. These t…
I also was expecting something new, but in their defense they've made a very appealing version of something old. I'm sure there are a lot of people out there who haven't thought about saturation with "large" data sets before.