Aggregating away the signal in your data
stackoverflow.blog
Aggregating away the signal in your data
1–10 of 12 posts
Re: Aggregating away the signal in your data
#2I'm only halfway through the article, but must say that it's amazing so far and the dataviz much better and careful than what I regularly see.
Re: Aggregating away the signal in your data
#3One person's junk is another's treasure; one person's "normalized data" is another person's "you removed the one data point I cared most about!"
One thing this article is reinforcing to me is the value of domain knowledge to an analyst. I am deeply skeptical of "one size fits all" analysis tools, services, and consultancies for exactly this reason. Making insight actionable requires knowing what actions can be taken, and how.
Re: Aggregating away the signal in your data
#4When I saw the yellow of the graph it made me think of Tufts famous book. A few paragraphs in I had to check that it actually wasn't authored by him.
The depth and knowledge the author displays is fantastic.
Re: Aggregating away the signal in your data
#5Excellent article. Faceted visualisation is an incredibly powerful technique.
Something the author hints at but isn't quite explicit: manual inspection of individual examples from your dataset can help you understand what questions to ask, what category to facet on, or the bug in your aggregation.
Re: Aggregating away the signal in your data
#6Excellent article, aggregation can also obfuscate problems with sensors (for example, weird quantization or duplicating points). It is useful whenever you have high frequency time series to look at the data points for the highest resolution possible for short segments of few data points.
Re: Aggregating away the signal in your data
#7An example that elucidates the point: https://en.wikipedia.org/wiki/Simpson's_paradox
Re: Aggregating away the signal in your data
#8The full title is "Stop aggregating away the signal in your data".
Re: Aggregating away the signal in your data
#9One person's junk is another's treasure; one person's "normalized data" is another person's "you removed the one data point I cared most about!" One thing this article is reinforcing to me is the value of domain knowledge to an analyst. I am deeply skeptical of "one size fits all" analysis tools, services, and consultancies for exactly this reason. Making insight actionable requires knowing what actions can be taken,…
I agree, but I think the visualizations presented here can be useful in many domains and aren’t generally used. Furthermore, I think showing uncertainty in visualizations is hugely important and this is a step in the right direction there.
Re: Aggregating away the signal in your data
#10I cannot recommend enough the "John Lamping - The One Weird Trick for Analyzing Big Data ... Eyeball it Early and Often!" video:
https://www.youtube.com/watch?v=jYH8CQS6Ab0
One of the best tips, straight from a practitioner - from a former Google search ranking engineer who touched multiple other domains later in the career. Stop tuning knobs and watching metrics, look at the data!