Live data from Hacker News

Misleading Graph Generator

yrden.de

31–37 of 37 posts

Re: Misleading Graph Generator

#31
I hear more and more chanting of "correlation does not equal causation!" which is great if your goal is to form a causal model of the world, but there are plenty of insights you can arrive at from correlation alone.

For starters in the world of machine learning and predictive analytics, it doesn't really matter if X causes Y so long as X is a consistently good predictor of Y. Maybe powerlines being over someone's home are not the cause of cancer, but if their presence can be used to predict cancer rates that's a good thing.

More important imho is the idea of latent or hidden variables. Two things that are clearly correlated but also seem to not have a causal relationship (just as transistors and longevity) may share a latent variable, that may be either non-quantifiable or completely unobservable. For either case measuring outputs that share a common latent variable and thus correlate with each other might be the only way to attempt to measure hidden, non-quantifiable causes.

For example employee happiness might be the cause of employee retention. However you can't currently measure or observe 'happiness', but there may be many, seemingly, unrelated employee activities that correlate with retention because they are also driven by this same latent variable. Studying them is the only way to get a quantifiable understanding of this latent cause.

tl;dr somethimes correlation is just as important as causation.

Re: Misleading Graph Generator

#32
post #3

How is this graphs fault? The only fault the graph has is that it clearly displays the data, the problem is in the idea that is represented. It's formally called "Correlation does not imply causation" and it's fault of the person who is suggesting it. There is a famous satirical version of this too: http://en.wikipedia.org/wiki/File:PiratesVsTemp(en).svg It has nothing to do with graphs, graphs are are just visual to…

This is my favorite example: http://i.imgur.com/gAGjP.png

Re: Misleading Graph Generator

#33
post #16
post #11

Earlier quoted context omitted.

> It's formally called "Correlation does not imply causation" and it's fault of the person who is suggesting it. Well, correlation sure DOES imply causation. In the dictionary sense of "imply", as: "suggested but not directly expressed; implicit". What it does not is necessitate causation, but it sure as hell does imply it.

This is not true. Correlation means that there is some kind of relation between two variables(for example, how they change over time) but it says nothing about the mechanics of the relation, which is the causation. Let me investigate this example: the observation: Students who watch less TV have better grades. It's not like pirates v.s. global warming, it actually makes sense at first and if you are the minister of e…

He didn't say it proved causation, but it increases the probability of causation. Let's say you have several possible hypotheses:

Watching TV causes worse grades.

Watching TV has no effect on grades.

Something else causes watching TV AND worse grades.

Worse grades cause watching TV.

Watching TV causes better grades.

Etc, for all the other possible correlations between these three variables.

Assume all these are equally likely. That makes it about 1/7 chance that Watching TV causes worse grades (and there is equal chance of the exact opposite.) Now the study that watching tv is correlated with worse grades comes out. You can eliminate all but the first few hypotheses that predicted the correlation. Now the hypothesis "watching TV causes worse grades" has a probability of 1/3. Almost twice as likely. And the hypothesis that watching TV has any positive effect on grades has been completely eliminated.

This is why I'm bothered when people say "correlation doesn't imply causation!!!". No it doesn't, but it significantly raises the probability. If the other hypotheses aren't that likely to begin with (i.e. "cancer causes cellphones") then it should really bring that hypothesis to your attention.

Re: Misleading Graph Generator

#34
post #3

How is this graphs fault? The only fault the graph has is that it clearly displays the data, the problem is in the idea that is represented. It's formally called "Correlation does not imply causation" and it's fault of the person who is suggesting it. There is a famous satirical version of this too: http://en.wikipedia.org/wiki/File:PiratesVsTemp(en).svg It has nothing to do with graphs, graphs are are just visual to…

> There is a famous satirical version of this too: http://en.wikipedia.org/wiki/File:PiratesVsTemp(en).svg

What do you mean, satirical? There's a religion founded on this image alone!

Re: Misleading Graph Generator

#35
post #23

Earlier quoted context omitted.

I would not go into lengths to call correlation an evidence. It can be taken as a clue for further investigation because it also does not deny causation.

It is evidence for causation, in the Bayesian statistics meaning of "evidence". The first article I linked explains the process in detail.

Thank you for the links.

Re: Misleading Graph Generator

#36
post #28
post #14

This isn't a graph generator. It's just a graph. You can plug data into it and get a graph, I guess, but the same goes for Excel. A misleading graph generator that automatically matched concurrent data sets based on correlation would be quite interesting, but this isn't it.

So are you saying that the term "graph generator" is misleading here?

Ba dum cha

Re: Misleading Graph Generator

#37
I feel that many missed my point: It’s not to say causation never implies correlation, that there’s no common source or that latent factors are fiction.

In my experience many people believe something is true, just because of “math” or “data”. So, this is basically a variation of the joke that 73.37% of people put more trust into statistics, if the value isn’t rounded. Since you all critically discussed the topic, you are far beyond of what this can little project can teach you.

If you have suggestions on how to express this more clearly, please let me know.

Post reply on HN