61–70 of 105 posts
There were some stunning claims being made on Twitter last month based on a recently published study. Instantly skeptical, I dug into the methodology section and found this gem: "It should be noted that the results cannot be estimated using a physician fixed effect due to a numeric overflow problem in Stata 15 which cannot be overcome without changing the assumptions of the logit model." ... The sad part was they did…
Earlier quoted context omitted.
This is why knowledge of causal inference is essential. Causality is not established via tweaking a correlation or regression analysis, and we social scientists should know that.
Casual inference is the bottom most rug of what gives hard sciences its power. It is that we understand the objects we are manipulating at a much deeper level so we don't sound like idiots. Suppose that we take: g = ma A perfectly valid way to find experimental values for gravity at a location. But that doesn't mean that if we push an object really hard we increase the gravity in that location, or decrease it if we p…
I'm fine sounding like an idiot so long as my slope is increasing :)
Right, Judea Pearl covers this in [i]The Book of Why[/i].
There were some stunning claims being made on Twitter last month based on a recently published study. Instantly skeptical, I dug into the methodology section and found this gem: "It should be noted that the results cannot be estimated using a physician fixed effect due to a numeric overflow problem in Stata 15 which cannot be overcome without changing the assumptions of the logit model." ... The sad part was they did…
This guy overstates his case somewhat. Consider: "If the original study says an intervention raises math scores by .5 standard deviations and the replication finds that the effect is .2 standard deviations (though still significant), that is considered a success that vindicates the original study!" Why the exclamation point here? The replication study isn't magically more accurate than the original study. If the orig…
1. Effect size is the most important thing. The point of the study is (usually) to guide decisions. Sticking with the article's example, let's say combining both studies shows the increase is likely 0.35 standard deviations. Is the intervention still worth the cost? Is it still the best option?
2. If there's enough data (e.g., an observational study) or a good chance of omitted variables, there's going to be a "statistically significant" difference. No matter what's measured. I would bet my life's savings there's a statistically significant difference in profits of New York businesses depending on whether the owner's named Jim or Bob. A replication of the experiment with all Jim and Bob businesses in another state would also guarantee significance. So it's a coin toss whether the second study would "successfully replicate" the same direction of effect.
It seems to not be based on actual replication results, but predicted replication results? But then the first chart isn't even predictions from the market, but just the author's predictions?
The author clearly has a real hatred for practices in the social sciences. But I don't see any actual proof of the magnitude of the problem, the article is mostly just a ton of the author's opinions.
Is there any actual "meat" here than I'm missing? Or is all this just opinions based on further opinions?
Earlier quoted context omitted.
You're right that I'm thinking of very basic criticisms. In particular, there's zero evidence that humans aren't p-zombies [0] and no definitive rejection of the Dodo Bird hypothesis [1]; in combination, this suggests that psychologists are both wrong to imagine that there's anything interesting going on inside of a human's mental states, and also wrong to try to classify those mental states into appropriate and inap…
There's also zero evidence that humans are p-zombies, and plenty of criticism that p-zombies don't work as a thought experiment. It's a pretty big leap to throw away consciousness on the back of equal outcomes in psychotherapy. There are partial rejections of the Dodo bird verdict in your link. The Cartesian theatre doesn't account for the mind's ability to imagine things that never were. As soon as you account for t…
While it's true that there's no empirical evidence within humans for the question of p-zombies, Occam's Razor neatly shows that the world without souls is the simpler one; both worlds look just like ours, but one of them requires all of these additional unfalsifiable unverifiable claims about souls and consciousness and inner experiences and etc.
The Dodo Bird's strength comes from the multitude of different models of therapy which have existed over the decades. We know, from history, that the memes of psychotherapists leaked out into popular culture and slowly altered how we talk about thinking. Nonetheless people are more neurotic (more diagnosed with mental disorders) than ever before! So the memes of psychotherapy do not on their own decrease mental disorders. People will look back on our current decade and think how silly we were to focus on "medications", "hormones", "chemical imbalances", "neurotransmitters", etc. just like we now think it's silly to focus on "repressed memories", "dream interpretation", "hysteria", "oedipal urges", etc.
Humans cannot imagine anything completely novel. Every thought which a human ever has is unoriginal and hopelessly tied to the human experience. This is known as the anthropic bias and has been known about for millennia. If you believe in souls, you have an uphill battle in terms of evidence, including here.
Please stop believing in souls. There's no evidence for it and it turns your arguments to mush.
Is there a similar study done on the physical sciences? I’m getting a bit of holier-than-thou feeling from this article. Edit: from all this talk of reproducibility, I wonder what percentage of cutting edge ML research is reproducible (either from lack of public training sets / not enough compute)
There were some stunning claims being made on Twitter last month based on a recently published study. Instantly skeptical, I dug into the methodology section and found this gem: "It should be noted that the results cannot be estimated using a physician fixed effect due to a numeric overflow problem in Stata 15 which cannot be overcome without changing the assumptions of the logit model." ... The sad part was they did…
> Publishing your own weak papers is one thing, but citing other people's weak papers? This seemed implausible... This is practically required by reviewers and editors. If you wade into a topic area, you need to review the field and explain where you fit in, even though you know full well many of those key citations are garbage. You basically need to pay homage to the "ground breakers" who claimed that turf first, ev…
You basically need to pay homage to the "ground breakers" who claimed that turf first, even if they did it via fraud. Even referring to it as “science” is fraudulent. Testable theories and repeatable outcomes, anyone? Time this whole field was defunded.
Most new social science research is wrong. But the research that survives over time will have a higher likelihood of being true. This is because a) it is more likely to have been replicated, b) its more likely to have been incorporated into prevailing theory, or even better, have survived a shift in theory, and c) is more likely to have informed practical applications or policy, with noticeable effect.
Physics and other hard sciences have a quick turnaround from publication to "established knowledge". But good social science is Lindy. So skip all the Malcolm Gladwell books and fancy psych findings, and prioritize findings that are still in use after 10 or 20 years.