Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Unbearable Asymmetry of Bullshit

quillette.com

Post reply on HN
Print view View original

121–130 of 163 posts

Re: The Unbearable Asymmetry of Bullshit

#121

Post by Lawtonfogle »

apo wrote: post #76

>... He Who Shall Not Be Named predictably rejects all of the studies that do not support his position as being “fatally flawed,” or as having been “refuted by experts”—namely, by himself and his close collaborators ... This speaks to part of the problem - the undue weight that non-scientists place on expert opinion. Trained scientists see appeal to authority arguments for what they are: bullshit. I see this most fre…

Part of the way through earning a degree in social science I found I had acquired an amazing power. I can take almost any research paper in a social science and poke enough holes in it that anyone who doesn't like the results will no longer accept it.

Re: The Unbearable Asymmetry of Bullshit

#122

Post by breischl »

noelsusman wrote: post #111
breischl wrote: post #78

Earlier quoted context omitted.

>>the undue weight that non-scientists place on expert opinion Speaking as a non-scientist, I can recognize an appeal to authority probably just as well as a scientist. But having recognized one, what do I do? I lack the training, knowledge, and time necessary to evaluate the research directly. I can choose to only trust studies that are peer reviewed, or in major journals, or backed by whatever relevant government b…

The alternative is consensus. As part of my research I found that a particular method of estimating disease prevalence in small geographic areas performed better than this other method when tested on real data from schools. The tendency of the general public would be to look at that study and say the first method is better than the second method (assuming the general public would care at all, which they don't). That'…

Consensus seems to be the best available alternative, but it still doesn't seem to be very good in many cases. As the article pointed out it is not difficult to manufacture enough bullshit that it looks like a "competing consensus." Prime examples of that being the anti-vaccination movement, climate change deniers, and the pro-smoking studies from cigarette companies.

Counting up and evaluating all the studies is itself a time-consuming task, hence the existence of meta-analyses. But then you get into duelling meta-analyses, and you have to choose which one(s) to trust, and you're back to expert opinion and appeals to authority.

Re: The Unbearable Asymmetry of Bullshit

#123

Post by worik »

compactmani wrote: post #62
tines wrote: post #46

Earlier quoted context omitted.

The point is that nobody knows if it's reproducible, because nobody's attempting to reproduce it.

Yes that may be true. Instead of trying to reproduce it, they attempt to build upon it. The projects which successfully build on previous projects become original research articles and the cycle continues.

No. That completely misrepresents the problem.

tl;dr A false technique can be described and it can be hard or impossible to detect the technique is flawed by using it

For example the paper at http://www.jstor.org/stable/222500 describes a method of using the stationary bootstrap to eliminate data snooping bias in studies of "technical analysis" in finance.

Published in the Journal of Finance in 1999 at the time I worked with it in 2010 it had been cited over 500 times.

The proof in the paper is inscrutable. I could find no one who could explain or verify the proofs at my institution.

I attempted to reproduce the results in the paper which is where the problems started. The authors did not give enough information to do this in all cases, but in some cases I was able to reconstruct the algorithms.

They did not perform as described. Of the five algorithms I could reproduce (from memory) one of them worked roughly as described and two others were not completely hopeless.

Looking more closely I realised that the original authors completely disregarded important factors in the implementation of the techniques described by the algorithms. Transaction costs. Allowing for transaction costs (a difficult but not impossible task) the effects noticed by the authors disappeared completely.

Looking even closer I examined the assumptions behind "White's Reality Check" that the paper relied on the Stationary Bootstrap by Politis and Romano from 1994. But financial returns are not stationary. Not at all.

So dodgy logic, misuse of statistics, irreproducible experiments, ignoring important aspects of the data and I suspect wishful thinking add up to a paper that is comprehensively false. Cited hundreds of times and used many times to verify other results.

taking a huge risk here is where my research is: https://ourarchive.otago.ac.nz/bitstream/handle/10523/4346/T... Section 5.12.1 "Bootstrapping and Other False Discoveries"

Re: The Unbearable Asymmetry of Bullshit

#124

Post by sevensor »

Another tactic I've seen from the Lord Voldemorts I know is to cite loads of references that aren't readily available online. Such a citation can be used to prop up _any_ argument, whether or not the citation actually supports the argument, or even has any bearing on it at all. It's the same trap, though. To disupte the citation, you have to wait months for an inter-library loan, read the cited work in detail, and then decide what it really has to say about the argument.

I personally fell into this trap, not because I was trying to refute something, but because I was trying to back up one of my own assumptions and I found that Lord Voldemort was citing Obscure Reference X to back up the same assumption. The joke was on me when I actually tracked down Obscure Reference X in the 30-years-out-of-print proceedings of a symposium on Y. Obscure Reference X had nothing at all to say about my assumption! Needless to say, I no longer trust Lord Voldemort or anyone who publishes with him.

Re: The Unbearable Asymmetry of Bullshit

#125

Post by simplemath »

sevensor wrote: post #124

Another tactic I've seen from the Lord Voldemorts I know is to cite loads of references that aren't readily available online. Such a citation can be used to prop up _any_ argument, whether or not the citation actually supports the argument, or even has any bearing on it at all. It's the same trap, though. To disupte the citation, you have to wait months for an inter-library loan, read the cited work in detail, and th…

Why can't we name these people? Libel?

Pastebin?

Fuck these frauds.

Re: The Unbearable Asymmetry of Bullshit

#126

Post by fapjacks »

privong wrote: post #106
cli wrote: post #98

Earlier quoted context omitted.

What are the differences between postdocs, research assistants, research scientists and visiting scientists?

> What are the differences between postdocs, research assistants, research scientists and visiting scientists? These are some working definitions, based on my experiences in astronomy: Postdocs : researchers with a a PhD who generally have fixed-term contracts. Generally these positions are full-time research, though some may include teaching components. Research assistants : researchers, typically without PhDs, work…

If I had another lifetime to live, I would pour my heart and soul into astronomy. Life is too short.

Re: The Unbearable Asymmetry of Bullshit

#127

Post by fapjacks »

Chris2048 wrote: post #75
gavanwoolery wrote: post #55

' Instead, “It’s about a methodology for investigation, which includes, at its core, a relentless drive towards questioning that which came before.” You can both “love science,” he concludes, “and question it.” ' I've noticed that questioning some scientific finding often makes people think I am either anti-science, anti-intellectual, conservative, religious, or any combination thereof (I am none of these). To state…

If the science has "social" before it, it always needs to be questioned - too much political interference. The entire field is low in actual results, high in political tooling..

It seems that's the case with more than just science, but possibly a good rule to follow with anything.

Re: The Unbearable Asymmetry of Bullshit

#128

Post by return0 »

I heard a well known neuroscientist suggest that studies should be required to be replicated by at least another lab before being published/ established. That may sound impractical for some fields, but especially in biology/neuroscience most projects are small enough to make this feasible. Is it worth the money? I would say absolutely, not only for the validation of the science, but for the shift in culture, to finally stop designing studies chasing minute (but statistically significant) effects just to make another publication.

I think its also about time we have post-publication peer review and about time scientists get off their high horses and start responding to it.

Re: The Unbearable Asymmetry of Bullshit

#129

Post by argonaut »

Balgair wrote: post #119
bunderbunder wrote: post #109

Earlier quoted context omitted.

The problem is that nobody's doing replication work in undergrad, either, nor does it happen in Master's programs. I do think it makes sense to stick with the PhD meaning you're a world class expert in some area, but if so then we need to adjust our expectations for what Master's level work means in the sciences. Right now it seems to just represent a hurdle you need to whiz past on your way to the PhD.

Its really dependent on the department, but yeah, the MS is now the HS degree for a lot of places. Especially in Engineering. Good luck trying to get hired with only a BS. Credential creep is real. More here: https://en.wikipedia.org/wiki/Credentialism_and_educational_...

In CS, a Bachelors is enough for most (if not all) jobs.

Re: The Unbearable Asymmetry of Bullshit

#130

Post by ktRolster »

bunderbunder wrote: post #109
Balgair wrote: post #107

Earlier quoted context omitted.

That is nearly the opposite of what the PhD was designed to do. Modern academic training comes out of the church and the old guilds of middle Europe and is still in use today in many fields (chefs and plumbers to name a few). The Bachelor's degree is loosely similar to an apprentice's role. The young boy (they were almost exclusively male) worked in a shop or with a priest for some time. He learned the trade, the too…

The problem is that nobody's doing replication work in undergrad, either, nor does it happen in Master's programs. I do think it makes sense to stick with the PhD meaning you're a world class expert in some area, but if so then we need to adjust our expectations for what Master's level work means in the sciences. Right now it seems to just represent a hurdle you need to whiz past on your way to the PhD.

Mainly because no one is willing to fund it. (Don't look at me, I haven't funded any of it either!)
Post reply on HN