Depression-Gene Studies Were Built on Shaky Foundations

However, sadly, many people believe in much more questionably "scientifically" proven things, largely due to how simple scientific proof appears due to things like the absolute acceptance of gravity on earth. Going with the religion analogy, there are many basic facts in religious scriptures which are true but this does not make many of the more questionable statements such as how exactly the earth will be destroyed and where you go when you die to be true.

I completely agree with you that science done properly is not a religion; however, that is simply not the case for most of us who either through lack of care or time will never go beyond seeing a news article about a certain scientific discovery and believing in it due to a supposed scientific breakthrough published in a fancy sounding journal.

As someone who works on depression GWAS, this isn't the end of finding causal genes for depression, it's the beginning.

In the next few years we're going to see a waterfall of new rare variants linked to disease, all which have a much higher chance of causing functional change.

As WGS comes online for association studies it will both validate and more deeply explore the genetic nature of every disease. It is going to be mindblowing.

The 'scientific method', of course, is that which is done by scientists.

I agree with what you're saying in general, but there's also problems in play in academics that are broader than just bad statistics. The article tries to convey the nature of this problem, but the "chasing a unreplicable effect" or "science sometimes takes awhile to work itself out" is just the tip of the iceberg.

This touches close to home because it's in my area of research, and for years I had many discussions with colleagues about this very same genetic effect and its problems. This SLC6A4 candidate gene research was not just a fluke of incompetence (unless by incompetence you mean much of an entire biomedical field of researchers), and it persisted wildly, with huge amounts of methodological research and money behind it.

Papers advocating for this type of research (and even more statistically problematic research) were published in Nature, with lots of methodological arguments, by established quantitative experts. This doesn't mean it's correct, just that by all superficial indications, it was solid. You had to question these authority figures, and a body of research, in good journals, the actual nature of the argument, and even then you were branded a naysayer or curmudgeon.

Even when people started questioning the effect, then you had people start advocating for more complex interactions (as intimated at in the article) that just amounted to unintentional (or intentional) data fishing and p-hacking with a theoretical cover.

When I started pointing out how problematic this all was to my colleagues, I had some of them outright explain to me that they thought it might be bunk, but it was popular, and if they found something significant, and it landed them a paper in a prestigious journal, why wouldn't they publish it? That is, you gotta publish what's popular because that's how you build an academic career.

I can't begin to explain all the shady stuff I've seen with the SLC6A4 effect being discussed in this article. Some of it was probably completely unintentional, and some of it probably amounts to conscious p-hacking and fishing.

The worst part about this, that's hard to convey, is that, yes, science probably works itself out eventually most of the time? But there's such a focus on popularity and prestige, fads, regardless of veridicality, and much less so on boring rigor and correctness, that entire careers can be made or broken on complete nonsense. The person who catapaults a completely empty finding to fad status has a career elevated permanently, to a nice named-chair-full professor position. The person who is trying to be rigorous, maybe even dispute the finding or disprove it? Much less clear what happens to them, and often it's a thankless task. That is, the fad makes a career, and even after the fad is discredited, people shrug and say "oh well, that person just happened to have a good idea that was wrong." The people who do the hard work of replicating it, disproving it? Well, that's not interesting or worthwhile to reward.

Academics is really broken, at least in many fields.

> As WGS comes online for association studies it will both validate and more deeply explore the genetic nature of every disease. It is going to be mindblowing.

But there is another layer of confusion, because WGS implies resequencing, and resequencing is only as good as your reference genome, and will distort results when the genome you're inferring is to far from the reference (reference bias).

The real mind bend will come when we have thousands or millions of whole genome de novo assemblies and we compare these to each other to do our GWAS. Only then are we going to have a hope of knowing what is actually causal in a genomic sense. Until then we remain in the land of association.

In nature, most adaptive variation appears to be large and structural. All the recent studies that have used whole genome assemblies to look at this have found the same thing. I would be surprised if this isn't the case for humans too. If it is, then much of current perspective on GWAS (both based on chips and WGS) will need to be rewritten.