Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Depression-Gene Studies Were Built on Shaky Foundations

theatlantic.com

Post reply on HN
Print view View original

71–80 of 91 posts

Re: Depression-Gene Studies Were Built on Shaky Foundations

#71

Post by rossdavidh »

lisper wrote: post #69

> Between them, these 18 genes have been the subject of more than 1,000 research papers, on depression alone. And for what? If the new study is right, these genes have nothing to do with depression. The article doesn't actually say anything about what these 1000 research papers actually claimed to show. It's entirely possible that most of them were negative results, there's no way to tell from the text of the article…

It worked as it was supposed to...eventually. Instead of double-checking the initial results, hundreds of research papers by many research teams looked into questions that would be very important, if the genes in question were connected to depression.

Sure, it is great that it's not (quite) religious dogma, and it eventually gets revisited. But, a couple decades worth of work and ~1000 research papers, many of which took place AFTER the earlier 2005 paper that called into question the link to depression, is by no means what we should shoot for.

More fundamentally, there was nothing in the system that motivated anyone to do replication of the original link. The problem is not that the original link turned out to be spurious, and it's not that anyone did anything unethical. More worrisomely, nobody appears to have done anything unethical, but because of the systemic incentives for research grants, tenure, etc., replication of the foundational result wasn't attempted in a serious enough way until many times that much work had been done on something spurious.

Re: Depression-Gene Studies Were Built on Shaky Foundations

#72

Post by jerf »

fyrabanks wrote: post #67
jerf wrote: post #62

Earlier quoted context omitted.

I'd say it depends on the topic, and it depends on the standard you hold the papers in question to. For instance, I can find you a paper showing almost anything you like in the field of nutrition in terms of what people should eat. It'll even hold up in the abstracts. Whether it holds up beyond that can get complicated, though; I've lost count of things like "a study that X intervention is good in 20 rats over 5 days…

Can you find me a paper that says eating a Sausage McMuffin every morning won't shorten my lifespan?

In those exact words no, but if you break down the nutritional content of the Sausage McMuffin, yes, probably. A lot of studies are done on rats where even the rats eating "quality" food in the study are still eating things much lower in quality than a fast food muffin. That doesn't even sound particularly challenging.

To be honest, if you're trying to make my point sound absurd by exaggeration, you've shot way too low. The mainstream nutritional view would be that a Sausage McMuffin every morning on its own isn't going to be a particularly bad thing. You need to be a lot more specific with an overall diet to be a problem. You should have asked something like whether you could find a study about eating nothing but lard is available.

To which my reply would be that the principle of charity would make it clear that in general I meant any semi-realistic nutritional view is available, not that there are studies that prove humans can be healthy on a diet of rocks and asbestos. It really isn't to anyone's advantage if I also have to append to my little post a complete discussion of what is and is not within the boundaries of nutritional theories that have been studied.

I mean... is that... really... what you want...?

Re: Depression-Gene Studies Were Built on Shaky Foundations

#73

Post by bobcostas55 »

mindgam3 wrote: post #61

“Nor... should his work be taken to mean that genes don’t affect depression. They do, and with newer, bigger studies, researchers are finally working out which ones do.” This part really needs some context/sources. Anyone know which studies he’s talking about?

Here's one: https://www.nature.com/articles/s41593-018-0326-7

Re: Depression-Gene Studies Were Built on Shaky Foundations

#74

Post by antidesitter »

jerf wrote: post #72
fyrabanks wrote: post #67

Earlier quoted context omitted.

Can you find me a paper that says eating a Sausage McMuffin every morning won't shorten my lifespan?

In those exact words no, but if you break down the nutritional content of the Sausage McMuffin, yes, probably. A lot of studies are done on rats where even the rats eating "quality" food in the study are still eating things much lower in quality than a fast food muffin. That doesn't even sound particularly challenging. To be honest, if you're trying to make my point sound absurd by exaggeration, you've shot way too l…

> what is and is not within the boundaries of nutritional theories that have been studied

I don’t think this is clear to the average person at all.

Re: Depression-Gene Studies Were Built on Shaky Foundations

#75

Post by joycian »

gpm wrote: post #64
joycian wrote: post #60

Earlier quoted context omitted.

Not really true. Tesla is at the forefront of applying machine learning in real-world settings. It's definitely not unrelated to science, in my opinion. If autonomous driving in 2020 is on the timetable, Karpathy (Head of AI) is probably confident that it is possible. Musk is very aggressive on timelines (but always wraps it in "it is probable that", which to him is cautious but to which translates to almost certainl…

There's probably some real science happening behind the scenes, experimenting with ML algorithms (Karpathy is certainly up to it). But the challenge of getting it to work sufficiently well in the real world isn't science, it's engineering. Meeting schedules isn't science, it's management. And so on. "Believing in science" has nothing to do with believing musk will or will not succeed, and I think you would find the v…

I don't see the line between science and engineering as clearly as you do, apparently. Is CERN a science or engineering project? Drug design? Genuinely curious what qualifies as science. Seeing some articles, it's not the quality. Applied vs. fundamental also seems like a difficult line to actually draw.

Edit: Especially in ML, a large part of the research is done in companies.

Re: Depression-Gene Studies Were Built on Shaky Foundations

#77

Post by asdff »

eggie wrote: post #49
searine wrote: post #43

As someone who works on depression GWAS, this isn't the end of finding causal genes for depression, it's the beginning. In the next few years we're going to see a waterfall of new rare variants linked to disease, all which have a much higher chance of causing functional change. As WGS comes online for association studies it will both validate and more deeply explore the genetic nature of every disease. It is going to…

> As WGS comes online for association studies it will both validate and more deeply explore the genetic nature of every disease. It is going to be mindblowing. But there is another layer of confusion, because WGS implies resequencing, and resequencing is only as good as your reference genome, and will distort results when the genome you're inferring is to far from the reference (reference bias). The real mind bend wi…

GWAS is nothing more than correlations and by themselves aren't very remarkable. What is remarkable is when someone does a functional analysis on the variant they found in their GWAS and is able to describe the mechanism that it uses to affect the disease or phenotype. The way you do that isn't more sequencing, but with model systems that you can easily manipulate in the lab.

GWAS also doesn't compare to an arbitrary reference sequence. Some good numbers for a GWAS are >1000 cases and >2000 controls from the same population. You have to match populations, or else all your association study is going to find is the differences between east asians and europeans, for example. You need a lot of samples to get enough stats power to even see these rare variants.

Re: Depression-Gene Studies Were Built on Shaky Foundations

#78

Post by asdff »

gnoppa wrote: post #39
icegreentea2 wrote: post #27

Earlier quoted context omitted.

Let's not throw the baby out with the bath water. No doubt there's a lot of sketchy stuff going on, but also the fields are very wide with lots of "verticals" going on. Just cause one vertical might be built on a shakey foundation, doesn't mean all of them are (and the converse is also true).

You might be right that not everything in those fields is bad, but most is. Nevertheless, what is so terrible is that all the money is put into research (gene mutation causing cancer) even though it has been proven wrong many years ago. That becomes very clear after reading Thomas Seyfried's work (cell fermentation causing cancer). Sadly, academics have the incentives to publish papers in prestigious journals that ma…

>You might be right that not everything in those fields is bad, but most is.

Citation needed.

Re: Depression-Gene Studies Were Built on Shaky Foundations

#79

Post by gpm »

joycian wrote: post #75
gpm wrote: post #64

Earlier quoted context omitted.

There's probably some real science happening behind the scenes, experimenting with ML algorithms (Karpathy is certainly up to it). But the challenge of getting it to work sufficiently well in the real world isn't science, it's engineering. Meeting schedules isn't science, it's management. And so on. "Believing in science" has nothing to do with believing musk will or will not succeed, and I think you would find the v…

I don't see the line between science and engineering as clearly as you do, apparently. Is CERN a science or engineering project? Drug design? Genuinely curious what qualifies as science. Seeing some articles, it's not the quality. Applied vs. fundamental also seems like a difficult line to actually draw. Edit: Especially in ML, a large part of the research is done in companies.

Science is about discovering things via experiments and observations about the world, engineering is about making things that work. There is a tiny bit of overlap.

CERN is a gigantic engineering project used to do a bit of science. Experimenting with different concrete mixes to find one with a set of qualities is science used to let you do some engineering. OpenAI's dota bots are the sort of thing that might fall in the overlap of both discovering things and making things that work.

Maybe more to the point, "believing in science" means "believing that those experiments and observations reveal true facts", which has nothing to do with whether or not we believe Musk will succeed at his self driving car ambitions.

Re: Depression-Gene Studies Were Built on Shaky Foundations

#80

Post by TheOperator »

lm28469 wrote: post #15
wallace_f wrote: post #11

Earlier quoted context omitted.

It's amazing how convincing statistics are--even for supposed experts--and not just for the reason you elaborate on. In 2019, they really are the best way to lie.

Science became a new religion. Instead of saying "shut up god made it that way" you say "shut up science says it's that way". You can find hundred of studies to support your arguments, no matter what they are, it's even easier if you don't actually read the studies and stop at the summaries made by popular medias.

Science has certainly become a fantastic way to bullshit people. Combined with how science is funded now its created a weird status quo where things that wealthy interest groups want to be true will tend to have more studies which point to the desired conclusion. Something that's true can be less scientifically backed than something that's false if the truth isn't in peoples financial interests.

I also see a weird phenomena where I'll see people who are wrong like flat earthers running DIY scientific experiments. Yet this crowd seems to be thought of as generally anti-science when they're probably more scientifically curious than the actual population. Being "scientific" means subscribing to a certain set of consensus beliefs among scientists. You can not use science at all yourself directly and be considered a scientific person so long as you just believe in certain things.

Post reply on HN