Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Medical students are using popular research tool to pump out misleading studies

science.org

Post reply on HN
Print view View original

101–110 of 110 posts

Re: Medical students are using popular research tool to pump out misleading studies

#101

Post by zemvpferreira »

mmooss wrote: post #77
zemvpferreira wrote: post #42

As a former scientist it's embarrassing how easily the dismal state of science today could have been predicted decades before by applying Goodhart's Law, or any simple train of thought on incentives and moral hazard. Instead we chose to assume scientists collectively behave on a higher plane. No wonder the general public distrusts "the intelectual elite", we deserved it.

If the solutions are so simple, what are they? Yes, incentives are distorted, but what system - that does not rely on everyone being on a higher plane - would have better incentives? By your definition, every human endeavor is dismal and always has been - all are corrupted and flawed to some degree. Is there evidence that current science is more dismal than others or than before? You can look at any day in history an…

I didn't claim there's a simple solution, rather that the simplified system of publish-or-perish doesn't work. Personally I believe you can't replace a management layer that's aligned with the institution with a career progression based on metrics. Embrace complexity, let managers decide who should be promoted, sack or pay those managers based on their performance. Propagate onwards through the ownership chain.

The fact that Covid vaccines worked so well is precisely an endorsement of a hands-on, results-driven approach.

Re: Medical students are using popular research tool to pump out misleading studies

#102

Post by mmooss »

zemvpferreira wrote: post #101
mmooss wrote: post #77

Earlier quoted context omitted.

If the solutions are so simple, what are they? Yes, incentives are distorted, but what system - that does not rely on everyone being on a higher plane - would have better incentives? By your definition, every human endeavor is dismal and always has been - all are corrupted and flawed to some degree. Is there evidence that current science is more dismal than others or than before? You can look at any day in history an…

I didn't claim there's a simple solution, rather that the simplified system of publish-or-perish doesn't work. Personally I believe you can't replace a management layer that's aligned with the institution with a career progression based on metrics. Embrace complexity, let managers decide who should be promoted, sack or pay those managers based on their performance. Propagate onwards through the ownership chain. The f…

> Embrace complexity, let managers decide who should be promoted

You know better than I do about academia, but speaking generally about management that sounds like an issue not of complexity but of bias and corruption, which are prevelant enough in academia as I understand it: Faculty advisors have so much power they can crush careers, leading to abuse (including sexual harassment and abuse) and corruption.

For a manager to do it effectively, they need objective, quantified metrics as a major input. 'I can't manage what I can't measure.' And to manage managers based on performance, we need to measure their performance. Which would seem to bring us back to publishing, impact factor, etc.

> management layer that's aligned with the institution

Why do we care about alignment with the institution (you mean the university?)? Very broadly, institutional interests are often quite corrupt and cruel to people and society. It would seem to pervert research, encourage corruption, and harm scientists personally.

> The fact that Covid vaccines worked so well is precisely an endorsement of a hands-on, results-driven approach

What do you mean here? I don't know how the Covid vaccine research was managed, other than non-tenure-track faculty earned a Nobel!

> the simplified system of publish-or-perish doesn't work

The point is, maybe there's no better solution. That's often the case in practice. The way I think of it is that theory prioritizes truth over having a solution - no 'truth', no theory; that's science. Practice prioritizes having a solution - no 'truth', do the best you can; that's engineering, management, clinical healthcare, etc.

Re: Medical students are using popular research tool to pump out misleading studies

#103

Post by engcoach »

bigmattystyles wrote: post #78

I’m currently facing a severe health condition and I can’t help but ask various LLMs about it. They all will eventually offer solutions or avenues that sound promising or ‘easy’ when I know better, that the path ahead of me is hard, however they will non-chalantly offer a path forward they insist will work. The best part is when it cites commercial websites promotional statements as facts, though it will also misinte…

Try OpenEvidence instead

Re: Medical students are using popular research tool to pump out misleading studies

#104

Post by [deleted] »

fluidcruft wrote: post #74
tstactplsignore wrote: post #59

Earlier quoted context omitted.

This article is about MD doctors, not scientists (although there may be some MD-PhDs in it, and yes MDs can get research positions). MD doctors poll at extraordinarily high levels of trust, over almost any other professional group in the United States. So it really isn't correct to directly link this article's topic to "distrust". The effect you're talking about may exist in science, but this article is essentially a…

MD doctors (and even to a large extent DO doctors nowadays) are philosophically grounded on science. An MD fundamentally practices science. That used to be one of the key differences between MD and DO physicians, but science has been so successful at advancing the standard of care that DOs cannot ignore it anymore. That's just to say that MDs are expected to be fluent in science and it's not some arbitrary expectatio…

[deleted]

Re: Medical students are using popular research tool to pump out misleading studies

#106

Post by thomasfedb »

tremon wrote: post #37
thomasfedb wrote: post #10

Earlier quoted context omitted.

As a clinician-academic who published in The Lancet during medical school, I think this goes a bit far. Unfortunately student doctors are encouraged to publish whether or not they actually have an interest in research… but that shouldn’t discount the work of those who are genuinely engaged. But certainly we should always approach the literature critically, including the author list, journal of publication and its pee…

> student doctors are encouraged to publish whether or not they actually have an interest in research… but that shouldn’t discount the work of those who are genuinely engaged. How do you propose the interested public make the distinction between genuine engagement and forced encouragement? Isn't it the task of journals to make that distinction before publishing? I don't think you can fault the public for dismissing e…

It is my (unpopular I suspect) opinion that the contents of medical or other academic journals is not intended for consumption by the “interested public” as much as I can understand the desire to do so.

That’s not the same as saying that the interested public shouldn’t be able to learn about these things, but I do think they will often need assistance from those who are trained to read and discriminate this content.

And yes, journals should also do better, and peer review should be high quality and paid (you get, ultimately, what you pay for…)

Re: Medical students are using popular research tool to pump out misleading studies

#107

Post by dghlsakjg »

jmalicki wrote: post #89
fluidcruft wrote: post #74

Earlier quoted context omitted.

MD doctors (and even to a large extent DO doctors nowadays) are philosophically grounded on science. An MD fundamentally practices science. That used to be one of the key differences between MD and DO physicians, but science has been so successful at advancing the standard of care that DOs cannot ignore it anymore. That's just to say that MDs are expected to be fluent in science and it's not some arbitrary expectatio…

An MD does not practice science. Science is a process of discovering new knowledge through experiments. An MD is much more akin to an engineer that applies scientific knowledge (and yes, some proportion of both MDs and engineers do create new knowledge through experiments and thus do science, but it's not the baseline expectation).

Doctors run small experiments constantly. “Let’s see if this ailment responds to this treatment”.

They test theses constantly “These symptoms may be cancer, let’s test that theory with a biopsy”

Science can be local to a single patient.

Re: Medical students are using popular research tool to pump out misleading studies

#108

Post by bigmattystyles »

engcoach wrote: post #103
bigmattystyles wrote: post #78

I’m currently facing a severe health condition and I can’t help but ask various LLMs about it. They all will eventually offer solutions or avenues that sound promising or ‘easy’ when I know better, that the path ahead of me is hard, however they will non-chalantly offer a path forward they insist will work. The best part is when it cites commercial websites promotional statements as facts, though it will also misinte…

Try OpenEvidence instead

Seems pretty good but you need to be a health care pro, it locks you out rather fast. Makes sense though, can’t be given unlimited free tokens.

Re: Medical students are using popular research tool to pump out misleading studies

#109

Post by jmalicki »

dghlsakjg wrote: post #107
jmalicki wrote: post #89

Earlier quoted context omitted.

An MD does not practice science. Science is a process of discovering new knowledge through experiments. An MD is much more akin to an engineer that applies scientific knowledge (and yes, some proportion of both MDs and engineers do create new knowledge through experiments and thus do science, but it's not the baseline expectation).

Doctors run small experiments constantly. “Let’s see if this ailment responds to this treatment”. They test theses constantly “These symptoms may be cancer, let’s test that theory with a biopsy” Science can be local to a single patient.

It's not a scientific experiment if you don't have some form of control, just an observational study.

The lack of training physicians get in the scientific method is probably one of the biggest things holding humans back today.

Re: Medical students are using popular research tool to pump out misleading studies

#110

Post by internet_user »

cucumber3732842 wrote: post #41
internet_user wrote: post #24

admissions and residency matching give a lot of weight to "research output", aka publications. For residency, the two most important things are: 1) board scores. 2) research output. It's not uncommon to see 40-50 publications for competitive residencies. incentives, incentives, incentives.

>incentives, incentives, incentives. Exactly. The people at the John Hopkins Camel-Winston Center for Respiratory Research know that Camel and Winston are gonna stop cutting the checks if they don't get something they can trot out to support them when they get hauled in front of congress even if that's only a minority of the work output they funded. Even if nobody means to do evil the evil will be done just as a resu…

>Where is really gets spicy is when you have dueling funding sources. Where I went to school you had the environmental compliance industry funding the public policy research to say that nobody should be allowed to install a fencepost without paying their way through some hoops while the Kochs (through some indirection) were funding different research in the same department to say no akshually that compliance stuff is making us all poorer.

Amazing. How was that solved in practice?

Post reply on HN