How Much Science Is Fake? • Hacker News Board

Re: How Much Science Is Fake?

#11

Post by derbOac » Mon, Oct 21, 2024, 1:36 PM UTC

Although identifying outright fraud is important, I think collectively the bigger, more systemic damage is done by other things that are more subtle: material that falls in a gray area (of questionable research practices, or even less obvious than that), studies that repeat what is obvious at some point (and I don't mean replication per se, which is important, but something more basic than that), the way credit is asserted and inferred, and so forth and so on.

Identifying fraud is critical but I worry a bit that baserates of such statistics get processed as an estimate of "what proportion of science is problematic", when the problems are broader than that.

Re: How Much Science Is Fake?

#12

Post by Dalewyn » Mon, Oct 21, 2024, 1:36 PM UTC

mbreese wrote: post #6

I don’t think the issue is how much of “science” is fake. It’s how you define “science”. There have always been lower quality work and journals that are inherently given lower weight in the scientific community. Many of these lower quality papers just aren’t considered by working scientists. It’s a bit like email… once you start seeing a significant amount of spam, you just get used to ignoring it. This is a feature…

>public trust in science.

The argument that science should be trusted (believed) is insane, the whole point of science is questioning everything and deferring skepticism only in lieu of anything that can disprove.

Trusting science is just another form of religion, especially since science ("science"?) is often positioned as the rival to religion.

Re: How Much Science Is Fake?

#13

Post by Bellf » Mon, Oct 21, 2024, 1:40 PM UTC

I’m a computer science student at a major university in my country, and I completely agree. I’ve come across many papers that seem questionable, and there are powerful individuals in the scientific community who only accept work from a select group. It feels like these people are building their reputations on what often appears to be fake or unsubstantiated research.

Re: How Much Science Is Fake?

#14

Post by arminiusreturns » Mon, Oct 21, 2024, 1:44 PM UTC

Lots. I am just a graybeard sysadmin type, but I got taught how to read papers critically by a world-class geneticist because we were taking papers to implementation in prod. The biggest takeaway for me was that citation padding networks and lack of reproducibility were the biggest issues and were prevalent, and this was in the top journals, you can't just say this stuff happens in the "lesser" ones!

Re: How Much Science Is Fake?

#15

Post by cynicalpeace » Mon, Oct 21, 2024, 1:48 PM UTC

40-60% of scientific studies are not replicable, therefore fake as far as I'm concerned.

That's why "show me the study" is such a midwit line of reasoning.

We need to go back to thinking more with first principles, received wisdom, and personal experimentation.

There's a place for scientific studies, but it's gone so far that many won't take an obvious point without a study behind it.

There are many such examples that are quite obvious but are hard to prove: processed foods are bad, screen time is bad for kids, starship won't fall on a shark etc. But people love to poke holes in obvious truths, especially in well educated spaces like HN.

Re: How Much Science Is Fake?

#16

Post by davisoneee » Mon, Oct 21, 2024, 1:48 PM UTC

Dalewyn wrote: post #12

mbreese wrote: post #6

I don’t think the issue is how much of “science” is fake. It’s how you define “science”. There have always been lower quality work and journals that are inherently given lower weight in the scientific community. Many of these lower quality papers just aren’t considered by working scientists. It’s a bit like email… once you start seeing a significant amount of spam, you just get used to ignoring it. This is a feature…

>public trust in science. The argument that science should be trusted (believed) is insane, the whole point of science is questioning everything and deferring skepticism only in lieu of anything that can disprove. Trusting science is just another form of religion, especially since science ("science"?) is often positioned as the rival to religion.

Nothing is _ever_ certain or completely specified in science (or elsewhere)... and society is going to move forward, so what level of 'evidence' is sufficient for you? This is going to come across as antagonistic, but...what's the alternative? "The argument that science should be trusted is insane..." ... so are we expected to twiddle our thumbs and do nothing for the rest of time?

I'm assuming you trust at some point? Have you ever flown? We have theory, we have evidence, and a high level of _justified true belief_ (to refer to recent HN posts) for how flight works, but perhaps all _these_ replications are just fortuitous and 'fake'? We 'trust' Newtonian physics enough for society to _act_ and _progress_ ... and then we quantum comes along.

Re: How Much Science Is Fake?

#17

Post by WarOnPrivacy » Mon, Oct 21, 2024, 1:51 PM UTC

Dalewyn wrote: post #12

mbreese wrote: post #6

I don’t think the issue is how much of “science” is fake. It’s how you define “science”. There have always been lower quality work and journals that are inherently given lower weight in the scientific community. Many of these lower quality papers just aren’t considered by working scientists. It’s a bit like email… once you start seeing a significant amount of spam, you just get used to ignoring it. This is a feature…

>public trust in science. The argument that science should be trusted (believed) is insane, the whole point of science is questioning everything and deferring skepticism only in lieu of anything that can disprove. Trusting science is just another form of religion, especially since science ("science"?) is often positioned as the rival to religion.

> The argument that science should be trusted (believed) is insane,

> the whole point of science is questioning everything and deferring skepticism only in lieu of anything that can disprove.

Childhood learning is about increasing trust of science thru experimentation of gravity and energy (heat/cold, momentum, friction, etc).

Without that trust, there would be no learning. Perhaps that is the point of unreasonable skepticism.

Re: How Much Science Is Fake?

#18

Post by cjonas » Mon, Oct 21, 2024, 1:52 PM UTC

Dalewyn wrote: post #12

mbreese wrote: post #6

I don’t think the issue is how much of “science” is fake. It’s how you define “science”. There have always been lower quality work and journals that are inherently given lower weight in the scientific community. Many of these lower quality papers just aren’t considered by working scientists. It’s a bit like email… once you start seeing a significant amount of spam, you just get used to ignoring it. This is a feature…

>public trust in science. The argument that science should be trusted (believed) is insane, the whole point of science is questioning everything and deferring skepticism only in lieu of anything that can disprove. Trusting science is just another form of religion, especially since science ("science"?) is often positioned as the rival to religion.

If you're not an expert in something, you should probably listen to those who are...

The general public does not have the time to study climate models and review raw data on vaccine efficacy.

Without overall "trust in science" (or perhaps "trust in the scientific community") we lose the ability to form data driven policy.

The exception being when there is monetary value to be gained by fooling the public. IOW, be skeptical of studies funded by special interest, but

"Do not be so open-minded that your brains fall out."

Re: How Much Science Is Fake?

#19

Post by davisoneee » Mon, Oct 21, 2024, 1:53 PM UTC

cynicalpeace wrote: post #15

40-60% of scientific studies are not replicable, therefore fake as far as I'm concerned. That's why "show me the study" is such a midwit line of reasoning. We need to go back to thinking more with first principles, received wisdom, and personal experimentation. There's a place for scientific studies, but it's gone so far that many won't take an obvious point without a study behind it. There are many such examples tha…

I think describing it as 'fake' is rather antagonistic. You could just as easily say 'poor quality' or 'unjustified'. I would say that more accurately describes the situation.

It could be...

    - unjustified and sloppy (not 'fake', but also not considered reliable evidence)
    - unjustified and malicious  (this i would consider 'fake')
    - unjustified and gamed (again, 'fake')
    - ...or just unjustified and under-specified (and would result in 'true' results if the conditions for replication were better studied and defined)

Lots of people like to think themselves smart for following 'first principles'...and then often end up falling in the same ditches. First principles + received wisdom is a bit of a contradiction... if it's 'wisdom' rather than evidence, you are skipping your principles to go with the received starting point...

Re: How Much Science Is Fake?

#20

Post by Dalewyn » Mon, Oct 21, 2024, 1:54 PM UTC

davisoneee wrote: post #16

Dalewyn wrote: post #12

Earlier quoted context omitted.

>public trust in science. The argument that science should be trusted (believed) is insane, the whole point of science is questioning everything and deferring skepticism only in lieu of anything that can disprove. Trusting science is just another form of religion, especially since science ("science"?) is often positioned as the rival to religion.

Nothing is _ever_ certain or completely specified in science (or elsewhere)... and society is going to move forward, so what level of 'evidence' is sufficient for you? This is going to come across as antagonistic, but...what's the alternative? "The argument that science should be trusted is insane..." ... so are we expected to twiddle our thumbs and do nothing for the rest of time? I'm assuming you trust at some poin…

>so what level of 'evidence' is sufficient for you?

Enough evidence such that no evidence to the contrary can stand.

We agree that liquid water is wet because we can't find anything (yet) that says liquid water is not wet, it's simple as that and science is ultimately as simple as that.

>Have you ever flown?

Yup, I'm a frequent flyer actually and fly more or less every month or two for business (and pleasure once the business is done, but that part's a secret).