Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I tried to report scientific misconduct. How did it go?

crystalprisonzone.blogspot.com

Post reply on HN
Print view View original

171–180 of 363 posts

Re: I tried to report scientific misconduct. How did it go?

#171

Post by shatnersbassoon »

ogogmad wrote: post #9
ogogmad wrote: post #7

"Social psychology" has had a serious problem with scientific misconduct and statistical incompetence. Is this related in some way to that subject area?

What about the rest of academic psychology for that matter? Is this endemic to that field?

I decided not to pursue an academic career after completing a Ph.D in Experimental Psychology. My main issue with it was:

1. You shouldn't know a priori whether you will be able to reject your null hypothesis (otherwise what is the point of doing the experiment). So what you need is luck, luck that your hunch turned out to be true.

2. If careers live and die by published results, then those who are lucky with finding significant effects early on in their career will win out.

3. Running a well-controlled experiment at scale is difficult in a way that I haven't found matched in the tech fields I have been in since leaving academia. I mean mega-hassle difficult.

4. You therefore have an incredible amount riding on the outcome of that experiment, because of its enormous opportunity cost.

5. The likelihood of being caught faking data is low (especially if you are halfway-competent which this researcher clearly is not).

6. The penalties of being caught faking data (as set out in this article) are relatively low.

7. The payoff of getting away with faking data can include a lot, up to and including a high profile academic career and tenure.

8. So from a game theoretic perspective, it's almost inevitable that quite a few people at the top have faked their way there.

This is not to say that good work is not being done - there is some amazing work out there. I just think that like athletes taking steroids before they reach the big leagues, many academics succumbed to temptation to get an edge in order to get to the top.

See the Dutch Social Psychology scandal for more on this.

Re: I tried to report scientific misconduct. How did it go?

#172

Post by winstont »

read_if_gay_ wrote: post #99
fireattack wrote: post #89

Earlier quoted context omitted.

I agree that's unnecessary detail.

It's not about race, it's about the quality of the academic system, which is bad in many countries. I suppose GP intended it to compound - as in the guy was sketchy per se, and from a sketchy place.

> the academic system, which is bad in many countries. I suppose GP intended it to compound - as in the guy was sketchy per se, and from a sketchy place.

is it good here? is here considered less sketchy?

Re: I tried to report scientific misconduct. How did it go?

#173

Post by DarkWiiPlayer »

Vinnl wrote: post #166
DarkWiiPlayer wrote: post #159

Earlier quoted context omitted.

Question is, why don't scientists just put everything on public platforms (read: github) and call it a day? Is it only a matter of funding, or do other factors also play a role in that?

Because nobody reads it there and, more importantly, funders don't recognise the work you've done there. The "prestige" (as indicated by the scientific-looking but mostly inaccurate "Impact Factor") of the journal you publish in determines how good they think your work is. I wrote about that a while ago here: https://medium.com/flockademic/the-ridiculous-number-that-ca...

> Because nobody reads it there

That's a problem that would fix itself the moment most useful research was mainly available on such platforms.

> more importantly, funders don't recognise the work you've done there

Once again, that sounds like mostly a problem that would disappear if a large migration to open platforms was to happen.

----

So it seems the main poroblem seems to be that there's no incentive to be among the first to make the move? IIRC it's often the journals that don't want content to be published elsewhere, so I guess just doing both is also not that simple.

Re: I tried to report scientific misconduct. How did it go?

#174

Post by Radim »

dnautics wrote: post #128
TeMPOraL wrote: post #123

Earlier quoted context omitted.

I think the point of GP's story is that this isn't just one or two bad apples here and there, but it's endemic in that domain - and most likely in others too (I'm leaning to believe that; it's not the first story like this I've read in recent years). Being an endemic problem means you have to switch your assumptions; when reading a random scientific paper, you're no longer thinking, "this is probably right, but I mus…

it's endemic in biology, and it's endemic in chemistry (I had feet in both sides). The sentiment you wrote in the last sentence is exactly what I feel whenever I read a paper, hit it on the nail. The crazy thing, is that the honest scientists are working at middling university. It is worse the higher up you go. I have had the opportunity to work in a upper-midrange research university [time-] sandwiched between two v…

While the "host" (people who pay, often with minimal decision power over their resources) turns a blind eye, "parasites" (cheaters who profit disproportionately) proliferate. Is that really so surprising?

When somebody else foots the bill, it's feast time!

To be clear, I'm with you. Also a PhD-turned-industry, for much the same reasons. But I realize what you describe is a completely rational strategy. The options always come down to:

1) Try not to be a host – if you have the wherewithal

2) Try to be a parasite – if you have the stomach

3) Suck it up & stay salty – otherwise. You can call it a balance, equilibrium, natural order of things – whatever helps you sleep at night.

Take your pick and then choose appropriate means. Romantic resolutions and wishful thinking – kinda like Atlas Shrugged solution for option 1) – rarely work.

Re: I tried to report scientific misconduct. How did it go?

#175

Post by hashkb »

m12k wrote: post #77
Johnny555 wrote: post #16

The article mentions Brandolini’s Law: “The amount of energy needed to refute bullshit is an order of magnitude larger than to produce it.” It's the first time I've heard it, but it's a very appropriate observation in today's world where misinformation travels faster and wider than correct information. If you're just making stuff up, it's much faster than looking up sources.

You're absolutely right - the big question of the day is not just "how do we counter disinformation?" but "how do we counter it at scale?" The bad-faith actors of the world have realized how incredibly cost effective disinformation is in an online world that can massively amplify messages, and which algorithmically selects for divisiveness and "engagement" rather than factually and utility. We need a CAPTCHA for trut…

Aligning the incentives is the solution. There's no downside to bullshitting because our institutions are built on it.

Re: I tried to report scientific misconduct. How did it go?

#176

Post by mjburgess »

lanevorockz wrote: post #109

The belief that any research is automatically true is so bogus and so abused that industries and lobbyists came to rely on it. It’s sad that then people blindly push as “it’s science”. Main issue is the sheer amount of papers being published and the lack of capacity of the body of experts to read all of it. I guess it’s the professionalisation of research. People publish papers to improve their rankings and not becau…

This comment really clarified an issue:

This is a slow-moving disaster for scientific credibility, and therefore for national safety and security.

There's going to be a point within two decades where "reproducibility crisis" is not a localised phenomenon, and "expert" misconduct is paraded out by the papers.

Totally destroying our societies ability to govern itself based on expert information. The early stages are already here (anti-climate, anti-vax, etc.).

Re: I tried to report scientific misconduct. How did it go?

#177

Post by DarkWiiPlayer »

MaxBarraclough wrote: post #142
DarkWiiPlayer wrote: post #137

Earlier quoted context omitted.

Every now and then I go through some of wikipedias sources on certain social topics because I don't trust them at all. The amount of BS I've found in papers, even though I don't eve have any research background at all is impressive. My favourite was probably this one paper where the author essentially made a reddit-post asking a community about themselves, then cherry-picked (the post is still up, with timestamps and…

Did you edit the Wikipedia page or challenge the interpretation in the talk page?

I already spend a lot of time trying to "fix the internet" and just don't have the stamina to also start fixing wikipedia now. I'm also being turned away by the constant stories of edit-wars that tend to happen about certain controversial topics.

Re: I tried to report scientific misconduct. How did it go?

#178

Post by cies »

Many stories like this and people wonder why the general public is less "believing" in science lately. I think sadly the general public is right not to believe as strongly in science as it maybe once did, the results though are dramatic in "baby bathwater" kind of proportions.

I think science should fix itself. Just publishing paper should not be the metric to reward. A retraction should seriously reward the flaw finder (like sometimes with exploits), and really harm the flaw author/publisher: both scientist and journal.

Re: I tried to report scientific misconduct. How did it go?

#179

Post by shatnersbassoon »

lldbg wrote: post #93
walleeee wrote: post #21

Earlier quoted context omitted.

The field is over-crowded, for one. Two, its participant pool is almost exclusively undergraduate students on university campuses in WEIRD (western, educated, industrialized, rich, and democratic) nations. Three, experimental designs may rely on questionably reliable, possibly retrospective self-report to infer patterns of affect and cognition. Four, experiments are often conducted in highly contrived laboratory envi…

What is an example of extremely good science in psychology? What's a result we can hold up to scrutiny and believe truthfully with conviction?

Cognitive psychology topics such as memory and reading have a strong paradigm, and established results. In general, their effects are much easier to replicate because you require far fewer participants. This is because you can validly make multiple observations of the same participant.

Re: I tried to report scientific misconduct. How did it go?

#180

Post by PeterisP »

marcus_holmes wrote: post #81
Ace17 wrote: post #42

> I was curious to see how the self-correcting mechanisms of science would respond [...] > I was disappointed by the response from Southwest University. Their verdict has protected [a fraudulent researcher] and enabled him to continue publishing suspicious research at great pace. The self-correcting mechanisms of science can only correct knowledge . Those mechanism work mainly by requiring the research works to be ch…

How does bad research get retracted and corrected then? As the author points out, the "data" in these papers is large enough to contaminate meta-analyses for years to come. And if the Bad Scientist continues to produce more of them, then decades to come. The consensus of the entire discipline will be swayed. Self-correcting this will be very difficult, require lots of data, and be unrewarding. It probably won't happe…

It's probably very different in different disciplines - in social studies like the ones in main article this is a big problem because, as you say, meta-studies will likely include these papers simply because they exist.

However, in more practical sciences if someone fakes data to show that their method A works better than baseline B, then other people building on that find out that method A doesn't really work well for them for a weird reason, shrug, and ignore the bad paper, so it doesn't get used and cited, while the correct assertions persist, get replicated, repeated and cited.

Post reply on HN