Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I tried to report scientific misconduct. How did it go?

crystalprisonzone.blogspot.com

Post reply on HN
Print view View original

11–20 of 363 posts

Re: I tried to report scientific misconduct. How did it go?

#11

Post by unishark »

thaumasiotes wrote: post #6
jojobas wrote: post #4

The model of science that worked from Newton to Landau seems to be falling apart in today's scale. There are millions of people whose livelihood depends on publishing, so they will publish anything they'll get away with. The amount of noise is beyond any researcher's ability to pick through. True incremental improvements in all areas are drowned in a steady flow of bad research. Top tier institutions seem to survive…

> There are millions of people whose livelihood depends on publishing, so they will publish anything they'll get away with. This is also the problem with the iOS App Store and the Google Play store. They are modeled off Linux repositories, and those do not cause major problems. The phone app stores are riddled with problems, because you can charge for apps in those stores. If you're willing to pay people to lie to yo…

> Research output is reliable where it is actively relied on by engineers -- and not elsewhere.

I'd word it this way: if the person producing the output is not responsible for it really working, it almost certainly won't. Even innocently this will be the case with anything complex, people get things backwards, miss a scale factor, etc. Finding that last bug can take more work than the rest of the project combined, much easier to publish what appears to work and move on. Much more so when there's direct career benefits to "hacking" the system over competing honestly. Especially considering the internet is awash with people trying to cheat through every other competition (exam questions, interview questions, etc.)

Re: I tried to report scientific misconduct. How did it go?

#13

Post by chaoxu »

thaumasiotes wrote: post #10

This is pretty interesting stuff, but one note: > The correction explains away the failures of randomization as an error in translation; the authors now claim that they let participants self-select their condition. This is difficult for me to believe. The original article’s stressed multiple times its use of random assignment and described the design as a "true experiment.” > They also had perfectly equal samples per…

How about the following process? Each person gets randomly assigned to one of the two groups, when one group is full, move the rest to the other group. Does this make sure every equal partition have the same probability of showing up?

Re: I tried to report scientific misconduct. How did it go?

#14

Post by sterlind »

thaumasiotes wrote: post #10

This is pretty interesting stuff, but one note: > The correction explains away the failures of randomization as an error in translation; the authors now claim that they let participants self-select their condition. This is difficult for me to believe. The original article’s stressed multiple times its use of random assignment and described the design as a "true experiment.” > They also had perfectly equal samples per…

couldn't you just assign each student an ID, get a random permutation of the array of students and assign violence to even indices and non-violence to odd? what am I missing here?

Re: I tried to report scientific misconduct. How did it go?

#15

Post by thaumasiotes »

unishark wrote: post #11
thaumasiotes wrote: post #6

Earlier quoted context omitted.

> There are millions of people whose livelihood depends on publishing, so they will publish anything they'll get away with. This is also the problem with the iOS App Store and the Google Play store. They are modeled off Linux repositories, and those do not cause major problems. The phone app stores are riddled with problems, because you can charge for apps in those stores. If you're willing to pay people to lie to yo…

> Research output is reliable where it is actively relied on by engineers -- and not elsewhere. I'd word it this way: if the person producing the output is not responsible for it really working, it almost certainly won't. Even innocently this will be the case with anything complex, people get things backwards, miss a scale factor, etc. Finding that last bug can take more work than the rest of the project combined, mu…

> if the person producing the output is not responsible for it really working, it almost certainly won't. Even innocently this will be the case with anything complex

Indeed, there's a great example in the article itself, in a totally unrelated area:

> I felt these journals generally did their best, and the slowness of the process likely comes from the bureaucracy of the process and the inexperience editors have with that process.

In other words, these reasonable journals weren't able to use their retraction process even though they wanted to, because the process never gets used and therefore isn't in a usable state.

Re: I tried to report scientific misconduct. How did it go?

#16

Post by Johnny555 »

The article mentions Brandolini’s Law:

“The amount of energy needed to refute bullshit is an order of magnitude larger than to produce it.”

It's the first time I've heard it, but it's a very appropriate observation in today's world where misinformation travels faster and wider than correct information. If you're just making stuff up, it's much faster than looking up sources.

Re: I tried to report scientific misconduct. How did it go?

#17

Post by thaumasiotes »

sterlind wrote: post #14
thaumasiotes wrote: post #10

This is pretty interesting stuff, but one note: > The correction explains away the failures of randomization as an error in translation; the authors now claim that they let participants self-select their condition. This is difficult for me to believe. The original article’s stressed multiple times its use of random assignment and described the design as a "true experiment.” > They also had perfectly equal samples per…

couldn't you just assign each student an ID, get a random permutation of the array of students and assign violence to even indices and non-violence to odd? what am I missing here?

You can do that, but it requires all of the assignments to be done simultaneously at the beginning of the study, which will cause problems for e.g. medical trials where not everyone enrolls at once.

But why bother? There's no special statistical value in having two exactly equal buckets as opposed to one bucket with 1,621 people in it and another with 1,427.

Re: I tried to report scientific misconduct. How did it go?

#18

Post by thaumasiotes »

chaoxu wrote: post #13
thaumasiotes wrote: post #10

This is pretty interesting stuff, but one note: > The correction explains away the failures of randomization as an error in translation; the authors now claim that they let participants self-select their condition. This is difficult for me to believe. The original article’s stressed multiple times its use of random assignment and described the design as a "true experiment.” > They also had perfectly equal samples per…

How about the following process? Each person gets randomly assigned to one of the two groups, when one group is full, move the rest to the other group. Does this make sure every equal partition have the same probability of showing up?

> Does this make sure every equal partition have the same probability of showing up?

I'm not sure, but I wouldn't bet against it. But what is the value of having an exactly equal partition?

On second thought, the algorithm you describe processes people in a particular order, and it is much more likely to put two people who both occur near the end of the list into the same bucket than to put them in different buckets. So if that processing order is constant, the algorithm cannot produce every equal partition with equal probability.

Re: I tried to report scientific misconduct. How did it go?

#19

Post by SulfurHexaFluri »

Johnny555 wrote: post #16

The article mentions Brandolini’s Law: “The amount of energy needed to refute bullshit is an order of magnitude larger than to produce it.” It's the first time I've heard it, but it's a very appropriate observation in today's world where misinformation travels faster and wider than correct information. If you're just making stuff up, it's much faster than looking up sources.

Its no wonder there is a trend now of just outright rejecting information presented when our "trusted" sources of information are very susceptible to malice and error without any real tools to combat it.

My current view is that academic research should not be used as proof of anything and only as the starting point for your own research. And by your own research I mean your own actual tests. The papers can point you in the right direction but their findings should not be taken as fact.

Re: I tried to report scientific misconduct. How did it go?

#20

Post by rkagerer »

Johnny555 wrote: post #16

The article mentions Brandolini’s Law: “The amount of energy needed to refute bullshit is an order of magnitude larger than to produce it.” It's the first time I've heard it, but it's a very appropriate observation in today's world where misinformation travels faster and wider than correct information. If you're just making stuff up, it's much faster than looking up sources.

Reminds me of "a lie can run around the world before the truth has got its boots on".

That precise quote is from Pratchett but there are similar, earlier citations https://quoteinvestigator.com/2014/07/13/truth/

Post reply on HN