Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fraud, so much fraud

science.org

Post reply on HN
Print view View original

491–500 of 868 posts

Re: Fraud, so much fraud

#491

Post by semiquaver »

mlsu wrote: post #80
jimbokun wrote: post #54

Earlier quoted context omitted.

My recommendation is for journals to place at least equal importance to publishing replications as for the original studies. Studies that have not been replicated should be published clearly marked as preliminary results. And then other scientists can pick those up and try to replicate them. And institutions need to give near equal weight to replications as to original research when deciding on promotions. Should be…

We can solve this at the grant level. Stipulate that for every new paper a group publishes from a grant, that group must also publish a replication of an existing finding. Publication would happen in pairs, so that every novel thing would be matched with a replication. Replications could be matched with grants: if you receive $100,000 grant, you'd get the $100,000 you need, plus another $100,000 which you could use t…

> I bet we could throw away half of publications tomorrow and see no effect on the actual pace of progress in science.

Undoubtedly this is true. The problem, like with advertising, is identifying _which_ half to cut.

Re: Fraud, so much fraud

#492

Post by A_D_E_P_T »

There may be a dark twist to this story.

The expose article writes:

> "UCSD neuroscientist Edward Rockenstein, who worked under Masliah for years, co-authored 91 papers that contain questioned images, including 11 as first author. He died in 2022 at age 57."

They say nothing else about this. But looking at Rockenstein's obituary, indications are that it was suicide. (It was apparently sudden, at quite a young age, and there are many commenters on his memorial page "hoping that his soul finds peace," and expressing similar sentiments.)

Re: Fraud, so much fraud

#493

Post by benlivengood »

echoangle wrote: post #442
benlivengood wrote: post #419

Earlier quoted context omitted.

One clear piece of evidence would be ruling out "AI models the corvid brain" or "AI models the cephalopod brain" which might narrow it toward the human brain. That it's functionally impossible to do either leads me to believe that "it models some form of intelligence" is about the best we can prove.

I don’t understand the standard of modeling you seem to assume. Modeling a human brain, a cephalopod brain and a corvid brain aren’t even mutually exclusive if your model is abstract enough. When I say „a neural network models a human brain“, I’m talking about the high level concept, not the specific structure of a human brain compared to other brains. You could also say that it models a dogs brain if you wanted to.…

I think a lot of people don't abstract their brain model when they say "models a human brain", or they'd say "models biological intelligence", etc. Specifically, I don't think there are any human traits in LLMs other than having mostly been trained on human outputs. They see tokens and predict tokens; very different sensorium from humans. There aren't any specific corvid or cephalopod traits either afaik.

Biological brains don't use gradient descent and don't seem to use 1-hot encoding/decoding for the most part.

Re: Fraud, so much fraud

#494

Post by Meekro »

worik wrote: post #464
tim333 wrote: post #429

Earlier quoted context omitted.

Preventing people from working if they didn't get a covid vaccine was a bit heavy handed. And saying it was likely made in a lab in China is kind of censored to this day. I think partly because the science community doesn't want to take flack for doing risky stuff and killing millions.

> Preventing people from working if they didn't get a covid vaccine was a bit heavy handed. Nobody did that. They prevented you from working with me Nobody here or there was forced to get a vaccine. But if you refused it was right to shun you Freedom is about more than the individual. We as a group should be free from the consequences of individual actions

The Biden administration had OSHA make rules to force employers to make their employees get the vax. The Supreme Court stopped them.

Re: Fraud, so much fraud

#495

Post by __turbobrew__ »

jimbokun wrote: post #268
__turbobrew__ wrote: post #254

Earlier quoted context omitted.

That doesn’t fix the incentives, people will just commit fraud when replicating results so they can perform more original research.

Don't require the replication to be successful. Failure to replicate a result is just as valuable as successfully replicating a result.

It is a lot easier to just falsely prove the experiment since the data is already there and the publisher of the paper is not going to push back if you confirm it.

Why go through all the work of actually proving/disproving the experiment when you can just change tweak the numbers of the original experiment, say you actually reproduced the experiment, and then move on?

Re: Fraud, so much fraud

#497

Post by mlsu »

mike_hearn wrote: post #378
mlsu wrote: post #80

Earlier quoted context omitted.

We can solve this at the grant level. Stipulate that for every new paper a group publishes from a grant, that group must also publish a replication of an existing finding. Publication would happen in pairs, so that every novel thing would be matched with a replication. Replications could be matched with grants: if you receive $100,000 grant, you'd get the $100,000 you need, plus another $100,000 which you could use t…

Replication is over-emphasised. Attempts to organise mass replications have struggled with basic problems like papers making numerous claims (which one do you replicate?), the question of whether you try to replicate the original methodology exactly or whether you try to answer the same question as the original paper (matters in cases where the methodology was bad), many papers making obvious low value findings (e.g.…

So then, how on earth does this stuff even get published? What exactly is it that we're all doing here?

If a finding either cannot be communicated enough for someone else to replicate it, or cannot be replicated because the method is shoddy, can we even call that science?

At some level I know that what I'm proposing isn't realistic because the majority of science is sloppy. P-hacking, lack of detail, bad writing, bad methods, code that doesn't compile, fraud. But maybe if we tried some version of this, it would cause a course correction. Reviewers, knowing that someone actually would attempt to replicate a paper at some point down the road, would be far more critical of ambiguity and lack of detail.

Papers that are not fit to be replicated in the future, whose claims cannot be tested independently, are actually not science at all. They are worth less than nothing because they take up air in the room, choking out actual progress.

Re: Fraud, so much fraud

#498

Post by acdha »

mbreese wrote: post #156
ChainOfFools wrote: post #95

Earlier quoted context omitted.

I predict it will get even worse than that, in the next couple of decades I expect any document or work that has a substantial reward associated with it, either financially or in terms of career advancement or a grade for critical course work in one's major, or penalty such as indictment or conviction, to be backed by a time stamped stack of developing documentation, drafts, revisions, with these time stamp validated…

I actually thought that digital timestamps would have been a great use-case for blockchains. They are publicly available and auditable. If you're working from hashes, you don't necessarily need to make the raw data public, just the hash. It is a use-case that has an intrinsic value to the data generator and the future auditor (so you could charge something for it). I know there was some work done on this, but I think…

The gold bugs really set back that entire field: the quasi-religious pursuit of “trustless” designs made everything more expensive, but so many problems are far more tractable with trusted third parties both for cost and reduced attack potential because institutional/professional reputations are harder to build than getting n% consensus on a cryptocurrency and don’t have the built-in bug bounty problem.

For example, imagine if university libraries ran storage systems based on Merkle trees with PKI signatures and researchers used those for their papers, code, data inventory (maybe not petabytes of data but the hashes of that data), etc. If there were allegations of misconduct you’d be able to see that whole history establishing that when things were changed and by whom, and someone couldn’t fudge the data without multiple compromised/complicit people in a completely different department (a senior figure can pressure a grad student in their field but they have far less leverage over staff at the library), and since you’re not sharing a database with the entire world you have a much easier time scaling with periodic cross checks (e.g. MIT and Caltech could cross-sign each other’s indexes periodically so you could have confidence that nobody had altered the inventory without storing the actual collection).

Re: Fraud, so much fraud

#499

Post by dekhn »

greenavocado wrote: post #482
dekhn wrote: post #434

Earlier quoted context omitted.

It's probably better to say that engineering based on science is usually real. Engineering cares a lot less about falsifying theories and more on what existing theories seem to have general predictive value, and can be used to do stuff, including things where human lives are at risk (tall buildings, fire reduction materials, airplanes). And if there are failures out in the field, they're inspected, and those results…

I think you've hit on an important point. Science isn't about finding absolute truth, but rather about generating testable hypotheses that can be validated through experimentation and observation. This is why it's so crucial for scientists to follow the scientific method - they need to be willing to revise their theories based on new evidence. Your comparison between science and engineering is a good one. Engineering…

The intent of science is to find absolute truth. It's just that the mechanism by which we do so typically involves demonstrating that a finding isn't absolutely true. And we also lack the epistemological confidence to say that what we're observing represents the absolute truth, or that the idea of absolute truth is meaningful.

Re: Fraud, so much fraud

#500

Post by fluoridation »

tedk-42 wrote: post #435
fluoridation wrote: post #428

Earlier quoted context omitted.

Well, the incentive is that he didn't want to run the tests out of laziness (i.e. he lacked an incentive to run them). He ran to Lisa to give his anticlimactic report not to be deceptive, but rather he just happened to be cycling through that part of town and just needed to use the bathroom really badly.

The writers of these episodes were really on another level considering it was a cartoon. Lisa's first word is still a personal favourite of mine, especially now as a father.

To be honest, it's difficult to tell if the subplot makes sense on purpose, or if the writers just wanted to make a joke and it just happened to end up making sense. I don't think I had ever put the three scenes together before now.
Post reply on HN