Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Semiconducting Transport in LK99 reproduction attempt

arxiv.org

Post reply on HN
Print view View original

131–140 of 269 posts

Re: Semiconducting Transport in LK99 reproduction attempt

#131

Post by djtango »

dmvdoug wrote: post #106
lucubratory wrote: post #98

Everyone in this thread needs to understand that particularly for materials science, and particularly when there is bad methodology documentation, it is really, really normal for early replication attempts to fail. What works in one lab will often need to be adjusted to work in another, with different equipment, altitude, humidity, all sorts of stuff. Making it even worse, apparently the original team can't even get…

Not just materials science though. I’m a longtime reader of Derek Lowe’s In the Pipeline blog, and he often emphasizes the trickiness of lab work and lab techniques. But the model we have of science from elementary and high school doesn’t give us room for the random, ephemeral things that come into play in actual experimental work. Our model is methods equal result, every time, even though jiggling your foot a little…

We also don't have a culture of replication. Basically all of the post docs I worked with in the lab lamented how hard it can be to replicate reported syntheses in chemistry wet labs.

Everything from humidity and altitude to undocumented "magic tricks" (eg baking something in an oven under vaccuum for a week to get.rid.of.all.the.water.)

I heard anecdotes that experiments were sometimes literally only reproducible in one exact lab because reasons.

Turns out there's just as much art as there is science in experimentation...

Re: Semiconducting Transport in LK99 reproduction attempt

#132

Post by Remnant44 »

faraggi wrote: post #113
regular_trash wrote: post #104

Earlier quoted context omitted.

This assumes the chemical structure is stable, which may not be the case. If the material is truly not superconductive at room temperature, it will take a long time to gather enough diverse experimental evidence to build a scientific consensus. A good rule of thumb is probably 3 months to reproduce, 6 months to discredit.

why is it slower to discredit than to reproduce?

Someone claims that a human can run a 5 minute mile.

To prove it true, you just need to show a human who can run that fast. To provide it fast, you have to accumulate incontrovertible evidence that it's simply beyond us...

Re: Semiconducting Transport in LK99 reproduction attempt

#133

Post by AYBABTME »

bibabaloo wrote: post #52
canjobear wrote: post #46

Earlier quoted context omitted.

If it’s fraud, what’s the end game?

There's betting markets on this, aren't there? The super cynical take is that the original scientists know that it's a fraud but in the lead up to it being proven as fraud can make a lot of money before coming out and saying "sorry, we messed up, experimental errors etc etc"

How are they going to make money with this? A few weeks of buzz isn't going to make anyone rich. It's not like they're going for some product placement along with the buzz.

Re: Semiconducting Transport in LK99 reproduction attempt

#134

Post by physicsguy »

p-e-w wrote: post #95
Maxion wrote: post #87

Earlier quoted context omitted.

Because that’s the scientific method? It would be entirely infeasible for them to require someone else to test the sample. I don’t think you’ve been in academia?

> Because that’s the scientific method? The world's leading science journal publishing grand claims without verification, in the face of mounting evidence that the claims might be incorrect, is most certainly not "the scientific method". > It would be entirely infeasible for them to require someone else to test the sample. What nonsense. Why would that be infeasible?

Nature basically publishes the sensational claims papers. That's their model, anything "high impact" goes there. It's then on other scientists to publish work refuting it or proving it's not correct. It's an issue that the refutations tend to go in lower impact journals, but it's the way it's works.

Re: Semiconducting Transport in LK99 reproduction attempt

#135

Post by mschuster91 »

djtango wrote: post #131
dmvdoug wrote: post #106

Earlier quoted context omitted.

Not just materials science though. I’m a longtime reader of Derek Lowe’s In the Pipeline blog, and he often emphasizes the trickiness of lab work and lab techniques. But the model we have of science from elementary and high school doesn’t give us room for the random, ephemeral things that come into play in actual experimental work. Our model is methods equal result, every time, even though jiggling your foot a little…

We also don't have a culture of replication. Basically all of the post docs I worked with in the lab lamented how hard it can be to replicate reported syntheses in chemistry wet labs. Everything from humidity and altitude to undocumented "magic tricks" (eg baking something in an oven under vaccuum for a week to get.rid.of.all.the.water.) I heard anecdotes that experiments were sometimes literally only reproducible in…

The problem IMHO is funding related. Often enough the original lab's grant is barely enough to account for the research and a shoddily put together paper, the rest is filled with a crapload of un(der)paid/"voluntold" labour.

It's rare for researchers to get the budget to replicate someone else's work - this case might be one of the few exceptions given that everyone and their dog want to be the first one to show the world a successful replication. Hell, there are Youtube and Tiktok streamers on the task as well, that way you know just how nuts everyone is going.

Re: Semiconducting Transport in LK99 reproduction attempt

#136

Post by jacquesm »

phanimahesh wrote: post #116
jacquesm wrote: post #81

Earlier quoted context omitted.

That's true but if a reputable journal publishes this before that verification is in and it ends up being mistaken or even a hoax or fraud then their reputation will be dinged so with the stakes this high given the material and the visibility there is a fair chance that they will indeed wait for independent replication before committing to publishing it. But I don't see it happening during the peer review process its…

The journals reputation will be dinged if it comes to light that they have not adequately performed their due diligence. Which, as has been explained above, does not include replication. If you require replication for publication practically nothing will get published ever. Once published, others will attempt to replicate, and it is the authors' reputation on the line if nobody manages to replicate it following the p…

They are not going to replicate this themselves, obviously. But extraordinary claims etc, and this one is about as extraordinary as they come. Papers that are non-controversial even if they might turn out to be wrong get published all the time. But papers where 95% of the scientists out there are going to be super skeptical will find it much harder to get published. Both the authors and the publishers have a reputation to protect. Keep in mind that the peer-reviewers will have to sign off on the publication as well.

Re: Semiconducting Transport in LK99 reproduction attempt

#137

Post by jacquesm »

RF_Savage wrote: post #128
jacquesm wrote: post #110

Earlier quoted context omitted.

It would be a variation on what's already out there right now. And the whole problem is that the recipe is vague, in other words even if they have a working sample they don't know exactly what goes into making it.

Which is why it in arxiv and not some journal. Seems like somebody pulled the trigger on publishing too early.

Exactly, there is some talk about the team not being in sync on this.

Re: Semiconducting Transport in LK99 reproduction attempt

#138

Post by eric-hu »

Loic wrote: post #125
dmvdoug wrote: post #106

Earlier quoted context omitted.

Not just materials science though. I’m a longtime reader of Derek Lowe’s In the Pipeline blog, and he often emphasizes the trickiness of lab work and lab techniques. But the model we have of science from elementary and high school doesn’t give us room for the random, ephemeral things that come into play in actual experimental work. Our model is methods equal result, every time, even though jiggling your foot a little…

For computer science people, I always use the yield on wafers as analogy to explain how hard it can be to nail an experiment right. Usually they understand directly because this is something they can easily relate to.

For those of us unfamiliar with the numbers, what do typical yields look like on silicon wafers?

Re: Semiconducting Transport in LK99 reproduction attempt

#139

Post by raincole »

bestcoder69 wrote: post #10
danbruc wrote: post #9

Maybe not yet the final nail in the coffin but I would no longer bet much on a successful replication. Which leaves me wondering how we got the initial papers. Does fraud make any sense, how could you possibly benefit from something like this? And if I would think that I might have produced a room temperature superconductor due to experimental errors, I would double and tripple check what I did. Assuming there is tim…

Could be a skill issue on the part of the replicators? The russian anime girl scientist had to make a bunch of modifications (and was cursing the orig authors) before she could make her rocks float.

Is there any reason to treat Iris_IGB seriously? (genuine question)

Re: Semiconducting Transport in LK99 reproduction attempt

#140

Post by Toutouxc »

Remnant44 wrote: post #132
faraggi wrote: post #113

Earlier quoted context omitted.

why is it slower to discredit than to reproduce?

Someone claims that a human can run a 5 minute mile. To prove it true, you just need to show a human who can run that fast. To provide it fast, you have to accumulate incontrovertible evidence that it's simply beyond us...

For those firing up Google: The mile run record has been sitting at 3:43 for over two decades now.
Post reply on HN