Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Semiconducting Transport in LK99 reproduction attempt

arxiv.org

Post reply on HN
Print view View original

111–120 of 269 posts

Re: Semiconducting Transport in LK99 reproduction attempt

#111

Post by sophiebits »

lll-o-lll wrote: post #89
p-e-w wrote: post #56

Earlier quoted context omitted.

So the groups trying to replicate right now are basically wasting their time?

> So the groups trying to replicate right now are basically wasting their time? If a few dozen groups all try and replicate, and there is something to replicate, then odds are good someone will get lucky (birthday paradox and all that). Each individual group is definitely wasting their time, but the aggregate time spent may not be a waste.

Birthday paradox is about random draws coinciding with each other (hence the sqrt behavior), not a target.

Re: Semiconducting Transport in LK99 reproduction attempt

#112

Post by szundi »

p-e-w wrote: post #95
Maxion wrote: post #87

Earlier quoted context omitted.

Because that’s the scientific method? It would be entirely infeasible for them to require someone else to test the sample. I don’t think you’ve been in academia?

> Because that’s the scientific method? The world's leading science journal publishing grand claims without verification, in the face of mounting evidence that the claims might be incorrect, is most certainly not "the scientific method". > It would be entirely infeasible for them to require someone else to test the sample. What nonsense. Why would that be infeasible?

Actually this is the scientific method. Anything else just makes publishing groundbraking results harder and depends on some people’s (faulty) judgements.

Re: Semiconducting Transport in LK99 reproduction attempt

#113

Post by faraggi »

regular_trash wrote: post #104
simplicio wrote: post #33

Earlier quoted context omitted.

Seems the fastest way to clear this up would just be for the original team to send a sample to some other scientists for testing. That would solve any issues regarding other teams reproducing the material incorrectly, and at a first pass, what everyone cares about is whether its a superconductor at room temp, not how to make it.

This assumes the chemical structure is stable, which may not be the case. If the material is truly not superconductive at room temperature, it will take a long time to gather enough diverse experimental evidence to build a scientific consensus. A good rule of thumb is probably 3 months to reproduce, 6 months to discredit.

why is it slower to discredit than to reproduce?

Re: Semiconducting Transport in LK99 reproduction attempt

#114

Post by tetha »

dmvdoug wrote: post #106
lucubratory wrote: post #98

Everyone in this thread needs to understand that particularly for materials science, and particularly when there is bad methodology documentation, it is really, really normal for early replication attempts to fail. What works in one lab will often need to be adjusted to work in another, with different equipment, altitude, humidity, all sorts of stuff. Making it even worse, apparently the original team can't even get…

Not just materials science though. I’m a longtime reader of Derek Lowe’s In the Pipeline blog, and he often emphasizes the trickiness of lab work and lab techniques. But the model we have of science from elementary and high school doesn’t give us room for the random, ephemeral things that come into play in actual experimental work. Our model is methods equal result, every time, even though jiggling your foot a little…

NileRed and other Maker Channels are great to demonstrate that. In every video, there's 1 - 2 of these situations - Nile has been following some documented, established procedure which should result in a specific reaction and it looks decent to the layman - and then it just doesn't work at all. And then he re-does pretty much the same thing and it works.

I can't imagine how frustrating that can be. And then you have to do it with something you don't even know works. Kind of like dealing with obscure retro computers. Does it still work? No one knows. What does it do if it works? No one knows. Can it even function or are parts missing? That's right - no one knows.

Re: Semiconducting Transport in LK99 reproduction attempt

#115

Post by lovich »

cosmojg wrote: post #91
lovich wrote: post #70

Earlier quoted context omitted.

Why would Nature have published the production method of a random material if they didn't have experimental evidence of novel behavior?

They do that every now and then. Peer review rarely, if ever, involves any sort of replication.

I’m asking on a legitimate basis. I am not involved in the science world and only know about Nature as a publication from high school and college. The way Nature was described to gave me the sense that you weren’t going to have a paper published unless you had some surprising results, and that didn’t include the null hypothesis.

If that’s not the case for this publication then I would appreciate if someone showed me how they changed in the past decade

Re: Semiconducting Transport in LK99 reproduction attempt

#116

Post by phanimahesh »

jacquesm wrote: post #81
tnecniv wrote: post #80

Earlier quoted context omitted.

Normally (at least in my field), that’s beyond the scope of peer review in the traditional sense. Peer review typically just involves review of just the material submitted to the journal (manuscript, data, maybe code depending on its size and complexity). Depending on the experiment, reproducing a result may require resources, equipment, or skills unavailable to the reviewer or that cannot be acquired in a reasonable…

That's true but if a reputable journal publishes this before that verification is in and it ends up being mistaken or even a hoax or fraud then their reputation will be dinged so with the stakes this high given the material and the visibility there is a fair chance that they will indeed wait for independent replication before committing to publishing it. But I don't see it happening during the peer review process its…

The journals reputation will be dinged if it comes to light that they have not adequately performed their due diligence. Which, as has been explained above, does not include replication.

If you require replication for publication practically nothing will get published ever. Once published, others will attempt to replicate, and it is the authors' reputation on the line if nobody manages to replicate it following the paper and consulting with the authors.

Re: Semiconducting Transport in LK99 reproduction attempt

#117

Post by threeseed »

danbruc wrote: post #9

Maybe not yet the final nail in the coffin but I would no longer bet much on a successful replication. Which leaves me wondering how we got the initial papers. Does fraud make any sense, how could you possibly benefit from something like this? And if I would think that I might have produced a room temperature superconductor due to experimental errors, I would double and tripple check what I did. Assuming there is tim…

The authors already said that they were not able to consistently replicate it.

And so we have one failed attempt based on limited information and then suddenly the authors are accused of fraud ?

I just think people need to take a deep breath before jumping to wild conclusions.

Re: Semiconducting Transport in LK99 reproduction attempt

#118

Post by mort96 »

szundi wrote: post #112
p-e-w wrote: post #95

Earlier quoted context omitted.

> Because that’s the scientific method? The world's leading science journal publishing grand claims without verification, in the face of mounting evidence that the claims might be incorrect, is most certainly not "the scientific method". > It would be entirely infeasible for them to require someone else to test the sample. What nonsense. Why would that be infeasible?

Actually this is the scientific method. Anything else just makes publishing groundbraking results harder and depends on some people’s (faulty) judgements.

Publishing incorrect results should be hard, especially if they would be ground breaking if true.

Re: Semiconducting Transport in LK99 reproduction attempt

#119

Post by [deleted] »

lucubratory wrote: post #98

Everyone in this thread needs to understand that particularly for materials science, and particularly when there is bad methodology documentation, it is really, really normal for early replication attempts to fail. What works in one lab will often need to be adjusted to work in another, with different equipment, altitude, humidity, all sorts of stuff. Making it even worse, apparently the original team can't even get…

[deleted]

Re: Semiconducting Transport in LK99 reproduction attempt

#120

Post by teaearlgraycold »

tetha wrote: post #114
dmvdoug wrote: post #106

Earlier quoted context omitted.

Not just materials science though. I’m a longtime reader of Derek Lowe’s In the Pipeline blog, and he often emphasizes the trickiness of lab work and lab techniques. But the model we have of science from elementary and high school doesn’t give us room for the random, ephemeral things that come into play in actual experimental work. Our model is methods equal result, every time, even though jiggling your foot a little…

NileRed and other Maker Channels are great to demonstrate that. In every video, there's 1 - 2 of these situations - Nile has been following some documented, established procedure which should result in a specific reaction and it looks decent to the layman - and then it just doesn't work at all. And then he re-does pretty much the same thing and it works. I can't imagine how frustrating that can be. And then you have…

Link to NileRed video: https://www.youtube.com/watch?v=RS7gyZJg5nc
Post reply on HN