Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep learning gets the glory, deep fact checking gets ignored

rachel.fast.ai

Post reply on HN
Print view View original

21–30 of 174 posts

Re: Deep learning gets the glory, deep fact checking gets ignored

#21

Post by a_bonobo »

kenjackson wrote: post #4

"And for most deep learning papers I read, domain experts have not gone through the results with a fine-tooth comb inspecting the quality of the output. How many other seemingly-impressive papers would not stand up to scrutiny?" Is this really not the case? I've read some of the AI papers in my field, and I know many other domain experts have as well. That said I do think that CS/software based work is generally easi…

Validation of biological labels easily takes years - in the OP's example it was a 'lucky' (huge!) coincidence that somebody already had spent years on one of the predicted proteins' labels. Nobody is going to stake 3-5 years of their career on validating some random model's predictions.

Re: Deep learning gets the glory, deep fact checking gets ignored

#23

Post by patagurbon »

amelius wrote: post #2

Before making AI do research, perhaps we should first let it __reproduce__ research. For example, give it a paper of some deep learning technique and make it produce an implementation of that paper. Before it can do that, I have no hope that it can produce novel ideas.

You would have to have a very complete audit trail for the LLM and ensure the paper shows up nowhere in the dataset.

We have rare but not unheard of issues with academic fraud. LLMs fake data and lie at the drop of a hat

Re: Deep learning gets the glory, deep fact checking gets ignored

#24

Post by toofy »

godelski wrote: post #11
aucisson_masque wrote: post #9

It’s like fake news is taking in science now. Saying any stupid thing will attract much more view and « likes » than those debunking them. Except that we can’t compare twitter to nature journal. Science is supposed to be immune to these kind of bullshit thanks to reputed journals and pair reviewing, blocking a publication before it does any harm. Was that a failure of nature ?

Yes. And let's not get started on that ML Quantum Wormhole bullshit... We've taken this all too far. It is bad enough to lie to the masses in Pop-Sci articles. But we're straight up doing it in top tier journals. Some are good faith mistakes, but a lot more often they seem like due diligence just wasn't ever done. Both by researchers and reviewers. I at least have to thank the journals. I've hated them for a long tim…

this seems strange to me, shouldn’t we expect a high quality journal to retract often as we gather more information?

obviously this is hyperbole of two extremes, but i certainly trust a journal far more if it actively and loudly looks to correct mistakes over one that never corrects anything or buries its retractions.

a rather important piece of science is correcting mistakes by gathering and testing new information. we should absolutely be applauding when a journal loudly and proactively says “oh, it turns out we were wrong when we declared burying a chestnut under the oak tree on the third thursday of a full moon would cure your brothers infected toenail.”

Re: Deep learning gets the glory, deep fact checking gets ignored

#25

Post by 8bitsrule »

Fits my limited experiences with LLM (as a researcher). Very impressive apparent written language comprehension and written expression. But when it comes to getting to the -best possible answer- (particulary on unresolved questions), the nearly-instant responses (e.g. to questions that one might spend a half-day on without resolution) are seldom satisfactory. Complicated questions take time to explore, and IME an LLM's lack-of-resolution (because of it's inability) is, so far, set aside in favor of confident-sounding (even if completely-wrong) responses.

Re: Deep learning gets the glory, deep fact checking gets ignored

#26

Post by tbrownaw »

aucisson_masque wrote: post #9

It’s like fake news is taking in science now. Saying any stupid thing will attract much more view and « likes » than those debunking them. Except that we can’t compare twitter to nature journal. Science is supposed to be immune to these kind of bullshit thanks to reputed journals and pair reviewing, blocking a publication before it does any harm. Was that a failure of nature ?

> It’s like fake news is taking in science now.

I didn't think this was new? Like, it's been a few years since that replication crisis things kicked off.

Re: Deep learning gets the glory, deep fact checking gets ignored

#27

Post by suddenlybananas »

kenjackson wrote: post #4

"And for most deep learning papers I read, domain experts have not gone through the results with a fine-tooth comb inspecting the quality of the output. How many other seemingly-impressive papers would not stand up to scrutiny?" Is this really not the case? I've read some of the AI papers in my field, and I know many other domain experts have as well. That said I do think that CS/software based work is generally easi…

My impression with linguistics is that people do go over the papers that use these techniques carefully and come up with criticisms of them, but people don't take linguists seriously so people from other related disciplines ignore the criticisms.

Re: Deep learning gets the glory, deep fact checking gets ignored

#28

Post by antithesizer »

godelski wrote: post #10

> although later investigation suggests there may have been data leakage I think this point is often forgotten. Everyone should assume data leakage until it is strongly evidenced otherwise. It is not on the reader/skeptic to prove that there is data leakage, it is the authors who have the burden of proof. It is easy to have data leakage on small datasets. Datasets where you can look at everything. Data leakage is rea…

The supposed location of the burden of proof is really not the definitive guide to what you ought to believe that people online seem to think it is.

Re: Deep learning gets the glory, deep fact checking gets ignored

#30

Post by TeMPOraL »

patagurbon wrote: post #23
amelius wrote: post #2

Before making AI do research, perhaps we should first let it __reproduce__ research. For example, give it a paper of some deep learning technique and make it produce an implementation of that paper. Before it can do that, I have no hope that it can produce novel ideas.

You would have to have a very complete audit trail for the LLM and ensure the paper shows up nowhere in the dataset. We have rare but not unheard of issues with academic fraud. LLMs fake data and lie at the drop of a hat

> You would have to have a very complete audit trail for the LLM and ensure the paper shows up nowhere in the dataset.

We can do both known and novel reproductions. Like with both LLM training process and human learning, it's valuable to take it in two broad steps:

1) Internalize fully-worked examples, then learn to reproduce them from memory;

2) Train on solving problems for which you know the results but have to work out intermediate steps yourself (looking at the solution before solving the task)

And eventually:

3) Train on solving problems you don't know the answer to, have your solution evaluated by a teacher/judge (that knows the actual answers).

Even parroting existing papers is very valuable, especially early on, when the model is learning how papers and research looks like.

Post reply on HN