Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep learning gets the glory, deep fact checking gets ignored

rachel.fast.ai

Post reply on HN
Print view View original

71–80 of 174 posts

Re: Deep learning gets the glory, deep fact checking gets ignored

#71

Post by godelski »

somenameforme wrote: post #52
godelski wrote: post #11

Earlier quoted context omitted.

Yes. And let's not get started on that ML Quantum Wormhole bullshit... We've taken this all too far. It is bad enough to lie to the masses in Pop-Sci articles. But we're straight up doing it in top tier journals. Some are good faith mistakes, but a lot more often they seem like due diligence just wasn't ever done. Both by researchers and reviewers. I at least have to thank the journals. I've hated them for a long tim…

Almost nobody is "anti-science". The source of that labeling and division came from appeals to authority. You must do or believe this because it's "the science." If you don't, or you disagree, then you are anti-science. It has nothing to do with science, but rather people not finding that a sufficient justification for unpopular actions. For instance it's 100% certain that banning sugary drinks would dramatically imp…

  > Almost nobody is "anti-science".
Last I checked:

  - 15% of Americans don't believe in Climate Change[0]
  - 37% believe God created man in our current form within the last ~10k years 
    (i.e. don't believe in evolution)[1]
I don't think these are just rounding errors.

They're large enough numbers that you should know multiple people who hold these beliefs unless you're in a strong bubble.

I'm obviously with you in news and pop-sci being terrible. I hate IFuckingLoveScience. They're actually just IFuckingLoveClickbait. My point was literally about this bullshit.

90% of the time it is news and pop-sci miscommunicating papers. Where they clearly didn't bother to talk to authors and likely didn't even read the paper. "Scientists say ". You see this from eating chocolate, drinking a glass of red wine, to eating red meat or processed meat. There are nuggets of truth in those things but they're about just as accurate as the grandma that sued McDonalds over coffee that was too hot. You sure bet this stuff creates distrust in science

[0] https://record.umich.edu/articles/nearly-15-of-americans-den...

[1] https://news.gallup.com/poll/647594/majority-credits-god-hum...

Re: Deep learning gets the glory, deep fact checking gets ignored

#72

Post by ErigmolCt »

amelius wrote: post #2

Before making AI do research, perhaps we should first let it __reproduce__ research. For example, give it a paper of some deep learning technique and make it produce an implementation of that paper. Before it can do that, I have no hope that it can produce novel ideas.

Reproducibility is the baseline. Until models can consistently read, understand, and implement existing research correctly, "AI scientist" talk is mostly just branding.

Re: Deep learning gets the glory, deep fact checking gets ignored

#73

Post by ErigmolCt »

choeger wrote: post #62

It's only logical that this happens. Just because we can nowadays throw a massive amount of compute on a problem doesn't mean our models are good. Why are people using transformers? Do they have any intuition that they could solve the challenge, let alone efficiently?

There's a tendency to treat transformers as a magic wand

Re: Deep learning gets the glory, deep fact checking gets ignored

#74

Post by ErigmolCt »

softwaredoug wrote: post #29

We also love deep cherry picking. Working hard to find that one awesome time some ML / AI thing worked beautifully and shouting its praises to the high heavens. Nevermind the dozens of other times we tried and failed...

Yup, the survivorship bias is strong. It's like academic slot machines

Re: Deep learning gets the glory, deep fact checking gets ignored

#75

Post by mike_hearn »

thrance wrote: post #41
amelius wrote: post #2

Before making AI do research, perhaps we should first let it __reproduce__ research. For example, give it a paper of some deep learning technique and make it produce an implementation of that paper. Before it can do that, I have no hope that it can produce novel ideas.

Side note: I wonder why it's not normalized for more papers to come with a reference implementation. Wouldn't have to be efficient, or even be easily runnable. Could be a link to a repository with a few python scripts.

Sometimes papers do. Like everything with academia, there's no consistency and it varies mostly by field. It's especially common in CS and less common in other fields.

The main reason people don't do it is because incentives are everything, and university/government management set bad incentives. The article points this out too. They judge academics entirely by some function of paper citations, so academics are incentivized to do the least possible work to maximize that metric. There's no positive incentive to publish more than necessary, and doing so can be risky because people might find flaws in your work by checking it. So a lot of researchers hide their raw data or code for as long as possible. They know this is wrong and will typically claim they'll publish it but there's a lot of foot dragging, and whatever gets released might not be what they used to make the paper.

In the commercial world the incentives are obviously different, but the outcomes are the same. Sometimes companies want the ideas to be used as they compliment the core business, other times the ideas need to be protected to be turned into a core business. People like to think academics and industrial research are very different but everyone is optimizing for some metric, whether they like it or not.

Re: Deep learning gets the glory, deep fact checking gets ignored

#76

Post by imiric »

It's interesting to see this article in juxtaposition to the one shared recently[1], where AI skeptics were labeled as "nuts", and hallucinations were "(more or less) a solved problem".

This seems to be exactly the kind of results we would expect from a system that hallucinates, has no semantic understanding of the content, and is little more than a probabilistic text generator. This doesn't mean that it can't be useful when placed in the right hands, but it's also unsurprising that human non-experts would use it to cut corners in search of money, power, and glory, or worse—actively delude, scam, and harm others. Considering that the latter group is much larger, it's concerning how little thought and resources are put into implementing _actual_ safety measures, and not just ones that look good in PR statements.

[1]: https://news.ycombinator.com/item?id=44163063

Re: Deep learning gets the glory, deep fact checking gets ignored

#77

Post by halpow »

r3trohack3r wrote: post #68
softwaredoug wrote: post #29

We also love deep cherry picking. Working hard to find that one awesome time some ML / AI thing worked beautifully and shouting its praises to the high heavens. Nevermind the dozens of other times we tried and failed...

Dude. I just asked my computer to write [ad lib basic utility script] and it spit out a syntactically correct C program that does it with instructions for compiling it. And then I asked it for [ad lib cocktail request] and got back thorough instructions. We did that with sand. That we got from the ground. And taught it to talk. And write C programs. Never mind what? That I had to ask twice? Or five times? What maximu…

Crows and parrots are amazing talkers too, but there's a hard limit to how much sense they make. Do you want those birds to teach your kids and serve you medicine?

Re: Deep learning gets the glory, deep fact checking gets ignored

#78

Post by darkoob12 »

amelius wrote: post #2

Before making AI do research, perhaps we should first let it __reproduce__ research. For example, give it a paper of some deep learning technique and make it produce an implementation of that paper. Before it can do that, I have no hope that it can produce novel ideas.

Reproduciblity was never a serious issue in AI research community. I think one of the main reasons for explosive progress in AI was the open community and people could easily reproduce other people's research. If you look at top tier conferences you see that they share everything paper, latex, code, data, lecture video etc.

After ChatGPT big cooperations stopped sharing their main research but it still happens at academia.

Re: Deep learning gets the glory, deep fact checking gets ignored

#79

Post by [deleted] »

mike_hearn wrote: post #70
SamuelAdams wrote: post #36

Earlier quoted context omitted.

Every system has problems. The better question is: what is the acceptable threshold? For an example Medicare and Medicade had a fraud rate of 7.66%. Yes, that is a lot of billions, and there is room for improvement, but that doesn’t mean the entire system is failing: 93% of cases are being covered as intended. The same could be said with these models. If the spoilage rate is 10%, does that mean the whole system is ba…

I think it's worth being highly skeptical about fraud rates that are stated to two decimal places of precision. Fraud is by design hard to accurately detect. It would be more accurate to say, Medicare decides 7.66% of its cases are fraudulent according to its own policies and procedures, which are likely conservative, and cannot take into account undetected fraud. The true rate is likely higher, perhaps much higher.…

[deleted]

Re: Deep learning gets the glory, deep fact checking gets ignored

#80

Post by disgruntledphd2 »

ErigmolCt wrote: post #69
b0a04gl wrote: post #66

Man, I’ve been there. Tried throwing BERT at enzyme data once—looked fine in eval, totally flopped in the wild. Classic overfit-on-vibes scenario. Honestly, for straight-up classification? I’d pick SVM or logistic any day. Transformers are cool, but unless your data’s super clean, they just hallucinate confidently. Like giving GPT a multiple-choice test on gibberish—it will pick something, and say it with its chest.…

Transformers will ace your test set, then faceplant the second they meet reality. I've also done the "wow, 92% accuracy!" dance only to realize later I just built a very confident pattern-matcher for my dataset quirks.

Honestly, if your accuracy/performance metrics are too good, that's almost a sure sign that something has gone wrong.

Source: bitter, bitter experience. I once predicted the placebo effect perfectly using a random forest (just got lucky with the train/test split). Although I'd left academia at that point, I often wonder if I'd have dug in deeper if I'd needed a high impact paper to keep my job.

Post reply on HN