Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scientists should use AI as a tool, not an oracle

aisnakeoil.com

Post reply on HN
Print view View original

41–50 of 113 posts

Re: Scientists should use AI as a tool, not an oracle

#41

Post by EGreg »

flatline wrote: post #15
LouisSayers wrote: post #4

Not just scientists, but everyone! My partner recently went a bit nuts writing an article with the help of GPT4. She was very proud of how productive she'd been until I asked if she'd actually searched for the papers GPT4 had referred to. Of course, many of the referred to papers didn't exist...

That is not writing with the help of GPT 4, that is letting it write for you! I can’t imagine doing anything creative and letting a computer source material for me without having reviewed the material first hand, even if it was accurate. Clearly, this is not where everyone’s head is at, and I suspect your wife’s workflow is more the common case. I’ve said from the outset that in academic settings you should be able t…

You sound like the people who used to know how to fix a car, or sew, or write cursive, or do multiplication times tables in their head, or know how to derive a formula, or check a mathematical proof.

Ask anyone below 30 if they can write cursive today, or know their times tables hehe. Ask them if they can derive a formula instead of using Mathematica.

Or ask a developer if they know how their pixel shaders work, or what’s going on under the hood of their favorite runtime, how hash tables work, or really anything. Previous generations did. When the complexity gets too high people just trust the machines I guess.

And no one actually knows what the LLM internals are anyway.

Re: Scientists should use AI as a tool, not an oracle

#42

Post by skrap »

...but why wouldn't they use AI as an oracle? From an outsider's perspective, it seems that there's already plenty of incentive to test the margins of acceptable academic practice in order to produce more papers or publish more quickly. Sadly I feel like it'll become the norm to have a chatbot interpret your results and write your paper rather than using those expensive grad students.

I don't have answers; just the lingering question "why are we building this?"

Re: Scientists should use AI as a tool, not an oracle

#45

Post by [deleted] »

LouisSayers wrote: post #4

Not just scientists, but everyone! My partner recently went a bit nuts writing an article with the help of GPT4. She was very proud of how productive she'd been until I asked if she'd actually searched for the papers GPT4 had referred to. Of course, many of the referred to papers didn't exist...

[deleted]

Re: Scientists should use AI as a tool, not an oracle

#46

Post by colechristensen »

LouisSayers wrote: post #4

Not just scientists, but everyone! My partner recently went a bit nuts writing an article with the help of GPT4. She was very proud of how productive she'd been until I asked if she'd actually searched for the papers GPT4 had referred to. Of course, many of the referred to papers didn't exist...

GPT-whatever can’t do sources.

I was trying to use it as a research tool and it hallucinated 95% of the references I asked for (not a made up percentage, I counted)

Ironically the one real source turned out to be quite useful.

Re: Scientists should use AI as a tool, not an oracle

#47

Post by foreverobama »

unkulunkulu wrote: post #5

> Unfortunately, most scientific fields have succumbed to AI hype, leading to a suspension of common sense. For example, a line of research in political science claimed to predict the onset of civil war with an accuracy2 of well over 90%, a number that should sound facially impossible. (It turned out to be leakage, which is what got us interested in this whole line of research.) This coupled with people acting on its…

[dead]

Re: Scientists should use AI as a tool, not an oracle

#48

Post by Terr_ »

raincole wrote: post #37
SrslyJosh wrote: post #20

Earlier quoted context omitted.

> LLMs still hallucinate Keep in mind that there's no difference between what happens inside a model when it "hallucinates" vs. when it generates "correct" output. It's the exact same process.

Uh... yes? I'm not sure why it's some significant insight. Surely when google gives bad results, it's "the same process" as when it gives good results. And when a book gives wrong information, it's the exact same kind of ink as correct information.

I think the point is that it's not some kind of bug to find and fix, it's a fundamental risk with the entire approach.

Re: Scientists should use AI as a tool, not an oracle

#50

Post by Terr_ »

EGreg wrote: post #40
LouisSayers wrote: post #4

Not just scientists, but everyone! My partner recently went a bit nuts writing an article with the help of GPT4. She was very proud of how productive she'd been until I asked if she'd actually searched for the papers GPT4 had referred to. Of course, many of the referred to papers didn't exist...

Hmm. In the future the AI in nefarious hands can retroactively make the papers first, and get them past the censors. Just make up a lot of bullshit and then it’s turtles all the way down lmao

I'm imagining how much easier it would have made work for the Ministry of Truth in 1984.
Post reply on HN