11–20 of 113 posts
> Unfortunately, most scientific fields have succumbed to AI hype, leading to a suspension of common sense. For example, a line of research in political science claimed to predict the onset of civil war with an accuracy2 of well over 90%, a number that should sound facially impossible. (It turned out to be leakage, which is what got us interested in this whole line of research.) This coupled with people acting on its…
We use search that way, don’t see why AI trained on similar content wouldn’t be just variable in terms of reliability.
Traditional pre-spam-overload Google results could often give a high quality answer, or if not, you’d at least get the sense of the low quality. Not so with LLMs.
> Unfortunately, most scientific fields have succumbed to AI hype, leading to a suspension of common sense. For example, a line of research in political science claimed to predict the onset of civil war with an accuracy2 of well over 90%, a number that should sound facially impossible. (It turned out to be leakage, which is what got us interested in this whole line of research.) This coupled with people acting on its…
The problem is you can't just "use logic and reason" because simple models are not good enough. The nuance dominates, but that's why we have experts.
What's funny to me is that people will confidently argue with experts and others value their opinion over the expert's knowledge. But on the other hand, people tend to just take machines at face value. Maybe these aren't overlapping groups, but it does appear that way. There's a great irony in trusting a machine but not the person/s that built said machine.
Some even think that the end goal is actually an autonomous research agent that can make decisions about what questions to ask and why, and that's one of the true marks of AGI. That to me is insane and misses the entire point of science altogether, even once we reach that technical feasibility. We ask questions about the universe to expand our human relationship with the universe, not to just amass more research capital for the sake of it. And the fact that the AI snake oil has infected big chunks of science reveals which parts of it are just gold rush speculation and which aren't.
There's a more fundamental challenge of training scientists to understand why we ask the questions we ask. You can't just offload that to some background task and trust that it makes sense.
Not just scientists, but everyone! My partner recently went a bit nuts writing an article with the help of GPT4. She was very proud of how productive she'd been until I asked if she'd actually searched for the papers GPT4 had referred to. Of course, many of the referred to papers didn't exist...
I’ve said from the outset that in academic settings you should be able to cite an AI as a writing assistant, it would clear up a lot of the confusion about its use. If you used it poorly it’s still on you, but at least there’s some transparency by which to judge the work.
> Unfortunately, most scientific fields have succumbed to AI hype, leading to a suspension of common sense. For example, a line of research in political science claimed to predict the onset of civil war with an accuracy2 of well over 90%, a number that should sound facially impossible. (It turned out to be leakage, which is what got us interested in this whole line of research.) This coupled with people acting on its…
How would that work, do you think?
The worst is having random people questioning your expertise because of what ChatGPT told them.
1) people confidently arguing with domain experts about topics that they have little to no experience in.
2) people valuing the opinions of arguers from 1 over experts.
It is it because it's actually shit but it's the best thing we've seen yet and everyone is just in denial?
Not just scientists, but everyone! My partner recently went a bit nuts writing an article with the help of GPT4. She was very proud of how productive she'd been until I asked if she'd actually searched for the papers GPT4 had referred to. Of course, many of the referred to papers didn't exist...
Sadly it's not even just references, LLMs still hallucinate or at least misrepresent even the most basic of facts. That and the stereotypical GPT-verbage makes it impossible to use for writing anything significant.
Keep in mind that there's no difference between what happens inside a model when it "hallucinates" vs. when it generates "correct" output. It's the exact same process.