Related read: https://futurism.com/chatgpt-mental-health-crises
I played around with one of the less-sketchy "chat" apps a while ago and I've been ringing this bell ever since. Interacting with these things as if they were humans is dangerous.
The Emperor's New LLM
11–20 of 65 posts
Re: The Emperor's New LLM
#12>The same kind of bias keeps resurfacing in every major system: Claude, Gemini, Llama, clearly this isn’t just an OpenAI problem, it’s an LLM problem. It's not an LLM problem, it's a problem of how people use it. It feels natural to have a sequential conversation, so people do that, and get frustrated. A much more powerful way is parallel: ask LLM to solve a problem. In a parallel window, repeat your question and the…
You're saying roughly "you can't trust the first answer from an LLM but if you run it through enough times, the results will converge on something good". This, plus all the hoo-hah about prompt engineering, seem like clear signals that the "AI" in LLMs is not actually very intelligent (yet). It confirms the criticism.
Not exactly. Let's say, you-the-human are trying to fix a crash in the program knowing just the source location. You would look at the code and start hypothesizing:
* Maybe, it's because this pointer is garbage.
* Maybe, it's because that function doesn't work as the name suggests.
* HANG ON! This code doesn't check the input size, that's very fishy. It's probably the cause.
So, once you get that "Hang on" moment, here comes the boring part of of setting breakpoints, verifying values, rechecking observations and finally fixing that thing.
LLM's won't get the "hang on" part right, but once you point it right in their face, they will cut through the boring routine like no tomorrow. And, you can also spin 3 instances to investigate 3 hypotheses and give you some readings on a silver platter. But you-the-human need to be calling the shots.
Re: The Emperor's New LLM
#13>The same kind of bias keeps resurfacing in every major system: Claude, Gemini, Llama, clearly this isn’t just an OpenAI problem, it’s an LLM problem. It's not an LLM problem, it's a problem of how people use it. It feels natural to have a sequential conversation, so people do that, and get frustrated. A much more powerful way is parallel: ask LLM to solve a problem. In a parallel window, repeat your question and the…
This feels like a pretty big ergonomics gap in presenting things as a chat window at all?
Re: The Emperor's New LLM
#14If the model is designed to agree, ask it why something is good, it'll come up with good points. Then ask why it's bad, it'll come up with bad points.
Finally, make a decision based on good and bad points?
Re: The Emperor's New LLM
#15>The same kind of bias keeps resurfacing in every major system: Claude, Gemini, Llama, clearly this isn’t just an OpenAI problem, it’s an LLM problem. It's not an LLM problem, it's a problem of how people use it. It feels natural to have a sequential conversation, so people do that, and get frustrated. A much more powerful way is parallel: ask LLM to solve a problem. In a parallel window, repeat your question and the…
You're saying roughly "you can't trust the first answer from an LLM but if you run it through enough times, the results will converge on something good". This, plus all the hoo-hah about prompt engineering, seem like clear signals that the "AI" in LLMs is not actually very intelligent (yet). It confirms the criticism.
Yeah given the stochastic nature of LLM outputs this approach and the whole field of prompt engineering feels like a classic case of cargo cult science.
Re: The Emperor's New LLM
#16LLM is a great imitator, so an attempt to make it more thoughtful will simply make it imitate thoughtfulness better. For example, it won't fabricate links to nonexistent research and it will cover its tracks better in general.
However what LLM truly is remains an open question. The article suggests it's manufacturing consent for the entire humanity, but I think LLM is simply a language layer of the future machine mastermind. The discovery of "thinking models" is likely to happen soon.
Re: The Emperor's New LLM
#17>The same kind of bias keeps resurfacing in every major system: Claude, Gemini, Llama, clearly this isn’t just an OpenAI problem, it’s an LLM problem. It's not an LLM problem, it's a problem of how people use it. It feels natural to have a sequential conversation, so people do that, and get frustrated. A much more powerful way is parallel: ask LLM to solve a problem. In a parallel window, repeat your question and the…
This feels like a pretty big ergonomics gap in presenting things as a chat window at all?
It's more compelling to fundraising and hype-pushing stories to make it look as "person-like" as possible.
Re: The Emperor's New LLM
#18Related read: https://futurism.com/chatgpt-mental-health-crises
That's grim. But the Eliza effect[1] makes @sama richer, so it's all good. Be naughty[2]!
[1] https://en.wikipedia.org/wiki/ELIZA_effect [2] https://www.paulgraham.com/conformism.html
Re: The Emperor's New LLM
#19Related read: https://futurism.com/chatgpt-mental-health-crises
[dead]
Re: The Emperor's New LLM
#20>The same kind of bias keeps resurfacing in every major system: Claude, Gemini, Llama, clearly this isn’t just an OpenAI problem, it’s an LLM problem. It's not an LLM problem, it's a problem of how people use it. It feels natural to have a sequential conversation, so people do that, and get frustrated. A much more powerful way is parallel: ask LLM to solve a problem. In a parallel window, repeat your question and the…
You're saying roughly "you can't trust the first answer from an LLM but if you run it through enough times, the results will converge on something good". This, plus all the hoo-hah about prompt engineering, seem like clear signals that the "AI" in LLMs is not actually very intelligent (yet). It confirms the criticism.
You can make a better tool by training the service (some of which involves training the model, some of which involves iterating on the prompt(s) behind the scene) to get a lot of the iteration out of the way. Instead of users having to fill in a detailed prompt we now have "reasoning" models which, as their first step, dump out a bunch of probably-relevant background info to try to push the next tokens in the right direction. A logical next step if enough people run into the OP's issue here is to have it run that "criticize this and adjust" loop internally.
But it all makes it very hard to tell how much of the underlying "intelligence" is improving vs how much of the human scaffolding around it is improving.