Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Being “Confidently Wrong” is holding AI back

promptql.io

Post reply on HN
Print view View original

191–200 of 274 posts

Re: Being “Confidently Wrong” is holding AI back

#191

Post by ninetyninenine »

kilpikaarna wrote: post #121
ninetyninenine wrote: post #72

Earlier quoted context omitted.

No. The experts in the field are past this argument. People have moved on. It is clear to everyone who builds LLMs that the AI is intelligent. The algorithm was autocomplete, but we are finding as an autocomplete bot is basically autocompleting things with humanity changing intelligent content. Your opinion is a minority now and not shared by people on the forefront of building these things. Your holding onto the ini…

> It is clear to everyone who builds LLMs that the AI is intelligent. So presumably we have a solid, generally-agreed-upon definition on intelligence now? > autocompleting things with humanity changing intelligent content. What does this even mean?

We do it’s fuzzy but we do. You point to a rock all humans say it’s not intelligent. You point to a human all humans say it is intelligent.

Because we can do this, by logic a universally agreed upon definition exists. Otherwise we wouldn’t be able to do this.

Of course the boundaries between what’s not intelligent and what is, is where things are not as universally agreed upon. Which is what you’re referring to and unlike you I am charitably addressing that nuance rather then saying some surface level bs.

The thing is the people who say the LLM (which obviously exists at this fuzzy categorical boundary) is not intelligent will have logical paradoxes and inconsistencies when they examine there own logic.

The whole thing is actually a vocabulary problem as this boundary line is an arbitrary definition given to a made up word that humans created. But one can still say an LLM is well placed in the category of intelligent not by some majority vote but because that placement is the only one that maintains logical consistency with OTHER entities or things all humans place in the intelligent bucket.

For example a lot of people in this thread say intelligence requires actual real time learning, therefore an LLM is NOT intelligent. But then there are humans who literally have anterograde amnesia and they literally cannnot learn. Are they not intelligent? Things like this are inconsistent and it happens frequently when you place LLMs in the not intelligent bucket.

State your reasoning for why your stance is "not intelligent" and I can point out where the inconsistencies lie.

Re: Being “Confidently Wrong” is holding AI back

#192

Post by ctoth »

jqpabc123 wrote: post #94
rar00 wrote: post #57

I know people are pushing back, taking "only" literally, but from a reasonable perspective what causes LLMs (technically their outputs) to give that impression is indeed the crux of what holds progress back: how/what LLMs learn from data. In my personal opinion, there's something fundamentally flawed the whole field has yet to properly pinpointing and fix.

there's something fundamentally flawed the whole field has yet to properly pinpointing and fix. Isn't it obvious? It's all built around probability and statistics. This is not how you reach definitive answers. Maybe the results make sense and maybe they're just nice sounding BS. You guess which one is the case. The real catch --- if you know enough to spot the BS, you probably didn't need to ask the question in the f…

> It's all built around probability and statistics.

Yes, the world is probabilistic.

> This is not how you reach definitive answers.

Do go on? This is the only way to build anything approximating certainty in our world. Do you think that ... answers just exist? What type of weird deterministic video game world do you live in where this is not the case?

Re: Being “Confidently Wrong” is holding AI back

#193

Post by pxc »

For programming, at least, there are also problems with overall output quality, instruction following, and the scopes of changes.

LLMs don't do well at following style instructions, and existing memory systems aren't adequate for "remembering" my style preferences.

When you ask for one change, you often get loads of other changes alongside it. Transformers suck at targeted edits.

The hallucination problem and the sycophancy/suggestibility problem (which perhaps both play into the phenomenon of being "confidently wrong") are both real and serious. But they hardly form a singular bottleneck for the usefulness of LLMs.

Re: Being “Confidently Wrong” is holding AI back

#194

Post by Dwedit »

captainclam wrote: post #172

Wow, there really is an xkcd for everything.

Those are original cartoons drawn in the style of XKCD. But strangely enough, in the second cartoon, the Megan clone seems to change from a thin stick figure to suddenly wearing clothes?

I'm not sure if the comic was AI-assisted or not. AI-generated images do not usually contain identical pixel data when a panel repeats.

Re: Being “Confidently Wrong” is holding AI back

#195

Post by kelseyfrog »

brookst wrote: post #188
traceroute66 wrote: post #32

Earlier quoted context omitted.

> is their willingness to correct themselves when asked Except they don't correct themselves when asked. I'm sure we've all been there, many, many, many,many,many times .... - User: "This is wrong because X" - AI: "You're absolutely right ! Here's a production-ready fixed answer" - User: "No, that's wrong because Y" - AI: "I apologise for frustrating you ! Here's a robust answer that works" - User: "You idiot, you ju…

You're conflating "correct themselves" with "are guaranteed to give the correct answer", which are two really different things. And in fact you're just echoing GP's point: their corrections can be wrong. You case is no different from: - AI: "The capital of France is Paris" - User: "This is wrong, it changed to Montreal in 2005" - AI: "You're absolutely right! The capital of France is Montreal"

Instead I get this:

    Nope—Paris is the capital of France and has been for centuries. Montreal is in Quebec, Canada. France’s presidency (Élysée), parliament (Assemblée nationale and Sénat), and ministries are all in Paris.

Re: Being “Confidently Wrong” is holding AI back

#196

Post by zero_iq »

traceroute66 wrote: post #32
lucideer wrote: post #8

While the thrust of this article is generally correct, I have two issues with it: 1. The words "the only thing" massively underplays the difficulty of this problem. It's not a small thing. 2. One of the issues I've seen with a lot of chat LLMs is their willingness to correct themselves when asked - this might seem, on the surface, to be a positive (allowing a user to steer the AI toward a more accurate or appropriate…

> is their willingness to correct themselves when asked Except they don't correct themselves when asked. I'm sure we've all been there, many, many, many,many,many times .... - User: "This is wrong because X" - AI: "You're absolutely right ! Here's a production-ready fixed answer" - User: "No, that's wrong because Y" - AI: "I apologise for frustrating you ! Here's a robust answer that works" - User: "You idiot, you ju…

I've seen ChatGPT get stuck in this loop all by itself, generating a long multi-page answer where it constantly catches itself, refutes itself, offers a new answer with the same problem, rinse and repeat... All in the same response!

Re: Being “Confidently Wrong” is holding AI back

#197

Post by recursive »

vidarh wrote: post #139
ACCount37 wrote: post #68

Earlier quoted context omitted.

1-turn instruction following and multi-turn instruction following are not the same exact capability, and some AIs only "get good" at the former. 1-turn gets more training attention - because it's more noticeable, in casual use and benchmarks both, and also easier to train for. With weak multi-turn instruction following, context data will often dominate over user instructions. Resulting in very "loopy" AI - and more s…

This is a good point, and to drive this home to people, if you have a conversation of this pattern: User: Fix this problem ... Assistant: X User: No, don't do X Assistant: Y User: No, Y is wrong too. Assistant: X It is generally pointless to continue. You now have a context that is full of the assistant explaining to you and itself why X and Y are the right answers, and much less context of you explaining why it is w…

This is also a thing that's bad about LLMs. You're holding it wrong if you continue to argue. But LLMs are presented as if we can use the conventions of natural language to communicate with them. That's how they're sold. So if they fail to live up to those expectations, that's still a problem with LLMs.

Re: Being “Confidently Wrong” is holding AI back

#198

Post by corytheboyd »

kovacs wrote: post #190
corytheboyd wrote: post #127

Isn’t it obvious that the confidently wrong problem will never go away because all of this is effectively built on a statistical next token matcher? Yeah sure you can throw on hacks like RAG, more context window, but it’s still built on the same foundation. It’s like saying you built a 3D scene on a 2D plane. You can employ clever tricks to make 2D look 3D at the right angle, buts it’s fundamentally not 3D, which obv…

This is the best analogy I've read to explain what's going on and takes me back to the days of Doom and how it was so transformative at the time. Perhaps in time the current generation will be viewed as the Doom engine as we await the holy grail of full 3D in Quake.

I guess technically 3D on computers is still clever 2D, but let’s not break the metaphor down too hard lol. Love the Doom/Quake comparison!

Re: Being “Confidently Wrong” is holding AI back

#199

Post by corytheboyd »

yifanl wrote: post #132
corytheboyd wrote: post #127

Isn’t it obvious that the confidently wrong problem will never go away because all of this is effectively built on a statistical next token matcher? Yeah sure you can throw on hacks like RAG, more context window, but it’s still built on the same foundation. It’s like saying you built a 3D scene on a 2D plane. You can employ clever tricks to make 2D look 3D at the right angle, buts it’s fundamentally not 3D, which obv…

There are people convinced that if we throw a sufficient amount of training data and VC money at more hardware, we'll overcome the gap. Technically, I can't prove that they're wrong, novel solutions sometimes happen, and I guess the calculus is that it's likely enough to justify a trillion dollars down the hole.

I’d say it was worth throwing down some cash for, because we get cool new things by full-assing new ideas. But… yeah… a TRILLION dollars is waaaay too far.

Re: Being “Confidently Wrong” is holding AI back

#200

Post by jazzyjackson »

mettamage wrote: post #46
rwmj wrote: post #6

Only thing? Just off the top of my head: That the LLM doesn't learn incrementally from previous encounters. That we appear to have run out of training data. That we seem to have hit a scaling wall (reflected in the performance of GPT5). I predict we'll get a few research breakthroughs in the next few years that will make articles like this seem ridiculous.

> Only thing? Just off the top of my head: That the LLM doesn't learn incrementally from previous encounters. That we appear to have run out of training data. Ha, that almost seems like an oxymoron. The previous encounters can be the new training data!

The old training was human responses to human questions. From this the bot learned to mimick human responses.

What would be the point of training an LLM on bot answers to human questions? This is only useful if you want to get an LLM that behaves like an already existing LLm

Post reply on HN