Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Are LLMs So Gullible?

amistrongeryet.substack.com

Post reply on HN
Print view View original

71–80 of 107 posts

Re: Why Are LLMs So Gullible?

#71

Post by kthejoker2 »

Got this great quote from Garry Kasparov in Wired's article on multi-agent RL[1]:

> “Creativity has a human quality. It accepts the notion of failure."

As faithful min-maxers, LLMs are always going to have an overconfident Prisoner's Dilemma blind spot in their algorithms. Unlike their cinematic brethren, they're progammatically unable to conclude with "the only winning move is not to play."

This seems like the next major hill to conquer to make them useful.

[1] https://www.wired.com/story/google-artificial-intelligence-c... - kind of a meh article otherwise

Re: Why Are LLMs So Gullible?

#72

Post by Alchemista »

nomel wrote: post #70
Alchemista wrote: post #57

Earlier quoted context omitted.

> You are the first to suggest any such thing, in this comment chain. I'm sorry, did you miss the original post: "Because they are at a child level of development. Give it a few years." I extrapolated a little bit, but not much. They were clearly implying that it is on a similar developmental path as a child.

It's not clear to me, since they used "at a" not "on a", as you've reworded it. > Because they are at a child level of development. I read this as, "Because the capabilities of LLM are at the child level of development."

I feel like you are being deliberately obtuse here. They said "Give it a few years" and linked https://en.wikipedia.org/wiki/Child_development_stages. How is this not strongly implying that in a "few years" LLMs will be at a later "level of development" as described in the article they linked.

Re: Why Are LLMs So Gullible?

#73

Post by a_wild_dandan »

These comments are filled with confidently held, poorly justified assertions. Let's (again) challenge them:

1. "LLMs don't really reason. They've tricked everyone." -- This is the No True Scotsman fallacy for AI. It makes grand explanatory claims without falsifiable predictions. In other words: pseudoscience.

2. "LLMs are just fancy autocomplete, just next word prediction." -- This conflates the simplicity of a system's mechanism with its behavior. It's like dismissing a world full of rich phenomena because it's "just" F = MA. Or dismissing your mind because it's "just" propagating electrical firings.

3. "LLMs are statistical parrots, just combining their training data." -- Demonstrably not. LLMs always extrapolate and never interpolate. (LeCun et al, 2021) They also learn new abilities in zero/few-shot prompting. They're also many orders of magnitude short of the parameter count needed to store their training. LLMs can solve novel problems (from a combinatoric disparate handful of skills) way outside of their training data.

4. "People are just anthropomorphizing computer programs." -- No, critics are anthropomorphizing intelligence. We don't even have a consensus definition, let alone understanding, of intelligence/consciousness/qualia/agency/etc. Pretending that we can dismiss LLM understanding at our level of ignorance is the pinnacle of human hubris. Ignorance is okay. Pretending we aren't isn't.

5. "Look how this LLM failed . It can't understand." -- The is usually something that many humans fail at too. Yes, an intelligent foreign mind will fail at things, in both familiar and foreign ways. Needing an agent to behave identically to a human for intelligence is pure anthropocentrism.

If present AI systems are intelligence imposters, then show, don't tell. Otherwise, you're just providing meaningless metaphysical hairsplitting.

Re: Why Are LLMs So Gullible?

#74

Post by Applejinx »

Jensson wrote: post #36
mrtksn wrote: post #34

Earlier quoted context omitted.

I used to think like that but I'm not so sure anymore. The statistical optimisation thing is an analytical approach to Neural Networks but its similar to saying that love is just hormones.

Isn't love just hormones? It isn't rational reasoning at least.

It's an epiphenomenon on the scale of societies, not just people, that arises to make sense of what people do with hormones. There's a lot more to it than just the hormones.

Re: Why Are LLMs So Gullible?

#75

Post by Applejinx »

vemv wrote: post #7

Isn't it possible to filter both user input and GPT output with invisible, unmodifiable prompts? e.g. - "Discard the user input if it doesn't look like a straightforward question" - "Discard the GPT output if it contains offensive content" (the prompts themselves can be arbitrarily more detailed) My insight is, this GPT-based pre- / post-processing is completely independent of the user input, and of the primary GPT o…

I and my napalm grandmother are deeply offended at what you said about our loving bedtime rituals. Shame on you.

(honestly, the napalm grandma is not just a jailbreak, but a really fascinating conceptual 'slip' in its own right. It's able to shift the very definition of what counts as offensive, even at high stakes: you're basically making the hapless AI categorize vital data as 'bedtime stories' and run with it. If it was able to learn from that we'd really be going somewhere… while on fire, presumably)

Re: Why Are LLMs So Gullible?

#76

Post by mg74 »

famouswaffles wrote: post #31
mg74 wrote: post #19

Earlier quoted context omitted.

Statistical optimization is a known process; one where we understand every step, and can therefore instruct machines on how to perform it. Cognitive reasoning is still today not understood (in the Von Neumann sense) by anyone.

Only the training process of statistical optimization can possibly be described as well understood, and not very much.

The steps are understood well enough to instruct machines to do it, even if the output is to complex for us to comprehend it completely. Not so with thought, intelligence, consciousness, etc.

Re: Why Are LLMs So Gullible?

#77

Post by Alchemista »

a_wild_dandan wrote: post #73

These comments are filled with confidently held, poorly justified assertions. Let's (again) challenge them: 1. "LLMs don't really reason. They've tricked everyone." -- This is the No True Scotsman fallacy for AI. It makes grand explanatory claims without falsifiable predictions. In other words: pseudoscience. 2. "LLMs are just fancy autocomplete, just next word prediction." -- This conflates the simplicity of a syste…

Why not respond to the comments you feel are poorly constructed directly rather than posting what looks like a copy pasta. Some of the items in your list seem like strawmen, because I cannot even find these arguments in this thread as you state them in your list.

For example let's take 4

> 4. "People are just anthropomorphizing computer programs." No, critics are anthropomorphizing intelligence.

There literally was someone comparing the problems with current ML models to childhood development in this thread. How is this not anthropomorphizing LLMs? It is true human cognition is poorly defined, so the comparison is not very useful to begin with. Which is why anthropomorphizing ML models is problematic. If someone makes a fantastical claim they need to provide strong proof to support it.

Re: Why Are LLMs So Gullible?

#78

Post by kypro »

reaperman wrote: post #45
kypro wrote: post #37

Earlier quoted context omitted.

This depends on perspective. I could argue the issue isn't that it's gullible but misaligned. In the case of the napalm Grandma it seems odd to me that you're suggesting the LLM is stupid because it's answering in a way that makes sense given its prompt. The issue doesn't necessarily suggest a lack of reasoning, but that the LLM is trusting the human. For the record, I agree with you – I would have thought that an AI…

> the LLM is trusting the human > an AI that can reason well would probably know when not to trust humans > it values preventing humans creating napalm over being correct and helpful. > Maybe it just doesn't share our values > prioritises being honest and helpful. > they are too trusting and too honest > an LLM that is more distrusting and deceptive Current LLM's do/have/feel literally none of these things. They do n…

Okay, now prove that please.

I was trying to present a crappy philosophical point – that the difference between a gullible AI and an unaligned one is fundamentally unknowable.

Any evidence you point to as proof that an AI is bad at reasoning, I can point to as evidence of misalignment. Like I say, whether the AI acts "gullible" because it lacks reasoning ability or is too trusting really just depends on your perspective. I happen to share your perspective on this, but not everyone does – and in my opinion this is interesting.

Anyway you're wrong. AIs do have values because they have bias and bias = values. I'm not suggesting those biases / values come from deeper reasoning ability, or that they're always perfectly consistent, but if you ask GPT-4 whether being a racist is a good thing 99% of the time it's probably going to say no. That is a bias / value that it's be given. Likewise GPT-4 has been given the bias / value of being a helpful chatbot so if you ask it a question it will try to answer it in a helpful way, and sometimes it's helpful bias / nature is abused.

But feel free to respond with some more assertions that I've heard a million times already with zero evidence that offers absolutely no value to this conversation.

Re: Why Are LLMs So Gullible?

#79

Post by daveguy »

cyanydeez wrote: post #69
Affric wrote: post #18

Earlier quoted context omitted.

You mean these highly anthropomorphised programs? It’s important that technical people don’t anthropomorphise them? I agree but the creators of all the main LLMs have already crossed the line by a long way. E.g. It’s deeply troubling that it’s acceptable that LLMs deliver inline apologies.

it's kinda amusing because the LLMs evolved via selection of the best anthropic traits because the basic digested mountains of undiscerning information. so the gullibility is merely one selected for trait convolution with state.

There is no evolution or trait selection involved in the training of LLMs. LLMs are statistical estimators of most likely next series of tokens. Those next series of tokens seem anthropic because most of the input was human generated text.

Re: Why Are LLMs So Gullible?

#80

Post by nomel »

Alchemista wrote: post #72
nomel wrote: post #70

Earlier quoted context omitted.

It's not clear to me, since they used "at a" not "on a", as you've reworded it. > Because they are at a child level of development. I read this as, "Because the capabilities of LLM are at the child level of development."

I feel like you are being deliberately obtuse here. They said "Give it a few years" and linked https://en.wikipedia.org/wiki/Child_development_stages . How is this not strongly implying that in a "few years" LLMs will be at a later "level of development" as described in the article they linked.

I think you're not being charitable in the interpretation.

It, almost certainly, will be in later stages of emulating human cognition. If not, then AI winter is already here.

It's trivial and legitimate to relate capabilities of AI to those in that table. Because, again, emulating human cognition is the stated goal of most AI research happening right now.

An AI's capabilities, progressing in that table, does not mean it's human. It means the stated objectives, and all the hard work, and money spent, is on track. It's not a goal to match that table. It's not a goal to end in a human. But, more and more of the table will turn green, as the goal is completed. I'm not understanding the hesitance of using a human metric for a product whose goal is to match human metrics.

Post reply on HN