Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Are LLMs So Gullible?

amistrongeryet.substack.com

Post reply on HN
Print view View original

81–90 of 107 posts

Re: Why Are LLMs So Gullible?

#81

Post by dist-epoch »

BazookaMusic wrote: post #43
Jensson wrote: post #36

Earlier quoted context omitted.

Isn't love just hormones? It isn't rational reasoning at least.

A simple proof that love is not just hormones is that love can last for some time. If it was just a chemical phenomenon why does it happen repeatedly. Why can someone feel love for someone just by bringing to mind the symbol which represents that person? Why can someone feel love by seeing an illustration of someone they love?

Thinking about food makes your mouth salivate.

Thinking about someone could release the hormones.

Re: Why Are LLMs So Gullible?

#82

Post by kypro »

mattlutze wrote: post #41
kypro wrote: post #37

Earlier quoted context omitted.

This depends on perspective. I could argue the issue isn't that it's gullible but misaligned. In the case of the napalm Grandma it seems odd to me that you're suggesting the LLM is stupid because it's answering in a way that makes sense given its prompt. The issue doesn't necessarily suggest a lack of reasoning, but that the LLM is trusting the human. For the record, I agree with you – I would have thought that an AI…

> For the record, I agree with you – I would have thought that an AI that can reason well would probably know when not to trust humans, but I suppose that assumes it values preventing humans creating napalm over being correct and helpful. Do we want LLMs, and later other multi-modal / servo systems, that are deciding they can't trust a human prompter and taking actions based on that? >... and that we must find a way…

> Tongue in cheek or actual argument here?

I think it's interesting that there is no clear answer – do we want AIs to trust us all the time, or is an aligned AI counterintuitively one that often distrusts us and perhaps sometimes even lies to us?

I thought it was interesting that the parent commenter suggested that the reason LLMs are so trusting is because they can't reason anyway. It would implying that in the future when AIs are smarter they'll be more distrusting of us, and that this is a good thing. We should question that I think. Even if there is some middle ground here it seems like a really really hard problem to solve – especially if we want to build an LLMs that are trustworthy and truthful.

Re: Why Are LLMs So Gullible?

#83

Post by daveguy »

thfuran wrote: post #22
Alchemista wrote: post #12

Earlier quoted context omitted.

This is silly, that's like talking about building a fusion reactor modeled after the sun. It is easy to propose something like that, but we always seem to be 10 years away from realizing it. In fact it could be easier to solve the fusion problem than trying to build a machine/software that closely approximates a human brain as you suggest. Yes it would be wonderful if sci-fi was real, but we need to deal in what is p…

Yes, that was obviously an extreme example. But we know that it is possible in reality to implement a physical system that does what we call thinking. There is, I think, no particular reason to suppose that it's physically impossible re-implement the functionality with much less meat. Supposing that you've done this, you then just need to more clearly define "thinking" and "LLM" to determine whether changing that re-…

I think you underestimate the efficiency of meat.

Re: Why Are LLMs So Gullible?

#84

Post by daveguy »

genman wrote: post #35
Alchemista wrote: post #12

Earlier quoted context omitted.

This is silly, that's like talking about building a fusion reactor modeled after the sun. It is easy to propose something like that, but we always seem to be 10 years away from realizing it. In fact it could be easier to solve the fusion problem than trying to build a machine/software that closely approximates a human brain as you suggest. Yes it would be wonderful if sci-fi was real, but we need to deal in what is p…

Perhaps the question was not fully understood. A function is simply a mapping from one state into another. In principle we can define it over what ever state. As such we can consider a human thinking as kind of a function. ANN are function effectively approximators. The question was - if an ANN would very closely approximate the "human thinking" function then can we still say that it is not thinking?

In order to train an ANN to approximate the "human thinking" function, we would have to know that function well enough to give it examples and counterexamples. Currently we are only training LLMs to approximate the "human blabbing" function.

Re: Why Are LLMs So Gullible?

#85

Post by daveguy »

blackbear_ wrote: post #68
kingkongjaffa wrote: post #5

Earlier quoted context omitted.

> they don't understand the question and respond with abstract reasoning. Yet. What makes you think LLM's as a class of technology will ever have the capacity to really do this. I thought that no matter how big a model gets it's never actually 'thinking'. All those prompts like 'think step by step' are just helpers along the way, because as you say it's 'really advanced autocomplete'

In principle you could attach a reasoning module to the neural network and only use the LLM part of the network for input/output. The design of such a reasoning module is an exercise left to the reader, obviously :p

Yeah, LLM + Reasoning Module is only as good as its weakest part. And we had more than one AI winter around the development of reasoning modules. I think you're 2x correct -- we need both and the exercise to create it is worthy of :p

Re: Why Are LLMs So Gullible?

#86

Post by daveguy »

kypro wrote: post #78
reaperman wrote: post #45

Earlier quoted context omitted.

> the LLM is trusting the human > an AI that can reason well would probably know when not to trust humans > it values preventing humans creating napalm over being correct and helpful. > Maybe it just doesn't share our values > prioritises being honest and helpful. > they are too trusting and too honest > an LLM that is more distrusting and deceptive Current LLM's do/have/feel literally none of these things. They do n…

Okay, now prove that please. I was trying to present a crappy philosophical point – that the difference between a gullible AI and an unaligned one is fundamentally unknowable. Any evidence you point to as proof that an AI is bad at reasoning, I can point to as evidence of misalignment. Like I say, whether the AI acts "gullible" because it lacks reasoning ability or is too trusting really just depends on your perspect…

Trust, reasoning, priorities, values, bias, desires... To attribute any of those to an AI in a general sense is an extraordinary claim. Therefore, the burden of proof is on you. The fact that it is so "gullible" demonstrates a lack of most of these. You seem to be twisting a lot of superficial feelings about LLMs into an argument without any proof... confusing poorly tuned statistical responses with bias and value.

Re: Why Are LLMs So Gullible?

#87

Post by thfuran »

daveguy wrote: post #83
thfuran wrote: post #22

Earlier quoted context omitted.

Yes, that was obviously an extreme example. But we know that it is possible in reality to implement a physical system that does what we call thinking. There is, I think, no particular reason to suppose that it's physically impossible re-implement the functionality with much less meat. Supposing that you've done this, you then just need to more clearly define "thinking" and "LLM" to determine whether changing that re-…

I think you underestimate the efficiency of meat.

Efficiency has absolutely nothing to do with it.

Re: Why Are LLMs So Gullible?

#88

Post by chx »

famouswaffles wrote: post #58
chx wrote: post #53

Earlier quoted context omitted.

Sure thing. People really don't learn the history of AI any more apparently or this question wouldn't come up all the time. There is basically any number of questions you can ask a two year old human who have never encountered that question nor anything even remotely similar to it and yet they can answer without fail. Meanwhile absolutely no AI can answer these unless the specific question / the rules underlying the…

You've not quantified anything, nor have you even provided any valid example of these so abundant mystery questions. If it's so easy, come up with one and show us.

If I do then it'll get gobbled up by fake AI and I won't make up one every time ignorant people who can't open an AI textbook demand one from me.

Apropos: https://existentialcomics.com/comic/289

You could use that comics to generate a few of these questions.

Re: Why Are LLMs So Gullible?

#89

Post by Alchemista »

thfuran wrote: post #87
daveguy wrote: post #83

Earlier quoted context omitted.

I think you underestimate the efficiency of meat.

Efficiency has absolutely nothing to do with it.

Efficiency has everything to do with it. To run your original thought experiment would likely take more computing capacity than humanity will ever produce with current silicon based processors.

It could be that we can "get there" with much rougher approximation, but that is by no means a given. If "getting there" does require a much more complex model that actually involves simulating human neurons with much more accuracy and precision, the computing power and energy needed, again based on semiconductor computers, might simply be intractable.

Re: Why Are LLMs So Gullible?

#90

Post by daveguy »

orbital-decay wrote: post #51
Alchemista wrote: post #29

Earlier quoted context omitted.

Don't you think the onus should be on the people making the fantastical claims to prove it? If human cognition is ill-defined, then define it before making grand claims like ML models being on some path of childhood development and is a few steps from being an adult.

I think that neither side has a convincing argument in these discussions, actually, I'm pointing out that both extremes - anthropomorphism and stochastic parrot - lack any foundation, so I wouldn't be so confident. The behavior of neither statistical models nor biological systems is well understood. It's entirely possible that every trait you consider human can naturally emerge from a dumb statistical model, and inde…

The stochastic parrot is much more grounded in the physical reality of how the models are structured, trained, and produce outputs than the "it's like people" argument. Statistical models are much better understood than biological systems. That said, your penultimate statement is true:

> It's entirely possible that every trait you consider human can naturally emerge from a dumb statistical model, and indeed certain processes are remarkably similar as the models get bigger, smarter, and have better training data.

I just don't think it's likely given the biological complexity we are washing over with the statistical model.

Post reply on HN