Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Are LLMs So Gullible?

amistrongeryet.substack.com

Post reply on HN
Print view View original

61–70 of 107 posts

Re: Why Are LLMs So Gullible?

#61

Post by nomel »

jfengel wrote: post #24
nomel wrote: post #14

Earlier quoted context omitted.

Comparing the intelligence of machine learning models that are designed to emulate human cognition and logical, to a well understood stage of human cognition and logic, is completely logical, and completely aligned with the purpose of the ML model's existence.

LLMs don't talk like children, and they don't seem to be taking that development path. They are trained on adult language. They don't make the kinds of mistakes children make and it doesn't look like we could use children as a model for improvement. It's entirely possible that LLMs will one day emulate adult speech without ever passing through the child development stages. The stages it takes to get there will be dis…

> They don't make the kinds of mistakes children make and it doesn't look like we could use children as a model for improvement.

I'm not sure how familiar with LLM or children you are, but in my experience, they absolutely make the same types of logic/"critical thinking" mistakes that children do, as this article you're replying under demonstrates.

Re: Why Are LLMs So Gullible?

#62

Post by [deleted] »

orbital-decay wrote: post #23
mg74 wrote: post #19

Earlier quoted context omitted.

Statistical optimization is a known process; one where we understand every step, and can therefore instruct machines on how to perform it. Cognitive reasoning is still today not understood (in the Von Neumann sense) by anyone.

That's my point. How can you compare the two if one is not even well-defined, let alone understood?

[deleted]

Re: Why Are LLMs So Gullible?

#63

Post by mrtksn »

nyrikki wrote: post #59
mrtksn wrote: post #34

Earlier quoted context omitted.

I used to think like that but I'm not so sure anymore. The statistical optimisation thing is an analytical approach to Neural Networks but its similar to saying that love is just hormones.

Real question, do you really want to know where you may be mistaken, or will you hand wave away information in order to protect what you believe? No judgement here but I am just tired of sharing information with people to explain why while interpretation of a complex model may be hard, we know the methods of how PAC learning works and some hard boundaries on what it can do. Obviously we need people to push boundaries…

Shoot. Tell me that information.

Re: Why Are LLMs So Gullible?

#64

Post by famouswaffles »

BazookaMusic wrote: post #43
Jensson wrote: post #36

Earlier quoted context omitted.

Isn't love just hormones? It isn't rational reasoning at least.

A simple proof that love is not just hormones is that love can last for some time. If it was just a chemical phenomenon why does it happen repeatedly. Why can someone feel love for someone just by bringing to mind the symbol which represents that person? Why can someone feel love by seeing an illustration of someone they love?

How is that a proof of anything?

Do you imagine all chemical reactions to be instantaneous with effects that last only a short while ?

>Why can someone feel love for someone just by bringing to mind the symbol which represents that person?

>Why can someone feel love by seeing an illustration of someone they love?

Why would other things triggering hormones and reactions mean it's not just hormones and reactions?

Re: Why Are LLMs So Gullible?

#65

Post by trott »

snewman wrote: post #56
kingkongjaffa wrote: post #4

because the output isn't the result of cognitive reasoning, it's the result of a statistical optimization problem where the goal is maximum acceptance by the user. these tools and approaches are neither gullible nor not-gullble.

(author here) Of course LLMs are not people. But human metaphors can (sometimes!) be useful in understanding, explaining, and even enhancing their behavior. For instance, techniques such as Chain-of-Thought prompting explicitly apply techniques that work well for people to improve the reasoning ability of LLMs. A point I attempt to make in this article is that one reason reason LLMs are so vulnerable to jailbreaks an…

Very good article. I wonder why it disappeared from HN? More comments than up-votes?

Re: Why Are LLMs So Gullible?

#66

Post by yongjik »

CharlesW wrote: post #16
kingkongjaffa wrote: post #4

because the output isn't the result of cognitive reasoning, it's the result of a statistical optimization problem where the goal is maximum acceptance by the user. these tools and approaches are neither gullible nor not-gullble.

Thank you for this. I think it's important that technical folks in particular not anthropomorphize LLMs, and help less technical people understand how they work and that they lack consciousness, emotions, and understanding.

Consciousness and emotions, sure. Understanding, I'm not so sure.

Saying that LLMs cannot understand concepts because "it's just statistical pattern matching" is not going to help laypeople understand what LLMs are. After all, what is a human brain if not a biological pattern matching machine?

Re: Why Are LLMs So Gullible?

#67

Post by cyanydeez »

CharlesW wrote: post #16
kingkongjaffa wrote: post #4

because the output isn't the result of cognitive reasoning, it's the result of a statistical optimization problem where the goal is maximum acceptance by the user. these tools and approaches are neither gullible nor not-gullble.

Thank you for this. I think it's important that technical folks in particular not anthropomorphize LLMs, and help less technical people understand how they work and that they lack consciousness, emotions, and understanding.

except the LLM has no goal, apriori, when _you_ are using it.

the goals of the trainer yes, but from the user, there's no goal.

Re: Why Are LLMs So Gullible?

#68

Post by blackbear_ »

kingkongjaffa wrote: post #5
sitharus wrote: post #3

With enough effort and priming you can trick _people_ in to believing things which are clearly untrue. Why do we expect LLMs, which are on a much earlier step of development, to be harder to trick than a child? LLMs at the moment are really advanced autocomplete - they can fill in the next step of conversation, but they don't understand the question and respond with abstract reasoning. Yet.

> they don't understand the question and respond with abstract reasoning. Yet. What makes you think LLM's as a class of technology will ever have the capacity to really do this. I thought that no matter how big a model gets it's never actually 'thinking'. All those prompts like 'think step by step' are just helpers along the way, because as you say it's 'really advanced autocomplete'

In principle you could attach a reasoning module to the neural network and only use the LLM part of the network for input/output.

The design of such a reasoning module is an exercise left to the reader, obviously :p

Re: Why Are LLMs So Gullible?

#69

Post by cyanydeez »

Affric wrote: post #18
CharlesW wrote: post #16

Earlier quoted context omitted.

Thank you for this. I think it's important that technical folks in particular not anthropomorphize LLMs, and help less technical people understand how they work and that they lack consciousness, emotions, and understanding.

You mean these highly anthropomorphised programs? It’s important that technical people don’t anthropomorphise them? I agree but the creators of all the main LLMs have already crossed the line by a long way. E.g. It’s deeply troubling that it’s acceptable that LLMs deliver inline apologies.

it's kinda amusing because the LLMs evolved via selection of the best anthropic traits because the basic digested mountains of undiscerning information.

so the gullibility is merely one selected for trait convolution with state.

Re: Why Are LLMs So Gullible?

#70

Post by nomel »

Alchemista wrote: post #57
nomel wrote: post #54

Earlier quoted context omitted.

> LLMs aren't designed to emulate human cognition I'm not sure that's fair, or correct. The behavior of an LLM is set by the design of the combination of loss function and training data. Achieving it is the desired result of the selection of those. > It happens that they seem to exhibit some similarities to human cognition as a side effect Yes, the desired behavior of nearly all LLM projects is to emulate the capabil…

> You are the first to suggest any such thing, in this comment chain. I'm sorry, did you miss the original post: "Because they are at a child level of development. Give it a few years." I extrapolated a little bit, but not much. They were clearly implying that it is on a similar developmental path as a child.

It's not clear to me, since they used "at a" not "on a", as you've reworded it.

> Because they are at a child level of development.

I read this as, "Because the capabilities of LLM are at the child level of development."

Post reply on HN