Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A non-anthropomorphized view of LLMs

addxorrol.blogspot.com

Post reply on HN
Print view View original

351–360 of 432 posts

Re: A non-anthropomorphized view of LLMs

#351

Post by Applejinx »

jibal wrote: post #289
CuriousSkeptic wrote: post #193

Earlier quoted context omitted.

> Why don't LLMs get frustrated with you if you ask them the same question repeatedly? To be fair, I have had a strong sense of Gemini in particular becoming a lot more frustrated with me than GPT or Claude. Yesterday I had it ensuring me that it was doing a great job, it was just me not understanding the challenge but it would break it down step by step just to make it obvious to me (only to repeat the same errors,…

Point out to an LLM that it has no mental states and thus isn't capable of being frustrated (or glad that your program works or hoping that it will, etc. ... I call them out whenever they ascribe emotions to themselves) and they will confirm that ... you can coax from them quite detailed explanations of why and how it's an illusion. Of course they will quickly revert to self-anthropomorphizing language, even after pr…

Of course this is deeply problematic because it's a cloud of HUMAN response. This is why 'they will' get frustrated or creepy if you mess with them, give repeating data or mind game them: literally all it has to draw on is a vast library of distilled human responses and that's all the LLM can produce. This is not an argument with jibal, it's a 'yes and'.

You can tell it 'you are a machine, respond only with computerlike accuracy' and that is you gaslighting the cloud of probabilities and insisting it should act with a personality you elicit. It'll do what it can, in that you are directing it. You're prompting it. But there is neither a person there, nor a superintelligent machine that can draw on computerlike accuracy, because the DATA doesn't have any such thing. Just because it runs on lots of computers does not make it a computer, any more than it's a human.

Re: A non-anthropomorphized view of LLMs

#352

Post by jumploops »

> I cannot begin putting a probability on "will this human generate this sequence".

Welcome to the world of advertising!

Jokes aside, and while I don't necessarily believe transformers/GPUs are the path to AGI, we technically already have a working "general intelligence" that can survive on just an apple a day.

Putting that non-artificial general intelligence up on a pedestal is ironically the cause of "world wars and murderous ideologies" that the author is so quick to defer to.

In some sense, humans are just error-prone meat machines, whose inputs/outputs can be confined to a specific space/time bounding box. Yes, our evolutionary past has created a wonderful internal RNG and made our memory system surprisingly fickle, but this doesn't mean we're gods, even if we manage to live long enough to evolve into AGI.

Maybe we can humble ourselves, realize that we're not too different from the other mammals/animals on this planet, and use our excess resources to increase the fault tolerance (N=1) of all life from Earth (and come to the realization that any AGI we create, is actually human in origin).

Re: A non-anthropomorphized view of LLMs

#353

Post by UncleOxidant »

grey-area wrote: post #120
Al-Khwarizmi wrote: post #117

I have the technical knowledge to know how LLMs work, but I still find it pointless to not anthropomorphize, at least to an extent. The language of "generator that stochastically produces the next word" is just not very useful when you're talking about, e.g., an LLM that is answering complex world modeling questions or generating a creative story. It's at the wrong level of abstraction, just as if you were discussing…

On the contrary, anthropomorphism IMO is the main problem with narratives around LLMs - people are genuinely talking about them thinking and reasoning when they are doing nothing of that sort (actively encouraged by the companies selling them) and it is completely distorting discussions on their use and perceptions of their utility.

It's not just distorting discussions it's leading people to put a lot of faith in what LLMs are telling them. Was just on a zoom an hour ago where a guy working on a startup asked ChatGPT about his idea and then emailed us the result for discussion in the meeting. ChatGPT basically just told him what he wanted to hear - essentially that his idea was great and it would be successful ("if you implement it correctly" was doing a lot of work). It was a glowing endorsement of the idea that made the guy think that he must have a million dollar idea. I had to be "that guy" who said that maybe ChatGPT was telling him what he wanted to hear based on the way the question was formulated - tried to be very diplomatic about it and maybe I was a bit too diplomatic because it didn't shake his faith in what ChatGPT had told him.

Re: A non-anthropomorphized view of LLMs

#354

Post by pmg101 »

jibal wrote: post #284
pmg101 wrote: post #261

Earlier quoted context omitted.

I remember Dawkins talking about the "intentional stance" when discussing genes in The Selfish Gene. It's flat wrong to describe genes as having any agency. However it's a useful and easily understood shorthand to describe them in that way rather than every time use the full formulation of "organisms who tend to possess these genes tend towards these behaviours." Sometimes to help our brains reach a higher level of a…

The intentional stance was Daniel Dennett's creation and a major part of his life's work. There are actually (exactly) three stances in his model: the physical stance, the design stance, and the intentional stance. https://en.wikipedia.org/wiki/Intentional_stance I think the design stance is appropriate for understanding and predicting LLM behavior, and the intentional stance is not.

Thanks for the correction. I guess both thinkers took a somewhat similar position and I somehow remembered Dawkins's argument but Dennett's term. The term is memorable.

Do you want to describe WHY you think the design stance is appropriate here but the intentional stance is not?

Re: A non-anthropomorphized view of LLMs

#355

Post by lelanthran »

JimDabell wrote: post #227
flir wrote: post #154

Earlier quoted context omitted.

Agreeing with you, this is a "can a submarine swim" problem IMO. We need a new word for what LLMs are doing. Calling it "thinking" is stretching the word to breaking point, but "selecting the next word based on a complex statistical model" doesn't begin to capture what they're capable of. Maybe it's cog-nition (emphasis on the cog).

> this is a "can a submarine swim" problem IMO. We need a new word for what LLMs are doing. Why? A plane is not a fly and does not stay aloft like a fly, yet we describe what it does as flying despite the fact that it does not flap its wings. What are the downsides we encounter that are caused by using the word “fly” to describe a plane travelling through the air?

> A plane is not a fly and does not stay aloft like a fly, yet we describe what it does as flying despite the fact that it does not flap its wings.

Flying doesn't mean flapping, and the word has a long history of being used to describe inanimate objects moving through the air.

"A rock flies through the window, shattering it and spilling shards everywhere" - see?

OTOH, we have never used to word "swim" in the same way - "The rock hit the surface and swam to the bottom" is wrong!

Re: A non-anthropomorphized view of LLMs

#356

Post by th0ma5 »

peeters wrote: post #266

> The moment that people ascribe properties such as "consciousness" or "ethics" or "values" or "morals" to these learnt mappings is where I tend to get lost. We are speaking about a big recurrence equation that produces a new word, and that stops producing words if we don't crank the shaft. If that's the argument, then in my mind the more pertinent question is should you be anthropomorphizing humans, Larry Ellison or…

I think you to as he is human, but I respect your desire to question it!

Re: A non-anthropomorphized view of LLMs

#357

Post by 01HNNWZ0MV43FF »

ryeats wrote: post #349
flir wrote: post #154

Earlier quoted context omitted.

Agreeing with you, this is a "can a submarine swim" problem IMO. We need a new word for what LLMs are doing. Calling it "thinking" is stretching the word to breaking point, but "selecting the next word based on a complex statistical model" doesn't begin to capture what they're capable of. Maybe it's cog-nition (emphasis on the cog).

It's more like muscle memory than cognition. So maybe procedural memory but that isn't catchy.

They certainly do act like a thing which has a very strong "System 1" but no "System 2" (per Thinking, Fast And Slow)

Re: A non-anthropomorphized view of LLMs

#358

Post by TimTheTinker »

UncleOxidant wrote: post #353
grey-area wrote: post #120

Earlier quoted context omitted.

On the contrary, anthropomorphism IMO is the main problem with narratives around LLMs - people are genuinely talking about them thinking and reasoning when they are doing nothing of that sort (actively encouraged by the companies selling them) and it is completely distorting discussions on their use and perceptions of their utility.

It's not just distorting discussions it's leading people to put a lot of faith in what LLMs are telling them. Was just on a zoom an hour ago where a guy working on a startup asked ChatGPT about his idea and then emailed us the result for discussion in the meeting. ChatGPT basically just told him what he wanted to hear - essentially that his idea was great and it would be successful ("if you implement it correctly" wa…

LLMs directly exploit a human trust vuln. Our brains tend to engage with them relationally and create an unconscious functional belief that an agent on the other end is responding with their real thoughts, even when we know better.

AI apps ought to at minimum warn us that their responses are not anyone's (or anything's) real thoughts. But the illusion is so powerful that many people would ignore the warning.

Re: A non-anthropomorphized view of LLMs

#359

Post by lelanthran »

Xss3 wrote: post #223
latexr wrote: post #160

Earlier quoted context omitted.

Respectfully, that is a reflection of the places you hang out in (like HN) and not the reality of the population. Outside the technical world it gets much worse. There are people who killed themselves because of LLMs, people who are in love with them, people who genuinely believe they have “awakened” their own private ChatGPT instance into AGI and are eschewing the real humans in their lives.

The other day a good friend of mine with mental health issues remarked that "his" chatgpt understands him better than most of his friends and gives him better advice than his therapist. It's going to take a lot to get him out of that mindset and frankly I'm dreading trying to compare and contrast imperfect human behaviour and friendships with a sycophantic AI.

> The other day a good friend of mine with mental health issues remarked that "his" chatgpt understands him better than most of his friends and gives him better advice than his therapist.

The therapist thing might be correct, though. You can send a well-adjusted person to three renowned therapists and get three different reasons for why they need to continue sessions.

No therapist ever says "Congratulations, you're perfectly normal. Now go away and come back when you have a real problem." Statistically it is vanishingly unlikely that every person who ever visited a therapist is in need of a second (more more) visit.

The main problem with therapy is a lack of objectivity[1]. When people talk about what their sessions resulted in, it's always "My problem is that I'm too perfect". I've known actual bullies whose therapist apparently told them that they are too submissive and need to be more assertive.

The secondary problem is that all diagnosis is based on self-reported metrics of the subject. All improvement is equally based on self-reported metrics. This is no different from prayer.

You don't have a medical practice there; you've got an Imam and a sophisticated but still medically-insured way to plead with thunderstorms[2]. I fail to see how an LLM (or even the Rogerian a-x doctor in Emacs) will do worse on average.

After all, if you're at a therapist and you're doing most of the talking, how would an LLM perform worse than the therapist?

----------------

[1] If I'm at a therapist, and they're asking me to do most of the talking, I would damn well feel that I am not getting my moneys worth. I'd be there primarily to learn (and practice a little) whatever tools they can teach me to handle my $PROBLEM. I don't want someone to vent at, I want to learn coping mechanisms and mitigation strategies.

[2] This is not an obscure reference.

Re: A non-anthropomorphized view of LLMs

#360

Post by godelski »

Tijdreiziger wrote: post #306
scarface_74 wrote: post #244

Earlier quoted context omitted.

Wait until a conversation about “serverless” comes up and someone says there is no such thing because there are servers somewhere as if everyone - especially on HN -doesn’t already know that.

Why would everyone know that? Not everyone has experience in sysops, especially not beginners. E.g. when I first started learning webdev, I didn’t think about ‘servers’. I just knew that if I uploaded my HTML/PHP files to my shared web host, then they appeared online. It was only much later that I realized that shared webhosting is ‘just’ an abstraction over Linux/Apache (after all, I first had to learn about those t…

I think they fumbled with wording but I interpreted them as meaning "audience of HN" and it seems they confirmed.

We always are speaking to our audience, right? This is also what makes more general/open discussions difficult (e.g. talking on Twitter/Facebook/etc). That there are many ways to interpret anything depending on prior knowledge, cultural biases, etc. But I think it is fair that on HN we can make an assumption that people here are tech savvy and knowledgeable. We'll definitely overstep and understep at times, but shouldn't we also cultivate a culture where it is okay to ask and okay to apologize for making too much of an assumption?

I mean at the end of the day we got to make some assumptions, right? If we assume zero operating knowledge then comments are going to get pretty massive and frankly, not be good at communicating with a niche even if better at communicating with a general audience. But should HN be a place for general people? I think no. I think it should be a place for people interested in computers and programming.

Post reply on HN