Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Multi-Level View of LLM Intentionality

disagreeableme.blogspot.com

Post reply on HN
Print view View original

11–20 of 75 posts

Re: A Multi-Level View of LLM Intentionality

#11

Post by Icko »

lukev wrote: post #9

I'm not sure the definition of "intention" the article suggests is a useful one. He tries to make it sound like he's being conservative: > That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it really has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case). And…

> Primarily, they are pure functions that accept a sequence of tokens and return the next token. The model itself is stateless, and it doesn't seem right to me to ascribe "intent" to a stateless function. Even if the function is capable of modeling certain aspects of chess.

I have two arguments against. One, you could argue that state is transferred between the layers. It may be inelegant for each chain of state transitions to be the same length, but it seems to work. Two, it may not have "states", but if the end result is the same, does it matter?

Re: A Multi-Level View of LLM Intentionality

#12

Post by lawlessone »

pixl97 wrote: post #8
lawlessone wrote: post #4

Earlier quoted context omitted.

I've found they fall apart after a couple of moves and lose track of the game. Edit: This might not be the case anymore it seems, my below point doesn't actually contradict you, seems it matters a lot how you tell the model your moves. Also saying things like "move my rightmost pawn" completely confuses them.

The token model of LLMs doesn't map well into how human experience the world of informational glyphs. Left and right is a intrinsic quality of our vision system. An LLM has to map the idea of left and right into symbols via text and line breaks. I do think it will be interesting as visual input and internal graphical output is integrated with text based LLMs as that should help correct their internal experience to be…

" An LLM has to map the idea of left and right into symbols via text and line breaks."

Oh yeah that's i suggested it :)

I do wonder though if we give the LLMs enough examples of texts with people describing their relative spatial position to each other and things will it eventually "learn" to work things these out a bit better

Re: A Multi-Level View of LLM Intentionality

#13

Post by haltist »

lukev wrote: post #9

I'm not sure the definition of "intention" the article suggests is a useful one. He tries to make it sound like he's being conservative: > That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it really has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case). And…

Ascribing human properties to computers and software has always seemed very bizarre to me. I always assume people are confused when they do that. There is no meaningful intersection between biology, intelligence, and computers but people constantly keep trying to imbue electromagnetic signal processors with human/biological qualities very much like how children attribute souls to teddy bears.

Computers are mechanical gadgets that work with electricity. Humans (and other animals) die when exposed to the kinds of currents flowing through computers. Similarly, I have never seen a computer drink water (for obvious reasons). If properties are reduced to behavioral outcomes then maybe someone can explain to me why computers are so averse to water.

Re: A Multi-Level View of LLM Intentionality

#14

Post by version_five »

lukev wrote: post #9

I'm not sure the definition of "intention" the article suggests is a useful one. He tries to make it sound like he's being conservative: > That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it really has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case). And…

  Not because of their capabilities or behaviors, but because we know how they work (mechanically at least) and it is incompatible with useful definitions of the word "intent."
I've never seen this deter anyone. I can't understand how people that know how they work can have such ridiculous ideas about llms.

I'd add though that inference is clearly fixed but there is some more subtlety about training. Gradient descent clearly doesnt have intelligence, intent (in the sense meant), consciousness either, but it's not stateless like inference and you could argue has a rudimentary "intent" in minimizing loss.

Re: A Multi-Level View of LLM Intentionality

#15

Post by famouswaffles »

lawlessone wrote: post #12
pixl97 wrote: post #8

Earlier quoted context omitted.

The token model of LLMs doesn't map well into how human experience the world of informational glyphs. Left and right is a intrinsic quality of our vision system. An LLM has to map the idea of left and right into symbols via text and line breaks. I do think it will be interesting as visual input and internal graphical output is integrated with text based LLMs as that should help correct their internal experience to be…

" An LLM has to map the idea of left and right into symbols via text and line breaks." Oh yeah that's i suggested it :) I do wonder though if we give the LLMs enough examples of texts with people describing their relative spatial position to each other and things will it eventually "learn" to work things these out a bit better

>I do wonder though if we give the LLMs enough examples of texts with people describing their relative spatial position to each other and things will it eventually "learn" to work things these out a bit better

GPT-4's spatial position understanding is actually really good all things considered. By the end, 4 was able to construct an accurate maze just from feedback about the current position and possible next moves after each move by GPT-4.

https://ekzhu.medium.com/gpt-4s-maze-navigation-a-deep-dive-...

I think we just don't write much about moving through space and that is why reasoning about it is more limited.

Re: A Multi-Level View of LLM Intentionality

#16

Post by jedharris »

lukev wrote: post #9

I'm not sure the definition of "intention" the article suggests is a useful one. He tries to make it sound like he's being conservative: > That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it really has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case). And…

The article provides a very clear reason for using the idea of "intention": that framing helps us understand and predict the behavior. In contrast framing a mathematical function as having "intention" doesn't help. The underlying mechanism isn't relevant to this criterion.

Clearly the system we're understanding as "intentional" has state; we can engage in multi-round interactions. It doesn't matter that we can separate the mutable state from the function that updates that state.

Re: A Multi-Level View of LLM Intentionality

#17

Post by version_five »

haltist wrote: post #13
lukev wrote: post #9

I'm not sure the definition of "intention" the article suggests is a useful one. He tries to make it sound like he's being conservative: > That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it really has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case). And…

Ascribing human properties to computers and software has always seemed very bizarre to me. I always assume people are confused when they do that. There is no meaningful intersection between biology, intelligence, and computers but people constantly keep trying to imbue electromagnetic signal processors with human/biological qualities very much like how children attribute souls to teddy bears. Computers are mechanical…

He was inspired by lesswrong which from my scan more mysticism and philosophy (with a handful of self importance) than anything about how computers work. Advanced technology is magic to laypeople. It's like how some people believe in homeopathy. If you don't understand how medicine works, it's just a different kind of magic.

Re: A Multi-Level View of LLM Intentionality

#18

Post by lo_zamoyski »

lukev wrote: post #9

I'm not sure the definition of "intention" the article suggests is a useful one. He tries to make it sound like he's being conservative: > That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it really has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case). And…

Uh, hold on. That's not what's meant by intentionality. No one is talking about what a machine intends to do. In philosophy, and specifically philosophy of mind, "intentionality" is, briefly, "the power of minds and mental states to be about, to represent, or to stand for, things, properties and states of affairs" [0].

So the problem with this guy's definition of intentionality is, first, that it's a redefinition. If you're interested in whether a machine can possess intentionality, you won't find the answer in interpretivism, because that's no longer a meaningful question.

Intentionality presupposes telos, so if you assume a metaphysical position that rules out telos, such as materialism, then, by definition, you cannot have "aboutness", and therefore, no intentionality of any sort.

[0] https://plato.stanford.edu/entries/intentionality/

Re: A Multi-Level View of LLM Intentionality

#19

Post by lo_zamoyski »

jedharris wrote: post #16
lukev wrote: post #9

I'm not sure the definition of "intention" the article suggests is a useful one. He tries to make it sound like he's being conservative: > That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it really has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case). And…

The article provides a very clear reason for using the idea of "intention": that framing helps us understand and predict the behavior. In contrast framing a mathematical function as having "intention" doesn't help. The underlying mechanism isn't relevant to this criterion. Clearly the system we're understanding as "intentional" has state; we can engage in multi-round interactions. It doesn't matter that we can separa…

A mathematical function isn't a mechanism. It has no causal power.

Re: A Multi-Level View of LLM Intentionality

#20

Post by jedharris »

passion__desire wrote: post #7
labrador wrote: post #3

The authors of the text the model was trained on certainly had intentions. Many of those are going to be preserved in the output.

Can we say ChatGPT or its future versions would be like an instantiation of the Boltzmann Brain concept if it has internal qualia? The "brain" comes alive with the rich structure only to disappear after the chat session is over.

Cool way of putting it. Let's run with that. A good actor can be seen as instantiating a Boltzmann Brain while on stage -- especially when improvising (as always may be needed). Maybe each of us is instantiating some superposition of Boltzmann Brains in everyday life as we wend our way through various social roles...

From now on I'll listen for the subtle popping sounds as these BBs get instantiated and de-instantiated all around me...

Of course a philosopher can object that (1) these BBs are on a substrate that's richer than they are so aren't "really" BBs and (2) they often leave traces that are available to them in later instantiations which again classical BBs can't. So maybe make up another name -- but a great way to think.

Post reply on HN