Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Multi-Level View of LLM Intentionality

disagreeableme.blogspot.com

Post reply on HN
Print view View original

51–60 of 75 posts

Re: A Multi-Level View of LLM Intentionality

#51

Post by thfuran »

sharedbeans wrote: post #41
lukev wrote: post #9

I'm not sure the definition of "intention" the article suggests is a useful one. He tries to make it sound like he's being conservative: > That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it really has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case). And…

They are extraordinarily complicated pure functions, to explore the entire space would take lifetime of the universe ^^^ lifetime of the universe or some absurd quantity like that. (The operator is titration.) Further, what happens when you give an LLM a bank of long-term storage and a read-modify-write loop around it? A sufficiently advanced "modify" function would be more than enough to give rise to intent even in…

>Further, what happens when you give an LLM a bank of long-term storage and a read-modify-write loop around it?

You create a very different sort of system, for one. Saying that because doing that in just the might way could yield a system with intention, an LLM has intention is rather like saying that my refrigerator is a sandwich.

Re: A Multi-Level View of LLM Intentionality

#52

Post by simplify »

benlivengood wrote: post #48
lukev wrote: post #9

I'm not sure the definition of "intention" the article suggests is a useful one. He tries to make it sound like he's being conservative: > That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it really has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case). And…

Humans, too, would likely be nearly stateless if we took a point-in-time snapshot of them and repeatedly simulated them from that point on various short (4000ms, similar to 4000 tokens) sequences of nerve impulses. Nevertheless the human would be acting intentionally (for in-distribution impulse patterns) for the brief period of simulation. Fine-tuning and RLHF seem to impart more intentionality to the pure stateless…

> Humans, too, would likely be nearly stateless if we took a point-in-time snapshot...

I highly doubt that would ever be possible in practice, as our inputs are much too complex. But I want to point out, you're basically saying here "humans are nearly stateless if we take a snapshot of their state and simulate that state..."

Re: A Multi-Level View of LLM Intentionality

#53

Post by ftxbro »

passion__desire wrote: post #7
labrador wrote: post #3

The authors of the text the model was trained on certainly had intentions. Many of those are going to be preserved in the output.

Can we say ChatGPT or its future versions would be like an instantiation of the Boltzmann Brain concept if it has internal qualia? The "brain" comes alive with the rich structure only to disappear after the chat session is over.

are our real brains boltzmann brains that have internal qualia and come alive with the rich structure only to lose it when we die

Re: A Multi-Level View of LLM Intentionality

#54

Post by x-complexity »

simplify wrote: post #52
benlivengood wrote: post #48

Earlier quoted context omitted.

Humans, too, would likely be nearly stateless if we took a point-in-time snapshot of them and repeatedly simulated them from that point on various short (4000ms, similar to 4000 tokens) sequences of nerve impulses. Nevertheless the human would be acting intentionally (for in-distribution impulse patterns) for the brief period of simulation. Fine-tuning and RLHF seem to impart more intentionality to the pure stateless…

> Humans, too, would likely be nearly stateless if we took a point-in-time snapshot... I highly doubt that would ever be possible in practice, as our inputs are much too complex. But I want to point out, you're basically saying here "humans are nearly stateless if we take a snapshot of their state and simulate that state..."

> I highly doubt that would ever be possible in practice, as our inputs are much too complex. But I want to point out, you're basically saying here "humans are nearly stateless if we take a snapshot of their state and simulate that state..."

1) We don't have infinite inputs

2) We (our brains) don't have infinite processing

3) We (our brains) don't have infinite lossless storage: Our brains often perform pruning of unimportant information.

Given that there is an ultimately finite upper bound of both # of inputs & processing power & storage, at some point, the simulation of a human from a given snapshot is reductively possible.

Re: A Multi-Level View of LLM Intentionality

#55

Post by naasking »

lukev wrote: post #9

I'm not sure the definition of "intention" the article suggests is a useful one. He tries to make it sound like he's being conservative: > That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it really has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case). And…

> The model itself is stateless, and it doesn't seem right to me to ascribe "intent" to a stateless function.

Statefulness can be modelled statelessly so "statelessness" is not a sufficient reason to dismiss intentionality. The only question is whether the relationship between the inputs of the function and its outputs correspond to what we might call "intent", which the cited definition attempts to outline. Obviously it's only a high-level view that leaves many details unanswered.

> Otherwise, we are in the somewhat absurd position of needing to argue that all mathematical functions "intend" to yield their result.

Not all mathematical functions have a continuity of internal identity and self-reference as seems to be the case with LLMs.

Re: A Multi-Level View of LLM Intentionality

#56

Post by lmm »

skissane wrote: post #40
quickthrower2 wrote: post #39

Earlier quoted context omitted.

Not sure I understand. Software (which is a mathematical function) runs on a processor and that is arguably a mechanism, or allows them to emerge, such as a button or input field.

> Software (which is a mathematical function) Software isn't a mathematical function. Software may be an embodiment of a mathematical function, but isn't a mathematical function itself. Mathematical functions are much more abstract than software–although exactly how much more abstract depends on which position you take in the philosophy of mathematics. For a mathematical Platonist, a mathematical function is an etern…

> By contrast, software is something which has a physical location (on this hard disk), it was created at a certain point, and will likely one day cease to exist (when the last copy is destroyed).

"Software" is a broad term, but it could certainly be taken to mean something more abstract than that. Sometimes a program written for a different CPU architecture, or not written for any CPU at all, is still recognisably "the same" program. Euclid's algorithm might well be considered "software", but it's very much the same kind of thing as a mathematical function.

Re: A Multi-Level View of LLM Intentionality

#57

Post by sfink »

labrador wrote: post #3

The authors of the text the model was trained on certainly had intentions. Many of those are going to be preserved in the output.

Came to say this.

Chess move training data is much more likely to consist of examples of people trying to win than it is to consist of random legal or even illegal moves. You could argue that the intention belongs to the people providing the input and not the LLM, but that seems like a distinction without a difference.

Re: A Multi-Level View of LLM Intentionality

#58

Post by labrador »

sfink wrote: post #57
labrador wrote: post #3

The authors of the text the model was trained on certainly had intentions. Many of those are going to be preserved in the output.

Came to say this. Chess move training data is much more likely to consist of examples of people trying to win than it is to consist of random legal or even illegal moves. You could argue that the intention belongs to the people providing the input and not the LLM, but that seems like a distinction without a difference.

True, because the LLM has no intention (sorry Blake Lemoine.) I'm puzzled about what people say about AI, when to me it is simply a talking library. I started going to public libraries at a young age (I'm 66) so to me I'm talking to the authors of the books, not some mysterious conscious machine.

Re: A Multi-Level View of LLM Intentionality

#59

Post by intended »

After having spent a ridiculous amount of effort to get LLMs to work, I am certain they are simply predicting the next token.

If LLMs actually could reason, there is a much much wider set of applications where they would be actively used.

The term “hallucination” does us all an injustice by propagating the idea of an anthropomorphized LLMs.

Everything an LLM does is a hallucination.

You and me can make out valid patterns from invalid patterns, because we have an idea of some reality.

(Incidentally there are some very weird implications/ perspectives deriving from these 2 positions. Eg - If you had infinite data, would a LLM ever need to calculate?)

Point being - the more intimate the use with an LLM, the more its emergent properties are non-emergent.

Re: A Multi-Level View of LLM Intentionality

#60

Post by prox »

naasking wrote: post #55
lukev wrote: post #9

I'm not sure the definition of "intention" the article suggests is a useful one. He tries to make it sound like he's being conservative: > That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it really has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case). And…

> The model itself is stateless, and it doesn't seem right to me to ascribe "intent" to a stateless function. Statefulness can be modelled statelessly so "statelessness" is not a sufficient reason to dismiss intentionality. The only question is whether the relationship between the inputs of the function and its outputs correspond to what we might call "intent", which the cited definition attempts to outline. Obviousl…

Their identity often falls apart after a few rounds and the self reference seems to my experience at least simply a veneer from linguistic training. It’s a great illusion but an illusion nonetheless.
Post reply on HN