201–210 of 432 posts
Earlier quoted context omitted.
Important attention heads or layers within an LLM can be repeated giving you an "unrolled" recursion.
An unrolled loop in a feed-forward network is all just that. The computation is DAG.
So the author’s core view is ultimately a Searle-like view: a computational, functional, syntactic rules based system cannot reproduce a mind. Plenty of people will agree, plenty of people will disagree, and the answer is probably unknowable and just comes down to whatever axioms you subscribe to in re: consciousness. The author largely takes the view that it is more productive for us to ignore any anthropomorphic re…
What you identify as emergent and mind-like is a direct result of these tools being able to mimic human communication patterns unlike anything we've ever seen before. This capability is very impressive and has a wide range of practical applications that can improve our lives, and also cause great harm if we're not careful, but any semblance of intelligence is an illusion. An illusion that many people in this industry obsessively wish to propagate, because thar be gold in them hills.
The author's critique of naive anthropomorphism is salient. However, the reduction to "just MatMul" falls into the same trap it seeks to avoid: it mistakes the implementation for the function. A brain is also "just proteins and currents," but this description offers no explanatory power. The correct level of analysis is not the substrate (silicon vs. wetware) but the computational principles being executed. A modern…
> A modern sparse Transformer, for instance, is not "conscious," but it is an excellent engineering approximation of two core brain functions: the Global Workspace (via self-attention) and Dynamic Sparsity (via MoE). Could you suggest some literature supporting this claim? Went through your blog post but couldn't find any.
https://www.frontiersin.org/journals/computational-neuroscie...
Earlier quoted context omitted.
Agreeing with you, this is a "can a submarine swim" problem IMO. We need a new word for what LLMs are doing. Calling it "thinking" is stretching the word to breaking point, but "selecting the next word based on a complex statistical model" doesn't begin to capture what they're capable of. Maybe it's cog-nition (emphasis on the cog).
What does a submarine do? Submarine? I suppose you "drive" a submarine which is getting to the idea: submarines don't swim because ultimately they are "driven"? I guess the issue is we don't make up a new word for what submarines do, we just don't use human words. I think the above poster gets a little distracted by suggesting the models are creative which itself is disputed. Perhaps a better term, like above, would…
This process goes on for a trillion trillion tokens, with the alien growing better through the process until it can do it better than a human could.
At that point we flash freeze it, and use a copy of it, without giving it any way to learn anything new.
--
I see it as a category error to anthropomorphize it. The closest I would get is to think of it as an alien slave that's been lobotomized.
Earlier quoted context omitted.
But the author used "anthropomorphism" the same way as I did. I guess we both mean "personification" then. > we talk about "behaviors", "ethical constraints", and "harmful actions in pursuit of their goals". All of these are anthropocentric concepts that - in my mind - do not apply to functions or other mathematical objects. One talking about a program's "behaviors", "actions" or "goals" doesn't mean they believe the…
> One talking about a program's "behaviors", "actions" or "goals" doesn't mean they believe the program is sentient. Except that is exactly what we’re seeing with LLMs. People believing exactly that.
A bit of anecdote: last year I hung out with a bunch of old classmates that I hadn't seen for quite a while. None of them works in tech.
Surprisingly to me, all of them have ChatGPT installed on their phones.
And unsurprisingly to me, none of them treated it like an actual intelligence. That makes me wonder where those who think ChatGPT is sentient come from.
(It's a bit worrisome that several of them thought it worked "like Google search and Google translation combined", even by the time ChatGPT couldn't do web search...!)
The problem with viewing LLMs as just sequence generators, and malbehaviour as bad sequences, is that it simplifies too much. LLMs have hidden state not necessarily directly reflected in the tokens being produced and it is possible for LLMs to output tokens in opposition to this hidden state to achieve longer term outcomes (or predictions, if you prefer). Is it too anthropomorphic to say that this is a lie? To say th…
I think that the hidden state is really just at work improving the model's estimation of the joint probability over tokens. And the assumption here, which failed miserably in the early 20th century in the work of the logical posivitists, is that if you can so expertly estimate that joint probability of language, then you will be able to understand "knowledge." But there's no well grounded reason to believe that and p…
Earlier quoted context omitted.
Sure it's state. It logically evolves stepwise per token generation. It encapsulates the LLM's understanding of the text so far so it can predict the next token. That it is merely a fixed function of other data isn't interesting or useful to say. All deterministic programs are fixed functions of program code, inputs and computation steps, but we don't say that they don't have state. It's not a useful distinction for…
I'll say it once more: I think it is useful to distinguish between autoregressive and recurrent architectures. A clear way to make that distinction is to agree that the recurrent architecture has hidden state, while the autoregressive one does not. A recurrent model has some point in a space that "encapsulates its understanding". This space is "hidden" in the sense that it doesn't correspond to text tokens or any oth…
"Transformers do not have hidden state" is, as we can clearly see from this thread, far more misleading than the opposite.