The short film version of the original is great, too. https://www.youtube.com/watch?v=T6JFTmQCFHg
It stars Tom Noonan and Ben Bailey!
They’re made out of weights
31–40 of 739 posts
Re: They’re made out of weights
#32Earlier quoted context omitted.
A tokenizer is not a dictionary any more than an alphabet is a dictionary.
The Chinese alphabet is very much a dictionary. All the major tokenizers are far larger.
A list of words isn’t a dictionary. What a dictionary adds over a list of words is all the relationships between the words needed to interpret them and use them, and all of that is in the weights.
Re: They’re made out of weights
#33Earlier quoted context omitted.
Even there it works a bit. > These models are the only other things we've ever met that can hold a conversation, and they're made out of weights Is a fair point.
Not especially. Depending on where you set your standards for "holding a conversation" you can satisfy the requirement with a classical markov chatterbot, a well-trained parrot, a copy of Eliza, or a telemarketer flowchart drawn on a sheet of paper. Only the markov bot is made out of "weights" in the sense of a statistical model. Parrots are intelligent animals, albeit with a limited capacity for vocabulary and synta…
Weights hold a better conversation at this point than the overwhelming majority of humans.
Re: They’re made out of weights
#34Earlier quoted context omitted.
I might have misunderstood the point you are making. I read the original article as "weights are like meat", and so I'm confused by what you consider fractally wrong.
The point that when the rules the model learns are simple enough they stop being spread out over all the layers and become as easily interpretable as any expert system. It's just that the rules we feed in the model are extremely poorly defined and we end up with the soup of disjoint rules smeared all across the weights. This isn't a feature of the models. It's a feature of the training set. Being shocked that you can…
Right, but all of that is still in the weights. The point of the article/joke isn’t literally that there is no grammar, it’s that there is no grammar separate from the weights. It’s all in the weights. And yes, it’s absurd. It’s a joke, but a thought provoking one.
Re: They’re made out of weights
#35It must have been kind of incredible early on to be exploring this tech and you’re suddenly getting what look like sentences.
Re: They’re made out of weights
#36It's not often I see something that's fractally wrong but here we are. There is a dictionary, it's called the tokenizer. There are grammar rules, they are just very weak because the structure of human language is generally quite weak. When presented with languages which have strong consistent grammars the weights are very easily interpretable as a grammar: https://arxiv.org/abs/2201.02177 The point of the original sh…
>There are grammar rules, they are just very weak because the structure of human language is generally quite weak. When presented with languages which have strong consistent grammars the weights are very easily interpretable as a grammar: https://arxiv.org/abs/2201.02177
That paper did not train the models on 'a language with strong consistent grammars'. Mathematical Operation tables are not a language. Grammar itself is a post-hoc rationalization and there's no evidence LLMs follow 'grammar rules' anymore than the brain follows grammar rules. Of Course, that's not to say transformers can't learn simple rules if the dataset calls for it.
Re: They’re made out of weights
#37Earlier quoted context omitted.
Yeah, I currently suspect that consciousness is an emergent property. I read elsewhere (it's somewhere in my HN history, I'm sure) that the biggest compute we can currently muster is something like three or four magnitudes away from the number of neurons / connections (or their analog) that our brains have, so it may be a while until we can expect to see it in our machines. But, if the emergent phenomenon hypothesis…
This is not meant as a gotcha, I am genuinely curious how you believe consciousness can be an emergent property. I assume you don't believe consciousness is a physical property in the brain, so what entity is actually experiencing that consciousness? Or, what does it even mean to experience consciousness? Or are these not even the right questions?
Those are the questions and there’s stacks and stacks of philosophy pages written about it. Go have a whirl.
Re: They’re made out of weights
#38This read like poetry to me. Thank you for sharing it. I have a linguistics background and a lot of my philosophizing lately has been on whether or not the emergent abilities of the LLMs is deep down a similar mechanism that creates our consciousness. For a little bit I was working on having linguistics based evals for a kaggle competition. My challenge was whether or not I could mask things well enough to not trigge…
I’ve wondered the same myself, without being a cunning linguist. I understand the math pretty well but still find it crazy that a bunch of matrices can converse in human languages without ever being “taught”. Imagine decoding an encyclopedia written in a foreign language where the characters, punctuation, and grammar are unknown — supplemented by a million other texts the same way. Feels like it should be utterly imp…
If time dilation is said to being a product of computation, why is it that anaesthetic drugs that are taken not to the point of actual unconsciousness cause it. Dont anaesthetics sort of shut everything down/inhibit all that kind of cognitive activity (compute?)
Re: They’re made out of weights
#39I have to agree. It is messed up that transformers can just talk, and it been pretty normalized. We are only talking about the impact they will have and whether they can do what people say they can, but we arent talking about how crazy it is that they can talk
Re: They’re made out of weights
#40Earlier quoted context omitted.
Yeah, I currently suspect that consciousness is an emergent property. I read elsewhere (it's somewhere in my HN history, I'm sure) that the biggest compute we can currently muster is something like three or four magnitudes away from the number of neurons / connections (or their analog) that our brains have, so it may be a while until we can expect to see it in our machines. But, if the emergent phenomenon hypothesis…
This is not meant as a gotcha, I am genuinely curious how you believe consciousness can be an emergent property. I assume you don't believe consciousness is a physical property in the brain, so what entity is actually experiencing that consciousness? Or, what does it even mean to experience consciousness? Or are these not even the right questions?
Is a video game a physical property of a computer?