I : hhmmppp a paper from Tenenbaum's group, let's read.
Paper : Hi! I am 94 pages long.
I : omg...
From word models to world models
71–80 of 119 posts
Re: From word models to world models
#72Earlier quoted context omitted.
A facsimile of sufficient equivalence to the world models we derive from our 5 senses may be approached through derivation of descriptive language only. "sufficient equivalence" is important because sure it may not _really_ know the color of red or the qualia of being, but if for all intents and purposes the LLM's internal model provides predictive power and answers correctly as if it does have a world model, then wh…
That's not how physics works. We understand the world by interacting with it. How do you know your internal model is right until it is tested in reality?
Seems you're unaware the amount of world knowledge that already exist in written form.
Think of all the top journals, textbooks, etc. People have understood the world by interacting with it, detailed their hypothesis, conducted experiments, recalled their learning and written down conclusions.
It's not at all obvious to say a useful world model cannot be derived strictly from all this written information.
Re: From word models to world models
#73It's a surprise to see a paper actually try to solve the problem of modelling thought via language. Nevertheless, it begins with far too many hedges: > By scaling to even larger datasets and neural networks, LLMs appeared to learn not only the structure of language, but capacities for some kinds of thinking There's two hypotheses for how LLMs generate apparently "thought-expressing" outputs: Hyp1 -- it's sampling fro…
>It is absolutely trivial to show Hyp2 is false No it's not > Current LLMs can produce impressive results on a set of linguistic inputs and then fail completely on others that make trivial alterations to the same underlying domain. >Indeed: because there're no relevant prior cases to sample from in that case. That's not what that tells us. Humans have weird failure modes that look absurd outside the context of evolut…
was with you until:
> look absurd outside the context of evolutionary biology
for humans, everything (everthing) is within the context of evolutionary biology!
> LLMs have failure modes that look like human failure modes too.
Yes - because LLM's are trained on 2020 Reddit.
Re: From word models to world models
#74It's a surprise to see a paper actually try to solve the problem of modelling thought via language. Nevertheless, it begins with far too many hedges: > By scaling to even larger datasets and neural networks, LLMs appeared to learn not only the structure of language, but capacities for some kinds of thinking There's two hypotheses for how LLMs generate apparently "thought-expressing" outputs: Hyp1 -- it's sampling fro…
I don't think you really disproved anything. You're just saying another hypothesis. Often, LLMs produce impressive results on domains that aren't in the training set.
>LLMs produce impressive results on domains that aren't in the training set.
How do we know? Who knows what they're trained on?
Re: From word models to world models
#75Earlier quoted context omitted.
>It is absolutely trivial to show Hyp2 is false No it's not > Current LLMs can produce impressive results on a set of linguistic inputs and then fail completely on others that make trivial alterations to the same underlying domain. >Indeed: because there're no relevant prior cases to sample from in that case. That's not what that tells us. Humans have weird failure modes that look absurd outside the context of evolut…
was with you until: > look absurd outside the context of evolutionary biology for humans, everything (everthing) is within the context of evolutionary biology! > LLMs have failure modes that look like human failure modes too. Yes - because LLM's are trained on 2020 Reddit.
>for humans, everything (everthing) is within the context of evolutionary biology!
Sure but if some alien species were observing us, some of our actions would look downright odd. Evolutionary biology doesn't necessarily hold the same reference frame for other species, even on earth. Octopi are weird to us. Not so much to other Octopi.
>Yes - because LLM's are trained on 2020 Reddit.
I wasn't making any comment on why this was the case. Simply that it was. There'll be failure models LLMs adopt from training data, but there's also bound to be failure modes LLMs adopt from the training scheme itself.
Re: From word models to world models
#76Earlier quoted context omitted.
A facsimile of sufficient equivalence to the world models we derive from our 5 senses may be approached through derivation of descriptive language only. "sufficient equivalence" is important because sure it may not _really_ know the color of red or the qualia of being, but if for all intents and purposes the LLM's internal model provides predictive power and answers correctly as if it does have a world model, then wh…
That's not how physics works. We understand the world by interacting with it. How do you know your internal model is right until it is tested in reality?
Yeah but we can serialize the world to numbers and already have.
I asked GPT3.5turbo "Pretend you are a character called Samatha and you're in your house. You go up to the thermostat and select a comfortable temperature and explained your reasoning"
> Next, I take into account my personal preferences and comfort levels. Everyone has their own ideal temperature range, and it's essential to find the sweet spot that makes me feel most comfortable. For me, it's usually between 22 to 24 degrees Celsius (72 to 75 degrees Fahrenheit). This range allows me to feel neither too cold nor too warm, striking the perfect balance.
It also goes on about how the humidity could effect the desired temperature, etc.
It doesn't need the ability to feel temperature (which could also be a single floating number using kelvin), but it can already describe a "comfortable temperature" and what factors would effect it.
Side note: It doesn't "know" anything, it can only make a "best guess" which is now fairly reliable enough to be useful. It doesn't need the ability to test things to learn, we did it already for it, and it's using that to predict the results. You could make a recursive system to allow it to test data if you'd like though.
Re: From word models to world models
#77Earlier quoted context omitted.
I think this is easy, just make Xp sentences of the kind = "I define `randomchars()` to be this `term-in-Xc()`" and swamp the dataset with Xc. Everything here actually just follows formally from what NNs are: they're just empirical function approximations. It will always be the case that they just model the probabilistic structure of the dataset and not the data generating process. Since, in language, there are discr…
> I think this is easy, just make Xp sentences of the kind = "I define `randomchars()` to be this `term-in-Xc()`" and `randomchars()` does not match your own requirement `but not that the tokens of Xp are themselves rare` and therefore is unsuitable.
good point --- so replace it with a `sample()` fn that selects from an appropriate distribution over the data
Re: From word models to world models
#78I doubt that word models can lead to world models. To quote Yann LeCun: "The vast majority of our knowledge, skills, and thoughts are not verbalizable. That's one reason machines will never acquire common sense solely by reading text." https://twitter.com/ylecun/status/1368235803147649028
What concepts exactly can’t be verbalized? All of our serialized file formats fall under the umbrella of “words”. GPT4 can draw images by outputting SVGs for example.
Re: From word models to world models
#79I doubt that word models can lead to world models. To quote Yann LeCun: "The vast majority of our knowledge, skills, and thoughts are not verbalizable. That's one reason machines will never acquire common sense solely by reading text." https://twitter.com/ylecun/status/1368235803147649028
That just seems like an unfounded hot take. Of course we can explain most of our knowledge, skills, and thoughts in words, that's how we don't lose everything when the next generation comes around lol. It's the core reason we're different from animals. Now sure you can't describe qualia, but that's basically a subjective artefact of how we sense the world and (to add another unfounded hot take) likely not critical to…
> Of course we can explain most of our knowledge, skills, and thoughts in words,
This is either some profound miscomprehension of just how many of your skills and thoughts are inexpressible in words, or some statement of how profoundly shallow your skills and thoughts actually are.
Re: From word models to world models
#80Earlier quoted context omitted.
That just seems like an unfounded hot take. Of course we can explain most of our knowledge, skills, and thoughts in words, that's how we don't lose everything when the next generation comes around lol. It's the core reason we're different from animals. Now sure you can't describe qualia, but that's basically a subjective artefact of how we sense the world and (to add another unfounded hot take) likely not critical to…
I disagree that this is an "unfounded hot take". It's far from a rare opinion on cognitive science, and if I had to guess it's probably the mainstream opinion (I can't really back that up with citations because I haven't followed the field closely in the last decade). And for what it's worth, I agree with Yann, although I have to admit that LLMs work far better than I would've guessed. It's a topic that's too large f…
Well I admit I used to be of a similar opinion as well, but seeing this explosion unravel over the past few months has me convinced that it's can't possibly be right, at least not to any degree that objectively matters.
Perhaps language is the wrong term to use, since it's not what LLMs are really about. They're about text. There are very few things that cannot be expressed as text, albeit in unconventional ways like base64. Being opaque to humans doesn't mean that with enough data a neural net can't be taught to "see" images that way or "hear" sound files for example. If the original assertion is true, then there must be some kind of universal barrier to skills that cannot be expressed in text. That sounds completely crazy to me, since we humans are also likely just organic data that could be expressed as text with some encoding. The main problem is interfacing with it in some way that's actually useful, which is the extremely hard part.
Another thing to consider is that with a formalized enough language (i.e. a programming language) one can be far more exact in explaining things accurately than any natural language with its cultural specifics and inferred nonsense. That's probably why LLMs designed as coding models first and foremost usually outperform those that aren't in solving unrelated arbitrary problems.
> Note that humans are animals too, btw. And conversely, I would consider nonverbal people as humans as well.
Humans are animals in the biological sense, yes. But very much not in the societal and skill-transferring sense.