Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language Models Represent Space and Time

arxiv.org

Post reply on HN
Print view View original

161–170 of 192 posts

Re: Language Models Represent Space and Time

#161

Post by YeGoblynQueenne »

xcv123 wrote: post #151
YeGoblynQueenne wrote: post #132

Earlier quoted context omitted.

Well, this has been claimed often, namely by some of the people who have developed statistical language models [1] but it's really not obvious how that should work. Where would a language model find the world model? Where would it store it? And why would it even need it? Obviously I don't know how the human linguistic ability works but it's clear that for us, text, words, language, isn't carrying around with it a rep…

> Why would it ever be possible to derive the representation just from the pointer? Of course it cannot derive a representation from just a pointer. Neither humans or machines can do that. The words are not pointers in isolation. They are connected in a network of semantic relations. The model is in the relationships between pointers. Artificial neural network.

The question then becomes where do the "semantic" relations in the network come from. Unless there is some way to extract the meaning of words with respect to the real-world concepts they refer to, from text, any relation between words that your model, models is not going to be semantic (i.e. it won't represent meaning).

Like I say in another comment, this kind of explanation is "words all the way down", but it really doesn't explain where the process ends (or begins) and where the meaning comes into the words.

To give you an example, here is a small semantic network that I just created by hand, that represents is-a and has-a relations:

  abc32 is-a c356 
  7yt4 is-a c356
  c356 has-a 902a
  c356 has-a 8773
The names of entities in the network, like abc32, replace real English words that I just changed manually (or, well, vim-illy). You can use the is-a and has-a relations in the network to infer e.g. that abc32 has-a 902a, and that 7yt4 and abc32 are things of the same kind, but go ahead and try to understand what the real-world entities I renamed were. You can't- and neither can any model that is trained on text consisting of words whose meaning it understands as much as you understand the meaning of abc32 and 7yt4.

Wikipedia has an article on semantic networks:

https://en.wikipedia.org/wiki/Semantic_network

I haven't read it but from a quick glance it looks informative.

Re: Language Models Represent Space and Time

#162

Post by xcv123 »

YeGoblynQueenne wrote: post #160
xcv123 wrote: post #143

Earlier quoted context omitted.

> Where would a language model find the world model? Where would it store it? And why would it even need it? It is pattern recognition with many layers of abstraction. Obviously it will infer semantic relations at some level. It is a type of machine learning. The entire point of machine learning is to generate a model of the data which can be generalized to new inputs. It would find the world model in semantic relati…

>> GPT-4 took months to train on a supercomputer and it generated a neural network of hundreds of gigabytes. What exactly was that supercomputer doing for several months and what exactly would the neural network represent if not a world model? I believe GPT-4 was trained on a server farm, not a single computer. In any case what it was doing all that time was going over and over the text in its gigantic training corpu…

> I believe GPT-4 was trained on a server farm, not a single computer.

Yes a server farm of Nvidia A100 with supercomputing performance.

> It is easy to see how this stuff works. .. you can train a Hidden Markov Model..

No. GPT is not a Markov model.

Re: Language Models Represent Space and Time

#163

Post by xcv123 »

YeGoblynQueenne wrote: post #161
xcv123 wrote: post #151

Earlier quoted context omitted.

> Why would it ever be possible to derive the representation just from the pointer? Of course it cannot derive a representation from just a pointer. Neither humans or machines can do that. The words are not pointers in isolation. They are connected in a network of semantic relations. The model is in the relationships between pointers. Artificial neural network.

The question then becomes where do the "semantic" relations in the network come from. Unless there is some way to extract the meaning of words with respect to the real-world concepts they refer to, from text, any relation between words that your model, models is not going to be semantic (i.e. it won't represent meaning). Like I say in another comment, this kind of explanation is "words all the way down", but it reall…

> but go ahead and try to understand what the real-world entities I renamed were.

Obviously not. Insufficient data. Try that substitution exercise again with terabytes of data (Common Crawl dataset) then feed it into a neural network and see if it can find any patterns.

Re: Language Models Represent Space and Time

#164

Post by Sharlin »

esafak wrote: post #130
Sharlin wrote: post #28

Earlier quoted context omitted.

It's the Chinese Room argument all over again. People hear "predicting the next token" and all they can imagine is some sort of a statistical database lookup, ignoring the fact that when you have a huge corpus of data with incredibly complex internal correlations and all that data also happens to correlate with some unknown external thing, it's almost certain that a powerful learner will end up modeling that external…

That is what a model is; a parsimonious explanation of observations that admits extrapolation.

Exactly!

Re: Language Models Represent Space and Time

#165

Post by famouswaffles »

joshuahedlund wrote: post #23
famouswaffles wrote: post #2

The Twitter thread is worth looking at. Very Fascinating https://twitter.com/wesg52/status/1709551516577902782?t=3b2F...

I don't understand what the colors represent and I can't read the thread. Can you summarize?

You should be able to read the thread here https://nitter.net/wesg52/status/1709551516577902782

The colors in the first tweet just represent different territories(continents)

Re: Language Models Represent Space and Time

#166

Post by mjburgess »

Timon3 wrote: post #158
mjburgess wrote: post #147

Earlier quoted context omitted.

The claim is that you cannot determine whether a system has a 'world model' through inspection of it's abstract token inputs/outputs.

But that's not all this paper is doing. So why bring it up?

My comment is a reply to another commenter, not to the paper

Re: Language Models Represent Space and Time

#167

Post by corethree »

fipar wrote: post #39
ilaksh wrote: post #20

Earlier quoted context omitted.

Have you used GPT-4?

No, I've only used GPT-3 so far. Should I be excited about GPT-4? :) Examples of things I'd file as silly mistakes: - mentioning non-existing settings when asking it about a particular software. - responding to roughly "give me a way to list all rds clusters that are running a version that will reach EOL within the next 12 months" (which I know is almost impossible to do without scraping, as the EOLs are not returned…

Paraphrasing hofsadter?

You realize Hofstadter is not delusional about LLMs at all. His view point is completely opposite of yours more logical and rational and he doesn't need to use chatGPT 4.

Hofstadter is not a normal person because he is extremely unbiased. chatGPT basically got him to do a 180 on everything he thought and talked about in every book he has written. He flipped, you subscribe to his viewpoints, and you haven't flipped.

Keep in mind he criticized early versions of gpt3 and gpt2. chatGPT changed the game.

https://www.nytimes.com/2023/07/13/opinion/ai-chatgpt-consci...

https://www.lesswrong.com/posts/kAmgdEjq2eYQkB5PP/douglas-ho...

He literally said something along the lines of his core beliefs are collapsing. Paraphrasing. I kid you not. It's abnormal. It's very rare to find a person that can change his core beliefs. Extremely rare. Let's be frank. You are not that person. What will happen is you will reinterpret reality such that it fits your core beliefs and you won't know you're doing this. It's very predictable: Using chatGPT 4 will not revise your opinion on AI because you don't want to face the truth. chatGPT 3.5 is already enough to see how we crossed a certain line here you can't admit it... so your experience will be the same with gpt4.

Re: Language Models Represent Space and Time

#168

Post by corethree »

corethree wrote: post #167
fipar wrote: post #39

Earlier quoted context omitted.

No, I've only used GPT-3 so far. Should I be excited about GPT-4? :) Examples of things I'd file as silly mistakes: - mentioning non-existing settings when asking it about a particular software. - responding to roughly "give me a way to list all rds clusters that are running a version that will reach EOL within the next 12 months" (which I know is almost impossible to do without scraping, as the EOLs are not returned…

Paraphrasing hofsadter? You realize Hofstadter is not delusional about LLMs at all. His view point is completely opposite of yours more logical and rational and he doesn't need to use chatGPT 4. Hofstadter is not a normal person because he is extremely unbiased. chatGPT basically got him to do a 180 on everything he thought and talked about in every book he has written. He flipped, you subscribe to his viewpoints, an…

Here's a quote from one of those links:

"Of course, it reinforces the idea that human creativity and so forth come from the brain's hardware. There is nothing else than the brain's hardware, which is neural nets. But one thing that has completely surprised me is that these LLMs and other systems like them are all feed-forward. It's like the firing of the neurons is going only in one direction. And I would never have thought that deep thinking could come out of a network that only goes in one direction, out of firing neurons in only one direction. And that doesn't make sense to me, but that just shows that I'm naive.

It also makes me feel that maybe the human mind is not so mysterious and complex and impenetrably complex as I imagined it was when I was writing Gödel, Escher, Bach and writing I Am a Strange Loop. I felt at those times, quite a number of years ago, that as I say, we were very far away from reaching anything computational that could possibly rival us. It was getting more fluid, but I didn't think it was going to happen, you know, within a very short time."

The guy literally flipped bro. He literally just admitted what he wrote in his books are wrong. The books may not be wrong but he believes it now. Degrading hundreds of pages of his own writing and his own core beliefs is a display of incredible rationality, scientific thinking and lack of bias.

Re: Language Models Represent Space and Time

#169

Post by xpe »

karelpeeters wrote: post #85
xpe wrote: post #81

Perhaps the most important public discussion, at least among technical people, is making sure we really understand the experimental setup. (I admit I could be projecting my lack of understanding; I’m not rock-solid on the experimental setup yet.) Still to me, it matters if the linearity been described can be observed directly _in_ the model. My understanding is that it is not; rather, it is tested with an additional…

It's important to note that these probe models are very simple - they're "trained" by just doing a linear regression between the hidden activations and the desired output. This means that the probes can barely do any computation themselves, so if they work at all this is a strong indication that the signal they predict was already in the hidden activations. For even more proof, see "Figure 5: Space and time neurons i…

Thanks. After some additional research, I tend to agree.

One thing I learned of note is that while a _positive_ result (i.e. a probe showing a relationship) is proof that a label is encoded somehow; a _negative_ result does _not_ prove that the network under test does _not_ encode the information _somehow_. (The information could be encoded in more complex ways that the probe does not discover).

Re: Language Models Represent Space and Time

#170

Post by circuit10 »

YeGoblynQueenne wrote: post #160
xcv123 wrote: post #143

Earlier quoted context omitted.

> Where would a language model find the world model? Where would it store it? And why would it even need it? It is pattern recognition with many layers of abstraction. Obviously it will infer semantic relations at some level. It is a type of machine learning. The entire point of machine learning is to generate a model of the data which can be generalized to new inputs. It would find the world model in semantic relati…

>> GPT-4 took months to train on a supercomputer and it generated a neural network of hundreds of gigabytes. What exactly was that supercomputer doing for several months and what exactly would the neural network represent if not a world model? I believe GPT-4 was trained on a server farm, not a single computer. In any case what it was doing all that time was going over and over the text in its gigantic training corpu…

> development of a world model, rather than, or in addition to, a model of the training corpus

But the training corpus describes the world, so a model of the training corpus is a world model

Post reply on HN