Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language Models Represent Space and Time

arxiv.org

Post reply on HN
Print view View original

31–40 of 192 posts

Re: Language Models Represent Space and Time

#31

Post by xpe »

ilaksh wrote: post #30

One thing that is clouding this discussion is that most people are mixing up a lot of different characteristics that animals like humans have as if they were all the same thing. So for many people they don't really distinguish between things like "reasoning", "self-aware", "conscious", "alive", "sentient", "intelligent", "has world model". They also don't distinguish between different types or varying levels of cogni…

Well said.

Also, the paper’s use of “world model” (instead of simply “model”) doesn’t help clarify the situation.

Re: Language Models Represent Space and Time

#32

Post by teaearlgraycold »

There was an interesting thought from a Microsoft researcher in an episode of This American Life. He had been given early access to GPT4 and found that it had gained an understanding of gravity and balancing. 3.5 would fail to describe a safe order for stacking 3 eggs, a bottle, a book and a nail. But 4 would give robust answers with logical justification added to each construction step for the tower.

The researcher remarked (paraphrasing) “The models are built to predict the next token. At some point the only way to improve token prediction is to start modeling the world and building an understanding of how things work.” Since I heard this I see it everywhere with LLMs. Load up a 30B parameter model on your computer and toy around with it. When it fails I now see that it’s reached a concept it doesn’t understand and is in more-or-less Markov Chain mode. Give the same task to GPT4 and it proves it’s modeled the concept appropriately because it can pull the right words out of the ether.

Re: Language Models Represent Space and Time

#33

Post by gamblor956 »

corethree wrote: post #4

A lot of HNers were so adamant that the LLMs understand absolutely nothing and that these models are just predicting the next most likely word. I think with this paper it becomes clear that the adamant denial was just human bias talking. LLMs crossed a certain line here. You would think people would be amazed or in awe or react in fear at technological break through a and they often are. The weird part for LLMs was t…

If you actually read the paper, the analysis absolutely does not support the claim that LLMs are capable of maintaining a spatial or chronological internal world model.

The LLMs simply activated when locations or times were mentioned, with no understanding of the relationship between different places or different times beyond the fact that different places and times had different tags. Or in other words, no internal model of the world.

Re: Language Models Represent Space and Time

#35

Post by famouswaffles »

YeGoblynQueenne wrote: post #19
andrewguenther wrote: post #8

Earlier quoted context omitted.

I think LLMs are presenting some uncomfortable philosophical questions for people about how our own brains work and admitting that there is any kind of "intelligence" (even if very basic) in an LLM is an admission that our own brains may work in a similar manner.

For me, one of the most interesting things that have come out of LLMs is the confirmation that humans are very bad at reasoning and, consequently, its' a very bad idea to try and make machines that "think like humans", because that way we'll only make machines with none of the advantages of machines and all the disadvantages of computers. For instance -I'm not trying to be mean and I'm certainly not blaming you in pa…

>because that way we'll only make machines with none of the advantages of machines and all the disadvantages of computers.

There is no evidence, basically none whatsoever that general "perfect logical reasoning" is a thing that actually exists in the real world. None.

No animal we've observed does it. Humans certainly don't do it. The only realm this idea actually works is Fiction. and this was not like for a lack of trying. Some of the greatest minds worked on this for decades and some people still don't seem to get it. Logic doesn't scale. They break at real world relationships.

Logic systems are that guy in the stands yelling that he could've made the shot, while he's not even on the field.

Re: Language Models Represent Space and Time

#36

Post by nyrikki »

klysm wrote: post #13

I’m most surprised that the embedding is actually linear. I wonder if that makes extrapolation perform better

Feed forward networks are pairwise linear in respect to inputs, they are effectively DAGs.

In theory you could represent an entire LLM as a single 2 dimensional graph of linear line segments.

It wouldn't be useful for much as the parameters are clustered in dense patches.

Re: Language Models Represent Space and Time

#37

Post by sebzim4500 »

xpe wrote: post #31
ilaksh wrote: post #30

One thing that is clouding this discussion is that most people are mixing up a lot of different characteristics that animals like humans have as if they were all the same thing. So for many people they don't really distinguish between things like "reasoning", "self-aware", "conscious", "alive", "sentient", "intelligent", "has world model". They also don't distinguish between different types or varying levels of cogni…

Well said. Also, the paper’s use of “world model” (instead of simply “model”) doesn’t help clarify the situation.

I do not see how referring to a world model as simply a "model" could possibly provide clarity.

LLM literally stands for Large Language Model, it has a 'model' in it by definition.

Re: Language Models Represent Space and Time

#38

Post by pyth0 »

fipar wrote: post #17
corethree wrote: post #4

A lot of HNers were so adamant that the LLMs understand absolutely nothing and that these models are just predicting the next most likely word. I think with this paper it becomes clear that the adamant denial was just human bias talking. LLMs crossed a certain line here. You would think people would be amazed or in awe or react in fear at technological break through a and they often are. The weird part for LLMs was t…

My only experience so far is with chatgpt, and I am not an expert in AI, or even just in LLMs. With those disclaimers, my interactions inform my opinion that the LLM behind chatgpt has no internal world model. It shows no understanding of basic facts and makes very silly mistakes very easily. I have my bias, like anyone else, but in the case of AI in particular, I should say that I don't think there's anything especi…

People are already asking you if you have tried but this pretty much echos my experience and I have used both free and paid versions of ChatGPT (3.5 and 4 respectively) as well as the GPT4 API directly. I had a similar experience between all three of them, although the GPT4 based versions were certainly better in terms of output quality. In my opinion it seems pretty clear that this is a fundamental limit of the architecture and all the RLHF and increase in model sizes wont change that.

Re: Language Models Represent Space and Time

#39

Post by fipar »

ilaksh wrote: post #20
fipar wrote: post #17

Earlier quoted context omitted.

My only experience so far is with chatgpt, and I am not an expert in AI, or even just in LLMs. With those disclaimers, my interactions inform my opinion that the LLM behind chatgpt has no internal world model. It shows no understanding of basic facts and makes very silly mistakes very easily. I have my bias, like anyone else, but in the case of AI in particular, I should say that I don't think there's anything especi…

Have you used GPT-4?

No, I've only used GPT-3 so far. Should I be excited about GPT-4? :)

Examples of things I'd file as silly mistakes:

- mentioning non-existing settings when asking it about a particular software.

- responding to roughly "give me a way to list all rds clusters that are running a version that will reach EOL within the next 12 months" (which I know is almost impossible to do without scraping, as the EOLs are not returned by the API, but it was worth trying) with a query that compares the version with - providing wrong steps in an answer to "give me steps to do X", wrong in a way that would cause the process to go very bad, and then when I point out the problematic step, it responds with a seemingly random alternate step that is also wrong. I guess this part is the one that is less bad, because it loosely reminds me of people who are bad at reasoning and just give back random answers to a logic or math question.

I must say it is surprisingly good (though it will still make up facts at times) when answering questions about elisp though.

Paraphrasing Hofstadter, I'd say if I'm asked about the intelligence of chatgpt using gpt-3 I'd say it's slightly better than a thermostat or a mosquito, but not much more. Furthermore, my (admittedly not very valuable to others) gut feeling is this is not the path to get to what I'd consider an intelligent being, mostly because in my interactions, it makes lots of bad mistakes, and none of the good ones (here I'm using good and bad as qualifiers for mistakes based on my experiences as a parent, and as a mentor to junior peers, where a good mistake is something that lets you know they're on the right path even if they haven't arrived at the right answer/place yet). Try asking it about analogies, for example ("what's a good analogy to explain X to someone new to it"), the results I've gotten are underwhelming, when not just plain wrong.

Edit: s/fall/file/ Formatting.

Re: Language Models Represent Space and Time

#40

Post by uoaei »

semitones wrote: post #18
canjobear wrote: post #11

Much simpler models like word2vec also showed linear embeddings of geography in this way.

source?

It was something like

    Paris - France + Japan = Tokyo
I know that's not a source but you could also just google "word2vec", as I recall most of the explainer blog posts had similar examples.
Post reply on HN