Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language Models Represent Space and Time

arxiv.org

Post reply on HN
Print view View original

41–50 of 192 posts

Re: Language Models Represent Space and Time

#41

Post by xpe »

teaearlgraycold wrote: post #32

There was an interesting thought from a Microsoft researcher in an episode of This American Life. He had been given early access to GPT4 and found that it had gained an understanding of gravity and balancing. 3.5 would fail to describe a safe order for stacking 3 eggs, a bottle, a book and a nail. But 4 would give robust answers with logical justification added to each construction step for the tower. The researcher…

> Give the same task to GPT4 and it proves it’s modeled the concept appropriately because it can pull the right words out of the ether.

I take your point, but we should be careful when using phrased like “modeled the concept appropriately” here. That implies correctness.

There are many ‘concept models’ that could work well enough. Unless we can inspect the concept model directly (inside the LLM, somehow!) we are left to reason about how the concept is represented via statistics over a set of interactions. So, if, over the course of many interactions, it seems that a model is saying reasonable things we can have more confidence that it has achieved a concept model that is “good enough”. But such an assessment is always in relationship to the test protocol. Claims to generality should be done carefully.

In some ways, my standard here is probably not that different than a rigorous human educational assessment. I’m not talking about standardized tests, I’m talking about adversarial challenges like one would hope to see during a Ph.D. defense.

Re: Language Models Represent Space and Time

#43

Post by School-Cotton »

corethree wrote: post #4

A lot of HNers were so adamant that the LLMs understand absolutely nothing and that these models are just predicting the next most likely word. I think with this paper it becomes clear that the adamant denial was just human bias talking. LLMs crossed a certain line here. You would think people would be amazed or in awe or react in fear at technological break through a and they often are. The weird part for LLMs was t…

> I think with this paper it becomes clear that the adamant denial was just human bias talking

It already becomes clear after playing with ChatGPT-4 for five minutes. But a lot of people still refuse to accept it.

Re: Language Models Represent Space and Time

#45

Post by School-Cotton »

fipar wrote: post #39
ilaksh wrote: post #20

Earlier quoted context omitted.

Have you used GPT-4?

No, I've only used GPT-3 so far. Should I be excited about GPT-4? :) Examples of things I'd file as silly mistakes: - mentioning non-existing settings when asking it about a particular software. - responding to roughly "give me a way to list all rds clusters that are running a version that will reach EOL within the next 12 months" (which I know is almost impossible to do without scraping, as the EOLs are not returned…

> No, I've only used GPT-3 so far. Should I be excited about GPT-4?

The difference is extremely dramatic. Any experience with 3 or 3.5 is completely irrelevant for 4.

Re: Language Models Represent Space and Time

#46

Post by sgt101 »

sebzim4500 wrote: post #37
xpe wrote: post #31

Earlier quoted context omitted.

Well said. Also, the paper’s use of “world model” (instead of simply “model”) doesn’t help clarify the situation.

I do not see how referring to a world model as simply a "model" could possibly provide clarity. LLM literally stands for Large Language Model, it has a 'model' in it by definition.

Would we think of it differently if we called it LLC - large language correlation?

Re: Language Models Represent Space and Time

#47

Post by nyrikki »

gamblor956 wrote: post #33
corethree wrote: post #4

A lot of HNers were so adamant that the LLMs understand absolutely nothing and that these models are just predicting the next most likely word. I think with this paper it becomes clear that the adamant denial was just human bias talking. LLMs crossed a certain line here. You would think people would be amazed or in awe or react in fear at technological break through a and they often are. The weird part for LLMs was t…

If you actually read the paper, the analysis absolutely does not support the claim that LLMs are capable of maintaining a spatial or chronological internal world model. The LLMs simply activated when locations or times were mentioned, with no understanding of the relationship between different places or different times beyond the fact that different places and times had different tags. Or in other words, no internal…

The lack of generally accepted definition for knowledge and intelligence makes these conversations difficult but the above post is important to remember.

The paper has nothing to do with the common sense problem.

While it is a preprint, I do wonder how useful this will be to improve generalization.

Termination analysis on linear polynomials is undecidable over the naturals without many constraints and is undecidable ofer the rationals.

If you think about how sstrees or even rtrees work, this isn't surprising to me that space and time are useful for compression which is required for learnability.

But I think people also see 'linear' and think that means easy. When it can mean serious restrictions on how it can be used.

Hard to explain the connection I have in mind without serious math but here is a paper that may cause some issues with trying to use these internal structures outside of the learnability implications.

https://arxiv.org/abs/2307.14805

Re: Language Models Represent Space and Time

#49

Post by teaearlgraycold »

xpe wrote: post #41
teaearlgraycold wrote: post #32

There was an interesting thought from a Microsoft researcher in an episode of This American Life. He had been given early access to GPT4 and found that it had gained an understanding of gravity and balancing. 3.5 would fail to describe a safe order for stacking 3 eggs, a bottle, a book and a nail. But 4 would give robust answers with logical justification added to each construction step for the tower. The researcher…

> Give the same task to GPT4 and it proves it’s modeled the concept appropriately because it can pull the right words out of the ether. I take your point, but we should be careful when using phrased like “modeled the concept appropriately ” here. That implies correctness. There are many ‘concept models’ that could work well enough. Unless we can inspect the concept model directly (inside the LLM, somehow!) we are lef…

Yeah I’m setting the bar at the level of a child’s understanding. Because smaller models are well below that, just picking tokens at random. That’s how misunderstanding manifests with LLMs. They don’t say “I don’t know”, they just go off the rails.

The quality bar for different use cases might be such that an understanding of a system on par with a 6 year old is isn’t useful. But right now we clearly have two clusters and I’m drawing the boundary where it’s most useful.

Re: Language Models Represent Space and Time

#50

Post by nopinsight »

fipar wrote: post #39
ilaksh wrote: post #20

Earlier quoted context omitted.

Have you used GPT-4?

No, I've only used GPT-3 so far. Should I be excited about GPT-4? :) Examples of things I'd file as silly mistakes: - mentioning non-existing settings when asking it about a particular software. - responding to roughly "give me a way to list all rds clusters that are running a version that will reach EOL within the next 12 months" (which I know is almost impossible to do without scraping, as the EOLs are not returned…

GPT-4 should do significantly better, but still worse than expert humans, on the tasks you mentioned. They would also benefit from good prompting, such as those in

- https://lilianweng.github.io/posts/2023-03-15-prompt-enginee...

- https://help.openai.com/en/articles/6654000-best-practices-f...

Not a large percentage of humans would be able to do the tasks listed above without significant experience or training either. Average humans are also not great at reasoning.

Post reply on HN