Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language Models Represent Space and Time

arxiv.org

Post reply on HN
Print view View original

111–120 of 192 posts

Re: Language Models Represent Space and Time

#111

Post by wanderingstan »

somewhereoutth wrote: post #91
ilaksh wrote: post #30

One thing that is clouding this discussion is that most people are mixing up a lot of different characteristics that animals like humans have as if they were all the same thing. So for many people they don't really distinguish between things like "reasoning", "self-aware", "conscious", "alive", "sentient", "intelligent", "has world model". They also don't distinguish between different types or varying levels of cogni…

If an LLM is trained on just text, then it's internal model can only be based on such text. I think the 'conceit' of LLM as AI advocates is that this model is just as (or at least nearly as) our own internal model - that the 'simplest' model to match the text is indeed our 'real' model that matches reality as a whole. A bit of a stretch surely!

I’m no expert, but the word “just” is tricky. You could say mammalian brain is trained on “just” nerve impulses.

Re: Language Models Represent Space and Time

#112

Post by famouswaffles »

YeGoblynQueenne wrote: post #109
famouswaffles wrote: post #68

Earlier quoted context omitted.

The second. Mathematical logic thrives on precision, clear definitions, and unambiguous axioms, but real-world systems are often marked by vagueness, uncertainty, and dynamic change. Gödel’s Incompleteness Theorems also demonstrates that in any sufficiently powerful mathematical system, there are true statements that cannot be proven within the system. This implies that no matter how refined a logical system you devi…

Gödel didn't say anything about real world phenomena. He was talking about formal languages and mathematics.

Of course. But if you truly cannot model every true statement in any formally devised system, then you are by definition going to have to reject valid rules that your logic cannot verify if you intend your system to perfectly logical.

Re: Language Models Represent Space and Time

#113

Post by ilaksh »

essive wrote: post #100
ilaksh wrote: post #30

One thing that is clouding this discussion is that most people are mixing up a lot of different characteristics that animals like humans have as if they were all the same thing. So for many people they don't really distinguish between things like "reasoning", "self-aware", "conscious", "alive", "sentient", "intelligent", "has world model". They also don't distinguish between different types or varying levels of cogni…

I think the term "world model" is a highly overloaded term. What does it really mean in your opinion?

A reasonably precise although not universal definition could be to take the data points from the study in relation to the use of those terms in the study.

I think the main idea though is that there is some structure in the memory that resembles a structure in the real world to some meaningful degree. One aspect of this is composition, the other is how closely it maps to the real world.

If it were literally just a lookup table where the indices did not correspond to physical proximity then it would definitely not be valid to say that this was really a world model in that context.

Re: Language Models Represent Space and Time

#114

Post by somewhereoutth »

mjburgess wrote: post #99
somewhereoutth wrote: post #91

Earlier quoted context omitted.

If an LLM is trained on just text, then it's internal model can only be based on such text. I think the 'conceit' of LLM as AI advocates is that this model is just as (or at least nearly as) our own internal model - that the 'simplest' model to match the text is indeed our 'real' model that matches reality as a whole. A bit of a stretch surely!

LLMs are a redemonstration of the lesson of sun worship: apes are easily fooled. If the output is indistinguishably 'world modelling', so must its mechanism be. There is no 'world' model in a hashmap from questions to answers about a world . There is only the condition that, given those questions, any old ape would accept those answers. Science is required to determine whether a system has properties, such as a ratio…

Nice try LLMy

Re: Language Models Represent Space and Time

#115

Post by nopinsight »

lsy wrote: post #90
nopinsight wrote: post #86

Earlier quoted context omitted.

Current LLMs clearly have much coarser models of the world than humans do. Their training data do not include as many modalities and training data with spatiotemporal dimensions are inadequate. Up and coming multimodal models are changing this.

"Multimodal" models only promise an additional layer of indirection and interpretation from the world through sensors into the same conceptual mechanism. An image is converted to tokens, a sensor is converted to coordinates, and it is all fed back into a correlating model. But this isn't adequate either. The famous dress illusion picture has RGB values closer to gold and white, but the actual physical dress was black…

> There's no doubt that an LLM can uncover whatever is structurally included in its training data, even if that encoding is implicit. What's less believable is that the LLM somehow achieves a grounded understanding of physical aspects of experience purely from streams of raw text.

I was responding to the point in your first paragraph in your upper level comment and this quoted paragraph. Multimodal models will address that.

Your new point about perception could still be a problem for multimodal models, but it’s also possible that with sufficient data, the new models could address that (eg it could say people perceive the colors differently, as A and B).

Multimodal models map between human language and raw sensory data. It’s fundamentally similar, in principle, to part of how we learn our first language. (Not saying we do only that though.)

Re: Language Models Represent Space and Time

#116

Post by famouswaffles »

YeGoblynQueenne wrote: post #105
famouswaffles wrote: post #35

Earlier quoted context omitted.

>because that way we'll only make machines with none of the advantages of machines and all the disadvantages of computers. There is no evidence, basically none whatsoever that general "perfect logical reasoning" is a thing that actually exists in the real world. None. No animal we've observed does it. Humans certainly don't do it. The only realm this idea actually works is Fiction. and this was not like for a lack of…

That's a common take but it doesn't really hold any water: computers are logic machines and all of Computer Science is based on logic; and it works just fine. Besides which, you may not hear about them in the news but pretty much all the classical, symbolic- and logic-based approaches of Good, Old-Fashioned AI are still going strong and are doing very well thank you in tasks in which statistical machine learning appr…

>computers are logic machines and all of Computer Science is based on logic; and it works just fine.

Not what I mean. Logic is part of the real world. Logic is not the real world. The idea that you can use this small subset of the world to model the whole thing is what is incredibly suspect. No one has demonstrated this and there is no real reason to believe it can.

>To clarify, those are all logic-based approaches that remain the state of the art in classical AI tasks where statistical machine learning has made no progress in the last many decades

Logic is good at what logic does. Please don't take this to mean me calling logic useless. It's not that statistical machine learning has not made progress. But you won't beat logic on problems with clear definitions and unambiguous axioms. That is very cool but that is clearly not all of reality.

Re: Language Models Represent Space and Time

#117

Post by haltist »

wnoise wrote: post #107
haltist wrote: post #87

> coherent model of the data generating process -- a world model. I'd never seen a definition of world model but this seems deficient in several ways because it does not mention anything about abstraction and logical reasoning.

Those aren't part of the model though. The model is just a representation of the world. An abstract representation can be useful, but so can more concrete ones. A representation that is amenable to logical reasoning is also useful, but the reasoning isn't part of the model.

Then why call it a world model? It's a "data" model and not actually a "world" model. World models would need to have logical relationships between the entities being modeled which would include non-trivial logical relations, R(x,y), and properties, P(x). Current language models do not have such representations and fail miserably on any task that is not explicitly included in the training set (sudoku being an obvious example).

Re: Language Models Represent Space and Time

#118

Post by wnoise »

ImHereToVote wrote: post #93
nyrikki wrote: post #89

Earlier quoted context omitted.

Not quite that simple, or well known. We know that the class of total Turing computable functions is not learnable in the limit. But as to what else is AI-complete is mostly open questions. But there is a difference. Obviously ML is far better than humans in some domains so it is not a simple dichotomy.

So what is it about a biological neuron that sidesteps the halting problem? Is it because there is a weight and bias set to it. Someone should tell OpenAI to add weights and biases to their LLMs. That will make them make have an emotional like intuition, will it not? EDIT: Maybe if we train the ANNs to focus their training on attention based techniques. Then they will simply tire on halts and continue on other proble…

Why do you think "biological neuron"s sidestep the halting problem?

People, who happen to run on biological neurons, have a sense of boredom that tries other approaches, and is also willing to eventually "give up", which aren't well captured in the standard algorithmic approaches.

Re: Language Models Represent Space and Time

#119

Post by famouswaffles »

haltist wrote: post #117
wnoise wrote: post #107

Earlier quoted context omitted.

Those aren't part of the model though. The model is just a representation of the world. An abstract representation can be useful, but so can more concrete ones. A representation that is amenable to logical reasoning is also useful, but the reasoning isn't part of the model.

Then why call it a world model? It's a "data" model and not actually a "world" model. World models would need to have logical relationships between the entities being modeled which would include non-trivial logical relations, R(x,y), and properties, P(x). Current language models do not have such representations and fail miserably on any task that is not explicitly included in the training set (sudoku being an obvious…

>World models would need to have logical relationships between the entities being

No they wouldn't. There is no evidence, basically none whatsoever that logical can model reality or that general "perfect logical reasoning" is a thing that actually exists in the real world in real world relationships. None.

No animal we've observed does it. Humans certainly don't do it. The only realm this idea actually seems to work is Fiction. and this was not like for a lack of trying. Some of the greatest minds worked on this for decades and some people still don't seem to get it. Logic doesn't scale. It breaks at real world relationships.

We were using logic for image, speech, language synthesis and just had to throw it away once something that actually worked came about.

If logic was so great at modeling the world, we wouldn't be using LLMs like GPT-4 , we'd be using Cyc, the system with decades of a headstart.

Logic systems are that guy in the stands yelling that he could've made the shot, while he's not even on the field.

>included in the training set (sudoku being an obvious example)

LLMs can play sudoku just fine. https://arxiv.org/abs/2305.08291

Re: Language Models Represent Space and Time

#120

Post by circuit10 »

YeGoblynQueenne wrote: post #101
circuit10 wrote: post #27

Earlier quoted context omitted.

I thought it was completely obvious that if it can predict things about the world as well as it can it has some form of world model, even if inaccurate. I’m surprised that anyone would argue that a language model doesn’t model things

Yes, I see what you mean. It's frustrating but the word "model" is severely overused in Computer Science and AI and it can cause a lot of confusion. Briefly, a "world model" is a theory possessed by an autonomous agent that describes the entities that exist in the world and how they interact with each other and with the agent, and that the agent can use to make decisions. This is the sense in which "model" is used wh…

Since language often describes the world, I think a good language model must include a world model
Post reply on HN