Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language Models Represent Space and Time

arxiv.org

Post reply on HN
Print view View original

131–140 of 192 posts

Re: Language Models Represent Space and Time

#131

Post by famouswaffles »

YeGoblynQueenne wrote: post #122
famouswaffles wrote: post #116

Earlier quoted context omitted.

>computers are logic machines and all of Computer Science is based on logic; and it works just fine. Not what I mean. Logic is part of the real world. Logic is not the real world. The idea that you can use this small subset of the world to model the whole thing is what is incredibly suspect. No one has demonstrated this and there is no real reason to believe it can. >To clarify, those are all logic-based approaches t…

>> The idea that you can use this small subset of the world to model the whole thing is what is incredibly suspect. No one has demonstrated this and there is no real reason to believe it can. I agree and I don't think there's any kind of logic that can do that, but there is also no other formal system that can, so far. I'm not sure if you are suggesting there is? >> But you won't beat logic on problems with clear def…

>so far. I'm not sure if you are suggesting there is?

No i wasn't. I guess i wasn't very clear in my first reply.

I was mainly getting at this,

>and, consequently, its' a very bad idea to try and make machines that "think like humans", because that way we'll only make machines with none of the advantages of machines and all the disadvantages of computers.

No one is scaling up and pouring millions of compute into LLMs for general intelligence because they thought it was an excellent idea before the fact(virtually no one did, even some of the most verbal proponents).

They're doing it because it's seems to be working in a way logic failed to. and logic had the headstart, both in research and public consciousness. Nearly all of fictional ai is an envisioning of the hard symbolic logic general intelligence system that dominated early ai research. Logic was not the underdog here.

The point i was really driving at is that you say "because that way we'll only make machines with none of the advantages of machines and all the disadvantages of computers." almost like it's a choice, like Logic and GPT are both on the field and people are going for the worse player. Logic is not even in consideration because ot couldn't make the cut.

Re: Language Models Represent Space and Time

#132

Post by YeGoblynQueenne »

circuit10 wrote: post #120
YeGoblynQueenne wrote: post #101

Earlier quoted context omitted.

Yes, I see what you mean. It's frustrating but the word "model" is severely overused in Computer Science and AI and it can cause a lot of confusion. Briefly, a "world model" is a theory possessed by an autonomous agent that describes the entities that exist in the world and how they interact with each other and with the agent, and that the agent can use to make decisions. This is the sense in which "model" is used wh…

Since language often describes the world, I think a good language model must include a world model

Well, this has been claimed often, namely by some of the people who have developed statistical language models [1] but it's really not obvious how that should work. Where would a language model find the world model? Where would it store it? And why would it even need it?

Obviously I don't know how the human linguistic ability works but it's clear that for us, text, words, language, isn't carrying around with it a representation of the entire world. So where would a language model find it in its training text?

For example if I say "I sat by the sea today", there's no representation of the sea in the word "sea", rather, the representation is inside my head and yours, and the word "sea" is only used to point to that representation [2]. Why would it ever be possible to derive the representation just from the pointer? Like I say, it's really not obvious and I don't think anyone has given a convincing explanation - it's usually words all the way down when people try. In any case, it seems that humans had most of the capabilities conferred by a world model (which we share with other animals) before we had language (which we do not - share with other animals), so it seems likely that a world model must be developed first, before linguistic ability can be built on top of it. But that's a conjecture, of course.

I guess maybe, in principle, it should be possible to treat all of human cognition as some kind of hidden (latent) variable and then train a model to represent it only from its expression in text, but we don't really have any kind of modelling technique with that power. That would be a truly omnipotent model. So powerful, indeed, that it wouldn't need to do that in the first place- it would already be a kind of machine god.

______________

[1] e.g. I believe Tomáš Mikolov, one of the creators of word2vec, has claimed something like that but I'm not sure if I have a source for my own claim; more recently Ilya Sutskever, of OpenAI, has said words to that effect- I have a link somewhere I think, if you are curious.

[2] That's obvious because the word "sea" doesn't mean anything to someone who doesn't know English, but they'll probably have a word for the sea in their language, so they'll have the concept, without needing a specific word for it.

Re: Language Models Represent Space and Time

#133

Post by ImHereToVote »

famouswaffles wrote: post #110
nyrikki wrote: post #97

Earlier quoted context omitted.

Pairwise linear in respect to inputs, for feed forward networks like those that use attention is the restriction related to linear or linearizable. Those words are important. But we know that ML cannot model all turning machines in the limit. As to more reasonable models that gets more complex. Rice's theorem in respect to functions doesn't fit with you claim: For any non-trivial property of partial functions, no gen…

>But we know that ML cannot model all turning machines in the limit. No computer is truly turing without infinite memory, including humans. >Rice's theorem in respect to functions doesn't fit with you claim: For any non-trivial property of partial functions, no general and effective method can decide whether an algorithm computes a partial function with that property. The generalization problem being the important pa…

I dunno. The guy that's freebasing copium sounds more convincing. Sorry.

Re: Language Models Represent Space and Time

#134

Post by YeGoblynQueenne »

famouswaffles wrote: post #131
YeGoblynQueenne wrote: post #122

Earlier quoted context omitted.

>> The idea that you can use this small subset of the world to model the whole thing is what is incredibly suspect. No one has demonstrated this and there is no real reason to believe it can. I agree and I don't think there's any kind of logic that can do that, but there is also no other formal system that can, so far. I'm not sure if you are suggesting there is? >> But you won't beat logic on problems with clear def…

>so far. I'm not sure if you are suggesting there is? No i wasn't. I guess i wasn't very clear in my first reply. I was mainly getting at this, >and, consequently, its' a very bad idea to try and make machines that "think like humans", because that way we'll only make machines with none of the advantages of machines and all the disadvantages of computers. No one is scaling up and pouring millions of compute into LLMs…

Like I say in my earlier comment, that's not right. Logic-based AI is still dominant in many fields. There is a lot of excitement about statistical machine learning (I know, it's an understatement) but that's only because statistical machine learning is finally working and doing things that couldn't be done with logic- not because logic can't do the things that statistical machine learning can't do (it can), and not because statistical machine learning can do the things that logic can do (it can't).

There are two worlds, if you want. For me it's a mistake to try and keep them separated. All the great pioneers of AI were not only this or only that people. e.g. Shannon's MSc thesis gave us boolean logic-based circuits (logic gates) and he also introduced information theory. The people who have made real contributions to AI and to computer science were never one-trick ponies.

An analogy I like to make is that we have both airplanes and helicopters. A flying machine is something so useful to have that we 're going to use any kind we can make. Obviously a helicopter will not compete with a jet for speed, but a jet isn't anywhere as manoeuverable or flexible as a helicopter. So we use both.

>> Logic was not the underdog here.

It wasn't, but there was a bit of a Triassic extinction event, with the last AI winter of the '90s that took the expert systems and basically severed the continuity of logic-based AI research. The story is more complex than that, but logic-based AI was dealt a powerful blow, and progress slowed down. Although again like I say in my other comment, it didn't get completely extinguished. Perhaps, like we recognise birds today as the remaining dinosaurs, we'll recognise the old-new wave of logic-based AI that is hidden by the AI effect.

Re: Language Models Represent Space and Time

#135

Post by nyrikki »

wnoise wrote: post #118
ImHereToVote wrote: post #93

Earlier quoted context omitted.

So what is it about a biological neuron that sidesteps the halting problem? Is it because there is a weight and bias set to it. Someone should tell OpenAI to add weights and biases to their LLMs. That will make them make have an emotional like intuition, will it not? EDIT: Maybe if we train the ANNs to focus their training on attention based techniques. Then they will simply tire on halts and continue on other proble…

Why do you think "biological neuron"s sidestep the halting problem? People, who happen to run on biological neurons, have a sense of boredom that tries other approaches, and is also willing to eventually "give up", which aren't well captured in the standard algorithmic approaches.

The halting problem simply doesn't apply to biological neurons, they don't side step them, unless your context is humans writing down an Algorithm.

The rules of a human writing down an algorithm is the same thing as a Turing machine running an algorithm.

The halting problem applies for any system of computation that is at least as powerful as a TM, including any type of arithmetic or non-arithmetic calculation that is well-defined, AKA deterministic.

Cortical neuron firing is non-deterministic and more closely is modeled as probabilistic but still stochastic.

https://www.biorxiv.org/content/10.1101/2022.12.03.518978v1

Machine learning is constrained by the halting problem.

HALT is the conical example for what is decidable, but other problems exist and sometimes PAC learnability hits practical limits far before the finite time limits of RE.

As an example not invoking HALT:

https://arxiv.org/abs/2208.10255

There are absolutely constraints on BNNs, but as BNNs aren't deterministic Turing machines, it doesn't apply.

The real question is why do people resort to elementary oversimplified models of biological brains?

If you are in the field of studying the brain you will look for deterministic models that fit your needs to make computation more likely to be tractable.

But the false equivalency of ANNs to BNNs is problematic as a distraction from finding tractable solutions for computation.

Re: Language Models Represent Space and Time

#136

Post by nyrikki »

ImHereToVote wrote: post #126
wnoise wrote: post #118

Earlier quoted context omitted.

Why do you think "biological neuron"s sidestep the halting problem? People, who happen to run on biological neurons, have a sense of boredom that tries other approaches, and is also willing to eventually "give up", which aren't well captured in the standard algorithmic approaches.

Isn't that exactly how attention based neural networks work?

Attention is probably most easily conceptualized as run time reweighting.

A powerful tool but it doesn't really change the underlying model it is just modifying the weights at runtime.

Re: Language Models Represent Space and Time

#137

Post by nyrikki »

ImHereToVote wrote: post #133
famouswaffles wrote: post #110

Earlier quoted context omitted.

>But we know that ML cannot model all turning machines in the limit. No computer is truly turing without infinite memory, including humans. >Rice's theorem in respect to functions doesn't fit with you claim: For any non-trivial property of partial functions, no general and effective method can decide whether an algorithm computes a partial function with that property. The generalization problem being the important pa…

I dunno. The guy that's freebasing copium sounds more convincing. Sorry.

Well as all you have as a response is an ad hominem you aren't the intended audience anyway.

I would love to learn why I am wrong.

Re: Language Models Represent Space and Time

#138

Post by ComplexSystems »

lsy wrote: post #78

The interpretation of the LLM having a "world model" is a big stretch in terminology. Encoding a coherent or accurate set of spatial coordinates for places is not equivalent to understanding the actuality of world space and the relationship between items in that space. By this definition an accurate spreadsheet of the lat-long of every major city would also constitute a "world model". It shouldn't be surprising to an…

"Encoding a coherent or accurate set of spatial coordinates for places is not equivalent to understanding the actuality of world space and the relationship between items in that space."

What do you mean by the "actuality" of world space? It certainly has memorized an enormous data set of geographic points for various important locations and can do some pretty sophisticated reasoning and inference from them, and with the code interpreter it can compute various metrics derived from them. What can this thing do that humans can't?

Re: Language Models Represent Space and Time

#139

Post by klysm »

xpe wrote: post #74
klysm wrote: post #13

I’m most surprised that the embedding is actually linear. I wonder if that makes extrapolation perform better

The papers cites earlier papers that posit a ‘linear representation hypothesis’. Does your surprise “factor in” these papers? Do you suspect something different in play for the current paper?

Didn’t have that factored into my surprise because I’m not that informed. I just thought that the latent space representation being linear was interesting.

Re: Language Models Represent Space and Time

#140

Post by circuit10 »

YeGoblynQueenne wrote: post #132
circuit10 wrote: post #120

Earlier quoted context omitted.

Since language often describes the world, I think a good language model must include a world model

Well, this has been claimed often, namely by some of the people who have developed statistical language models [1] but it's really not obvious how that should work. Where would a language model find the world model? Where would it store it? And why would it even need it? Obviously I don't know how the human linguistic ability works but it's clear that for us, text, words, language, isn't carrying around with it a rep…

No one can experience the real world directly. Whether it’s through language, vision (which feels quite “real” but is really just a 2D projection of light that we interpret), sound or anything else, it’s all just some byproduct of the world that we nonetheless can make useful predictions with. LLMs are more limited in their input data, but I don’t think the difference is fundamental; it’s all just different abstract representations of the world. Like how by looking at the sea we can gain some useful understanding of how it works, an LLM can, via descriptions of the sea, pick up enough information about it to answer questions about it or write text about it. It wouldn’t “find” a world model, it learns it based on its training data, it gets stored in the weights (because there’s nowhere else to store it), and it needs it because it’s making predictions about text, which often describes the world. If you don’t have a basic idea of things like cause and effect it’s difficult to write coherent text
Post reply on HN