Language Models Represent Space and Time
1–10 of 192 posts
Re: Language Models Represent Space and Time
#2The Twitter thread is worth looking at. Very Fascinating
https://twitter.com/wesg52/status/1709551516577902782?t=3b2F...
Re: Language Models Represent Space and Time
#3I think we have a new member to add to the group: Lies, damn lies, and statistics… and now neural activations. How exciting!
This reminds me of all those fMRI studies that look at brain blood flow before the dead salmon experiment came out.
Re: Language Models Represent Space and Time
#4A lot of HNers were so adamant that the LLMs understand absolutely nothing and that these models are just predicting the next most likely word.
I think with this paper it becomes clear that the adamant denial was just human bias talking. LLMs crossed a certain line here.
You would think people would be amazed or in awe or react in fear at technological break through a and they often are. The weird part for LLMs was the delusion. Many people were in denial that it was any sort of breakthrough.
Re: Language Models Represent Space and Time
#5The next step in the research should be ablation of these time and space neurons to see how it affects accuracy on space/time completions which would help rule out the memorization of linear probes.
Re: Language Models Represent Space and Time
#6A lot of HNers were so adamant that the LLMs understand absolutely nothing and that these models are just predicting the next most likely word. I think with this paper it becomes clear that the adamant denial was just human bias talking. LLMs crossed a certain line here. You would think people would be amazed or in awe or react in fear at technological break through a and they often are. The weird part for LLMs was t…
This is a preprint that was just uploaded to arxiv two days ago. Don't be hasty and assume that it settles any matter at all. Many such claims have been made before and many counter-claims also. There is still a lively debate on the subject and it will be some time before there is agreement.
More generally, any scholarly article is a claim, and should never be read as automatically true. That's something to keep in mind.
Re: Language Models Represent Space and Time
#7No, no they don't. People looking at the output of LLMs project their own internal models of the world, time, and space onto the output and project that model as if it were the LLM's.
Re: Language Models Represent Space and Time
#8A lot of HNers were so adamant that the LLMs understand absolutely nothing and that these models are just predicting the next most likely word. I think with this paper it becomes clear that the adamant denial was just human bias talking. LLMs crossed a certain line here. You would think people would be amazed or in awe or react in fear at technological break through a and they often are. The weird part for LLMs was t…
I think LLMs are presenting some uncomfortable philosophical questions for people about how our own brains work and admitting that there is any kind of "intelligence" (even if very basic) in an LLM is an admission that our own brains may work in a similar manner.
Re: Language Models Represent Space and Time
#9A lot of HNers were so adamant that the LLMs understand absolutely nothing and that these models are just predicting the next most likely word. I think with this paper it becomes clear that the adamant denial was just human bias talking. LLMs crossed a certain line here. You would think people would be amazed or in awe or react in fear at technological break through a and they often are. The weird part for LLMs was t…
Indeed. I was offering examples of how you could bootstrap cognitive processes at the beginning of the year and people were just sticking their fingers in their ears. A lot of people exercise their intellectual, economic, or political freedoms by rejecting others' efforts to gain the same things.
Re: Language Models Represent Space and Time
#10No, no they don't. People looking at the output of LLMs project their own internal models of the world, time, and space onto the output and project that model as if it were the LLM's.
The paper is looking at internal representations, not output.