Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language Models Represent Space and Time

arxiv.org

Post reply on HN
Print view View original

71–80 of 192 posts

Re: Language Models Represent Space and Time

#71

Post by gpderetta »

somenameforme wrote: post #16
civilitty wrote: post #3

I think we have a new member to add to the group: Lies, damn lies, and statistics… and now neural activations. How exciting! This reminds me of all those fMRI studies that look at brain blood flow before the dead salmon experiment came out.

Haha, I have no idea how I was not aware of the dead salmon experiment prior. This article [1] has a nice overview for anybody else in my shoes. Thanks for the reference. [1] - https://blogs.scientificamerican.com/scicurious-brain/ignobe...

First of all that's an excellent article. But also:

> Some people like to use the salmon study as proof that fMRI is woo, but this isn't the case, it's actually a study to show the importance of correcting your stats.

I.e. be aware of the "look elsewhere effect".

Re: Language Models Represent Space and Time

#72

Post by kelseyfrog »

sebzim4500 wrote: post #37
xpe wrote: post #31

Earlier quoted context omitted.

Well said. Also, the paper’s use of “world model” (instead of simply “model”) doesn’t help clarify the situation.

I do not see how referring to a world model as simply a "model" could possibly provide clarity. LLM literally stands for Large Language Model, it has a 'model' in it by definition.

Is-a vs has-a relationship.

Re: Language Models Represent Space and Time

#73

Post by nyrikki »

ImHereToVote wrote: post #64
nyrikki wrote: post #54

Earlier quoted context omitted.

No, this paper doesn't change that claim at all. This paper is about learnability and not generalization of what is learned. From the papers summary, note that they are referencing learning and not some implications that it gets us closer to a universal quantifier. "We have provided evidence that LLMs learn linear representations of space and time that are unified across entity types and fairly robust to prompting, a…

I mean parrots have neurons. I didn't realize the line referred to the actual birds "parrot". Parrots are wicked smart.

Parrots, in the context of stochastic parrots is in reference to repeating or imitation mechanically without understanding.

The 'common sense' problem in AI is the important part, not the as of yet undefined abstract concept of intelligence.

LLMs are still pattern matching and finding with some stochastic aspects.

But artificial nurons are inspired by biological nurons, they are not equivalent in respect to computability. But artificial nural networks are useful and powerful for some problems that traditionally weren't solvable by other means.

Biological nurons have to embrace non-linear stochastic processes, which is a different problem set.

While not expressed in popsci this has been known for decades.

Re: Language Models Represent Space and Time

#74

Post by xpe »

klysm wrote: post #13

I’m most surprised that the embedding is actually linear. I wonder if that makes extrapolation perform better

The papers cites earlier papers that posit a ‘linear representation hypothesis’. Does your surprise “factor in” these papers? Do you suspect something different in play for the current paper?

Re: Language Models Represent Space and Time

#75

Post by kridsdale3 »

moffkalast wrote: post #51
ilaksh wrote: post #30

One thing that is clouding this discussion is that most people are mixing up a lot of different characteristics that animals like humans have as if they were all the same thing. So for many people they don't really distinguish between things like "reasoning", "self-aware", "conscious", "alive", "sentient", "intelligent", "has world model". They also don't distinguish between different types or varying levels of cogni…

It's not the first time this has been a problem to accurately specify either. Throughout history animals proved nigh impossible to classify in these terms and were considered so many things, from something that's intertwined with nature as a hive mind of sorts, to complete automatons by Descartes, and later to beings that are mostly completely identical to us by Darwin. I suppose it's also an interesting juxtapositio…

So all we have to do is plug a USB cable from a device running an LLM in to the back of a monkey.

Re: Language Models Represent Space and Time

#76

Post by kridsdale3 »

nmarinov wrote: post #25
famouswaffles wrote: post #2

The Twitter thread is worth looking at. Very Fascinating https://twitter.com/wesg52/status/1709551516577902782?t=3b2F...

Seconded, it's a good summary with graphs and commentary. Also nitter to avoid the login wall: https://nitter.net/wesg52/status/1709551516577902782

Thank you for your service.

Re: Language Models Represent Space and Time

#77

Post by gpderetta »

fipar wrote: post #55
nopinsight wrote: post #22

Earlier quoted context omitted.

Current LLMs likely have multiple world models with varying qualities. Good prompts are required to activate suitable models for each task. 1. Did you use GPT-3.5 (free version) or GPT-4 (paid one) to form the judgment? Their performances on harder tasks differ significantly as shown in https://openai.com/research/gpt-4 . 2. Have you tried adding "Please think step by step" to some harder requests? This simple phrase…

As stated in another response, I'm using 3, so I'm probably missing some good stuff. I have tried adding things like "please think step by step" (quite literally that question actually), and also "please make sure you check the facts before answering so you don't include non-existing arguments" (when asking about a cli tool that takes arguments), but I didn't notice a significant improvement. I like what you say in y…

I don't think baseline ChatGPT si capable to "please check the facts before answering so you don't include non-existing arguments".

Re: Language Models Represent Space and Time

#78

Post by lsy »

The interpretation of the LLM having a "world model" is a big stretch in terminology. Encoding a coherent or accurate set of spatial coordinates for places is not equivalent to understanding the actuality of world space and the relationship between items in that space. By this definition an accurate spreadsheet of the lat-long of every major city would also constitute a "world model". It shouldn't be surprising to anyone that a model trained on (likely many) such spreadsheets would also encode that coordinate data. What's notable is that it still takes a team of human researchers to plot these raw numbers onto a map and to interpret them as experiential differentiators in physical space.

There's no doubt that an LLM can uncover whatever is structurally included in its training data, even if that encoding is implicit. What's less believable is that the LLM somehow achieves a grounded understanding of physical aspects of experience purely from streams of raw text.

I think the paper engages in some equivocation here, as the abstract differentiates between "an enormous collection of superficial statistics" and "a coherent model of the data generating process", without admitting that the first should well imply the second. But that doesn't then further imply a "world model" in the sense that we understand it as sentient beings. For us, an internal model is useful, but when our model or texts don't agree with the actuality of the world, the actuality of the world takes precedence. For an LLM there is no distinguishing between its trained representations and an actual exterior place, or even any sense that an exterior space exists.

Re: Language Models Represent Space and Time

#79

Post by ImHereToVote »

nyrikki wrote: post #73
ImHereToVote wrote: post #64

Earlier quoted context omitted.

I mean parrots have neurons. I didn't realize the line referred to the actual birds "parrot". Parrots are wicked smart.

Parrots, in the context of stochastic parrots is in reference to repeating or imitation mechanically without understanding. The 'common sense' problem in AI is the important part, not the as of yet undefined abstract concept of intelligence. LLMs are still pattern matching and finding with some stochastic aspects. But artificial nurons are inspired by biological nurons, they are not equivalent in respect to computabi…

Great so biological neurons can perform feats of intuition, that simulated neurons simply can't. Even theoretically. Gotcha. Thank you very much for the explanation.
Post reply on HN