Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can language models serve as text-based world simulators?

arxiv.org

Post reply on HN
Print view View original

41–50 of 68 posts

Re: Can language models serve as text-based world simulators?

#41

Post by sk11001 »

catlifeonmars wrote: post #40
sk11001 wrote: post #17

Recently I'm getting some strange behavior from ChatGPT (with GPT-4) where it outputs a code snippet in which it declares a variable, and then on the very next line it forgets the name of the variable and refers to it as something else. Or it refers to the variable but it makes a typo in the name. If that's the behavior by one of the best models on a few lines of code, I'm not hopeful for a world simulator any time s…

Think about what the model is doing: It’s sampling from a statistical distribution of likely tokens, one of which is the correctly spelled variable name.

My point is that in previous versions it was sampling more correctly more often than it has been recently.

Re: Can language models serve as text-based world simulators?

#42

Post by TeMPOraL »

danjc wrote: post #18
sbotzek wrote: post #15

I tried for a few months getting ChatGPT4 to work with a MUD. In my experience it's not very good at that particular task. One problem I ran into was its ability to logically connect rooms. In a MUD you navigate by going north, south, east, west, up, or down. Not every room lets you go any direction. And usually, if you go east, in your new room, you can go west. Rarely a level creator will make this not be true. Cha…

Somewhat related, Yann LeCun posted a few months back about how many concepts aren't understood through language and therefore can't be modeled through it (which is why LLM's are terrible with things like position and direction). https://x.com/ylecun/status/1768353714794901530?s=46

My bet is that this is wrong, and at the same time, language isn't required - just sufficient. I see concepts as defined only through associations with other concepts (which can be modeled as proximity in high-dimensional space, and that's precisely what LLMs are doing) and, sometimes, through memorized sensory data - the latter isn't the typical case, but it's needed to make the recursive definition (concepts defined in terms of concepts) stay anchored to reality.

From that follows that written language is enough to build that structure of concepts (latent space in ML terms). So is spoken language. So is vision in general, or hearing in general[0]. The brain will build one concept space out of all inputs available; it is necessary and sufficient to have at least one, but none of them alone is itself necessary.

--

[0] - Languages are higher-level regularities used for communication, growing on top of those senses, but not strictly necessary for understanding the real world. I'd use people who have no perceptible inner voice and high visualization skills as a counter to the idea that concepts need to be thought of symbolically in something resembling a written or spoken language.

Re: Can language models serve as text-based world simulators?

#43

Post by TeMPOraL »

catlifeonmars wrote: post #40
sk11001 wrote: post #17

Recently I'm getting some strange behavior from ChatGPT (with GPT-4) where it outputs a code snippet in which it declares a variable, and then on the very next line it forgets the name of the variable and refers to it as something else. Or it refers to the variable but it makes a typo in the name. If that's the behavior by one of the best models on a few lines of code, I'm not hopeful for a world simulator any time s…

Think about what the model is doing: It’s sampling from a statistical distribution of likely tokens, one of which is the correctly spelled variable name.

Yes. The presence of the correctly spelled name earlier in the context should've dominated the distribution so much that it should've been extremely unlikely to select an incorrectly spelled variable name in the place where the correct name should go.

Re: Can language models serve as text-based world simulators?

#45

Post by rany_ »

sk11001 wrote: post #41
catlifeonmars wrote: post #40

Earlier quoted context omitted.

Think about what the model is doing: It’s sampling from a statistical distribution of likely tokens, one of which is the correctly spelled variable name.

My point is that in previous versions it was sampling more correctly more often than it has been recently.

From what I've heard, the reason it's doing that is to add an AI fingerprint. It has to pick less likely tokens to encode this information to the detriment of the output quality. Unfortunately it's only hearsay but it made sense to me so I thought I'd share.

Re: Can language models serve as text-based world simulators?

#48

Post by moritzwarhier »

echelon wrote: post #46
binary132 wrote: post #34

Can headlines written in the interrogative case ever be answered in the affirmative?

There exists at least one headline written in the interrogative case that can be answered in the affirmative.

https://web.archive.org/web/20160410022155/http://ccdb5fs.ke...

Re: Can language models serve as text-based world simulators?

#49

Post by skywhopper »

Folcon wrote: post #25
danjc wrote: post #18

Earlier quoted context omitted.

Somewhat related, Yann LeCun posted a few months back about how many concepts aren't understood through language and therefore can't be modeled through it (which is why LLM's are terrible with things like position and direction). https://x.com/ylecun/status/1768353714794901530?s=46

To people who claim that "thinking and reasoning require language", here is a problem: Imagine standing at the North Pole of the Earth. Walk in any direction, in a straight line, for 1 km. Now turn 90 degrees to the left. Walk for as long as it takes to pass your starting point. Have you walked: 1. More than 2xPi km 2. Exactly 2xPi km 3. Less than 2xPi km 4. I never came close to my starting point. Think about how yo…

Language is useful to transmit the concepts but is not sufficient to actually solve problems with those concepts.

Re: Can language models serve as text-based world simulators?

#50

Post by elliotto »

sk11001 wrote: post #41
catlifeonmars wrote: post #40

Earlier quoted context omitted.

Think about what the model is doing: It’s sampling from a statistical distribution of likely tokens, one of which is the correctly spelled variable name.

My point is that in previous versions it was sampling more correctly more often than it has been recently.

People seem to have noticed that chatgpt has good days and bad days. we don't really know what openai is doing in the background, so it could be that some days you get assigned a really bad B in an A/B model deployment test, or maybe they're throttling the performance because they're at peak, or likely some other factor. But it's an observed phenomenon that it performs well and then doesn't.
Post reply on HN