Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can language models serve as text-based world simulators?

arxiv.org

Post reply on HN
Print view View original

31–40 of 68 posts

Re: Can language models serve as text-based world simulators?

#31

Post by erik_seaberg »

Folcon wrote: post #25
danjc wrote: post #18

Earlier quoted context omitted.

Somewhat related, Yann LeCun posted a few months back about how many concepts aren't understood through language and therefore can't be modeled through it (which is why LLM's are terrible with things like position and direction). https://x.com/ylecun/status/1768353714794901530?s=46

To people who claim that "thinking and reasoning require language", here is a problem: Imagine standing at the North Pole of the Earth. Walk in any direction, in a straight line, for 1 km. Now turn 90 degrees to the left. Walk for as long as it takes to pass your starting point. Have you walked: 1. More than 2xPi km 2. Exactly 2xPi km 3. Less than 2xPi km 4. I never came close to my starting point. Think about how yo…

If the brain actually used language to represent ideas in use, we should have succeeded in finding its https://en.wikipedia.org/wiki/Universal_grammar. Instead it seems more like language is a kind of lossy compression we keep reinventing for ideas, to export them from a brain in some tangible way and get them into another (even in the future).

Re: Can language models serve as text-based world simulators?

#32

Post by Folcon »

dragonwriter wrote: post #28
Folcon wrote: post #25

Earlier quoted context omitted.

To people who claim that "thinking and reasoning require language", here is a problem: Imagine standing at the North Pole of the Earth. Walk in any direction, in a straight line, for 1 km. Now turn 90 degrees to the left. Walk for as long as it takes to pass your starting point. Have you walked: 1. More than 2xPi km 2. Exactly 2xPi km 3. Less than 2xPi km 4. I never came close to my starting point. Think about how yo…

I don't think the example is a good rebuttal of "thinking and reasoning require language". It may be a decent challenge, probably still not an actual rebuttal, of "language is sufficient for all thinking and reasoning", but "X is required for Y" and "X is sufficient for everything encompassed by Y" are very different claims.

That's fair, it's not supposed to be a rebuttal of that :)...

Though now that you said it, I'm really thinking about the original statement.

Honest question, can you give me an example of thinking or reasoning that happens fully independently of reasoning via symbols or their manipulation?

I feel like I'm missing something obvious, but nothing is coming to mind right now :)...

Just thinking about the underlying statement and simplifying language down to symbolic expression. (I was originally going to say manipulation, but it doesn't feel like it quite fits...)

Re: Can language models serve as text-based world simulators?

#33

Post by Folcon »

erik_seaberg wrote: post #31
Folcon wrote: post #25

Earlier quoted context omitted.

To people who claim that "thinking and reasoning require language", here is a problem: Imagine standing at the North Pole of the Earth. Walk in any direction, in a straight line, for 1 km. Now turn 90 degrees to the left. Walk for as long as it takes to pass your starting point. Have you walked: 1. More than 2xPi km 2. Exactly 2xPi km 3. Less than 2xPi km 4. I never came close to my starting point. Think about how yo…

If the brain actually used language to represent ideas in use, we should have succeeded in finding its https://en.wikipedia.org/wiki/Universal_grammar . Instead it seems more like language is a kind of lossy compression we keep reinventing for ideas, to export them from a brain in some tangible way and get them into another (even in the future).

Hmm, I'm not sure if that holds?

Just because you use a mechanism for expressing your ideas and reasoning, doesn't mean that underlying reality has to confirm in any way to it?

We invent new symbols and terms all the time as we experience new phenomena. A universal grammar may still be possible, barring incompleteness anyway, we may just be lacking a whole bunch of ideas still...

Re: Can language models serve as text-based world simulators?

#36

Post by nodja »

sk11001 wrote: post #17

Recently I'm getting some strange behavior from ChatGPT (with GPT-4) where it outputs a code snippet in which it declares a variable, and then on the very next line it forgets the name of the variable and refers to it as something else. Or it refers to the variable but it makes a typo in the name. If that's the behavior by one of the best models on a few lines of code, I'm not hopeful for a world simulator any time s…

Chat GPT4 has been in general both better and worse than 3.5. I sometimes start a conversation with GPT4 and then have 3.5 fix it in another window (by pasting the code and pretending it's mine). It feels like things have degraded for coding tasks specifically, but the more recent knowledge of GPT4 is helpful if I'm asking how to use a certain library that's more recent as it won't just pretend the library exists, GPT4 is also much slower.

I wish there was a ChatGPT 3.5 updated with more recent knowledge.

Re: Can language models serve as text-based world simulators?

#37

Post by dragonwriter »

Folcon wrote: post #32
dragonwriter wrote: post #28

Earlier quoted context omitted.

I don't think the example is a good rebuttal of "thinking and reasoning require language". It may be a decent challenge, probably still not an actual rebuttal, of "language is sufficient for all thinking and reasoning", but "X is required for Y" and "X is sufficient for everything encompassed by Y" are very different claims.

That's fair, it's not supposed to be a rebuttal of that :)... Though now that you said it, I'm really thinking about the original statement. Honest question, can you give me an example of thinking or reasoning that happens fully independently of reasoning via symbols or their manipulation? I feel like I'm missing something obvious, but nothing is coming to mind right now :)... Just thinking about the underlying state…

> Honest question, can you give me an example of thinking or reasoning that happens fully independently of reasoning via symbols or their manipulation?

I don't think we have anything but subjective, indirect understanding of how thinking happens, but I think that, at a minimum, what we describe as "reasoning" specifically is tightly conceptually related to, if not a subset of, manipulation of abstract symbols to which concrete experiences may be approximately mapped.

I'm not sure I'd say this is the same thing as language, but there's at a minimum a shared common symbol-manipulation underlying both. Do the capacities always come together? I'm not sure how we would know that, I think our ability to recognize reasoning is tied to it being mapped to language, and are ability to distinguish something as language rather than nonlinguistic signaling or mimicry of something else that is using language is tied to independent expression of reasoning through it.

Re: Can language models serve as text-based world simulators?

#38

Post by leonardspeiser »

I was able to get this to work with an LLM, but had to build some short term memory to keep awareness as you explored. The current site allows you to interact with an Oregon Trail or Zork like world, but you can specify any time in history and then it will create the stories and stay within that world. I also have it generate some images to go along with you for fun. https://grue.is/ (PS, I don't know how to code, so this is also proof that you can use an LLM to write all the software, here is my github if you are interested in learning more about that: https://github.com/lrspeiser/Grue.is)

Re: Can language models serve as text-based world simulators?

#39

Post by bottom999mottob »

Folcon wrote: post #33
erik_seaberg wrote: post #31

Earlier quoted context omitted.

If the brain actually used language to represent ideas in use, we should have succeeded in finding its https://en.wikipedia.org/wiki/Universal_grammar . Instead it seems more like language is a kind of lossy compression we keep reinventing for ideas, to export them from a brain in some tangible way and get them into another (even in the future).

Hmm, I'm not sure if that holds? Just because you use a mechanism for expressing your ideas and reasoning, doesn't mean that underlying reality has to confirm in any way to it? We invent new symbols and terms all the time as we experience new phenomena. A universal grammar may still be possible, barring incompleteness anyway, we may just be lacking a whole bunch of ideas still...

@erik_seaberg the lack of success in finding a universal grammar is a logical leap. The failure to find something does not necessarily mean it doesn't exist. Language is powerful for expressing abstract ideas without explicitly saying them, which suggests language more than "lossy" compression because it's more similar to Shannon's lossless compression with prefix codes. I see where you're coming from though.

@Felcon If language is a mechanism for expressing ideas and reasoning, it should reflect our cognitive processes that generate those ideas, so " [...] Just because you use a mechanism for expressing your ideas and reasoning, doesn't mean that underlying reality has to confirm in any way to it" is a bit contradictory. Are our cognitive processes not included in reality?

The existance of a universal grammar is a specific hypothesis that requires empirical evidence. It's tiring to hear Chomsky's ideas parroted despite no empirical framework to stand on. What ideas could we be lacking? This argument is similar to String Theory proponents who kept pulling ideas out of the ether to support an unsubstantiated theory.

Re: Can language models serve as text-based world simulators?

#40

Post by catlifeonmars »

sk11001 wrote: post #17

Recently I'm getting some strange behavior from ChatGPT (with GPT-4) where it outputs a code snippet in which it declares a variable, and then on the very next line it forgets the name of the variable and refers to it as something else. Or it refers to the variable but it makes a typo in the name. If that's the behavior by one of the best models on a few lines of code, I'm not hopeful for a world simulator any time s…

Think about what the model is doing:

It’s sampling from a statistical distribution of likely tokens, one of which is the correctly spelled variable name.

Post reply on HN