Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

From word models to world models

arxiv.org

Post reply on HN
Print view View original

81–90 of 119 posts

Re: From word models to world models

#81

Post by novaRom »

agnosticmantis wrote: post #43
skepticATX wrote: post #32

Earlier quoted context omitted.

I have not yet read the paper, but based on this description it seems like it provides grounding in the context of the training data , which is kind of the rub with current LLMs to begin with, right? We don't have a set of high quality training data that is completely unbiased and factual.

> … which is kind of the rub with current LLMs to begin with, right? No, the bigger problem with current LLMs is that even with high quality factual training data, they often generate seemingly plausible nonsense (e.g. cite nonexistent websites/papers as their sources.) This is by design imo; they’re trained to generate ‘likely’ text, and they do that extremely well. There’s no guarantee for faithful retrieval from a…

Important addition to your partially right statement: "they’re trained to generate ‘likely’ text" is they are trained to produce most probable next word so that the current context look as "similar" to training data as possible. Where "similar" is not "equal".

Re: From word models to world models

#82

Post by lostmsu »

mjburgess wrote: post #77
lostmsu wrote: post #70

Earlier quoted context omitted.

> I think this is easy, just make Xp sentences of the kind = "I define `randomchars()` to be this `term-in-Xc()`" and `randomchars()` does not match your own requirement `but not that the tokens of Xp are themselves rare` and therefore is unsuitable.

good point --- so replace it with a `sample()` fn that selects from an appropriate distribution over the data

Now you have a strong statistical dependency between Xc and Xp the lack of which was required for your proof to show that the algorithm is unable to learn Xp. BTW it was already there because you already had `term-in-Xc()`.

Re: From word models to world models

#83

Post by [deleted] »

mjburgess wrote: post #22
redox99 wrote: post #10

Earlier quoted context omitted.

If it's "absolutely trivial" to show that LLMs don't have the capacity to form thought, then please publish a paper proving that. So all the "stupid" people studying LLMs that can't come up with such trivial proofs can move on to other stuff.

You may wish to read the paper above. But if you want a quick proof: 1. A thought is a representation of a situation 2. A representation generates entailments of that situation 3. Language is many-to-one translation from these representations to symbols 4. Understanding language is reversing these symbols into thoughts (ie., reprs) So, 5. If agent A understands sentence X then A forms the relevant representation of X…

[deleted]

Re: From word models to world models

#84

Post by antiquark »

valine wrote: post #78
antiquark wrote: post #5

I doubt that word models can lead to world models. To quote Yann LeCun: "The vast majority of our knowledge, skills, and thoughts are not verbalizable. That's one reason machines will never acquire common sense solely by reading text." https://twitter.com/ylecun/status/1368235803147649028

What concepts exactly can’t be verbalized? All of our serialized file formats fall under the umbrella of “words”. GPT4 can draw images by outputting SVGs for example.

> What concepts exactly can’t be verbalized?

I would like to explain, but I can't quite put it into words...

:)

Re: From word models to world models

#85

Post by valine »

antiquark wrote: post #84
valine wrote: post #78

Earlier quoted context omitted.

What concepts exactly can’t be verbalized? All of our serialized file formats fall under the umbrella of “words”. GPT4 can draw images by outputting SVGs for example.

> What concepts exactly can’t be verbalized? I would like to explain, but I can't quite put it into words... :)

Trying drawing a picture of it and save it as an SVG.

Re: From word models to world models

#86

Post by andsoitis »

cs702 wrote: post #2

After a quick/superficial read, my understanding is that the authors: (a) induce an LLM to take natural language inputs and generate statements in a probabilistic programming language that formally models concepts, objects, actions, etc. in a symbolic world model, drawing from a large body of research on symbolic AI that goes back to pre-deep-learning days; and (b) perform inference using the generated formal stateme…

Humans’ experience and understanding of the world around them isn’t limited to a symbolic representation.

It remains to be seen whether you can truly be an effective intelligence with understanding of the world if all you have are symbols that you have to manipulate.

Re: From word models to world models

#87

Post by canjobear »

gibsonf1 wrote: post #31

Unfortunately, this effort fully misses the boat. Human cognition is about concepts, not language, and that's where one must start to understand it. Language simply serializes our conceptual thinking in multiple language formats, the key is what's being serialized and how that actually works in conceptual awareness.

Read more carefully. Their "language of thought" is not a natural language, it's a variant of lambda calculus with probabilistic semantics.

Re: From word models to world models

#88

Post by canjobear »

mjburgess wrote: post #3

It's a surprise to see a paper actually try to solve the problem of modelling thought via language. Nevertheless, it begins with far too many hedges: > By scaling to even larger datasets and neural networks, LLMs appeared to learn not only the structure of language, but capacities for some kinds of thinking There's two hypotheses for how LLMs generate apparently "thought-expressing" outputs: Hyp1 -- it's sampling fro…

> it's sampling from similar text which is distributed so-as-to-express a thought by some agent;

Your hypotheses 1 and 2 are not so different when you consider that the similarity function used to match text in the training data must be highly nontrivial. If it were not, then things like GPT-3 would have been possible a long time ago. As a concrete example, LLMs can do decent reasoning entirely in rot13; the relevant rot13'ed text is likely very rare in their training data. The fact that the similarity function can "see through" rot13 means that it can in principle include nontrivial computations.

Re: From word models to world models

#89

Post by gibsonf1 »

canjobear wrote: post #87
gibsonf1 wrote: post #31

Unfortunately, this effort fully misses the boat. Human cognition is about concepts, not language, and that's where one must start to understand it. Language simply serializes our conceptual thinking in multiple language formats, the key is what's being serialized and how that actually works in conceptual awareness.

Read more carefully. Their "language of thought" is not a natural language, it's a variant of lambda calculus with probabilistic semantics.

Right, derived from word pattern statistics. The CYC project tried first order predicate calculus with complete failure. This is not how we think or how conceptual awareness works. The key give away is what they don't talk about, Concepts.

Re: From word models to world models

#90

Post by drdeca »

gibsonf1 wrote: post #89
canjobear wrote: post #87

Earlier quoted context omitted.

Read more carefully. Their "language of thought" is not a natural language, it's a variant of lambda calculus with probabilistic semantics.

Right, derived from word pattern statistics. The CYC project tried first order predicate calculus with complete failure. This is not how we think or how conceptual awareness works. The key give away is what they don't talk about, Concepts.

What's a concept?
Post reply on HN