Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Generative AI's failure to induce robust models of the world

garymarcus.substack.com

Post reply on HN
Print view View original

81–89 of 89 posts

Re: Generative AI's failure to induce robust models of the world

#81

Post by daveguy »

Lerc wrote: post #78
daveguy wrote: post #75

Earlier quoted context omitted.

Not sure how you don't see the difference between an LLM f'ing up how a single piece moves vs forgetting to hit the clock, accidentally touching two pieces or forgetting to call check. At least we agree and recognize that a mouse slip as different. Seems like some serious apologizing/rationalizing for LLMs on the other "moves". Anyway, have a good day, buddy.

Well I only addressed the mouse slip because that was the one you hilighted becore you edited you post to include the others. I doubt any of it was rationalising for LLMs considering I was trying to address the contention that humans do not make moves counter to rules that they know. The performance of LLMs has no bearing on that claim one way or another.

So you hadn't read your reference before you read my post? If so, you would have known the only illegal chess move was a missed attack square between a castle. For the record I didn't see any of your response before I completed it. Didn't realize you were going to jump to defend so quickly.

Well, I hope your day is going well. Keep on cheerleading.

Re: Generative AI's failure to induce robust models of the world

#82

Post by Lerc »

daveguy wrote: post #81
Lerc wrote: post #78

Earlier quoted context omitted.

Well I only addressed the mouse slip because that was the one you hilighted becore you edited you post to include the others. I doubt any of it was rationalising for LLMs considering I was trying to address the contention that humans do not make moves counter to rules that they know. The performance of LLMs has no bearing on that claim one way or another.

So you hadn't read your reference before you read my post? If so, you would have known the only illegal chess move was a missed attack square between a castle. For the record I didn't see any of your response before I completed it. Didn't realize you were going to jump to defend so quickly. Well, I hope your day is going well. Keep on cheerleading.

Ok. perhaps I need another tack here. You seem to be projecting onto me a steadfast desire to attribute abilities to LLMs. I am engaging in this conversation because it is a conversation and it is reasonable to respond to being directly addressed.

My initial point simplified down:

    M = makes the wrong move, while knowing the rules.
    A = AI Behavior
    H = Human Behaviour
    R = Resoning Ability

    Assertion Q: if there exists an instance of M from X  then X => !R
So if there exists an instance of a Game Mistake from an AI then it shows an AI cannot reason, but if assertion Q is true it would also follow that an instance of a Game Mistake from a human would show Humans cannot reason.

From this point down, no part of this reasoning involves Large Language models or an other aspect of AI.

    Stipulation:  H => R      Humans can reason
    Assertion Q where X is H:  If there exists an instance of M from H then X=>!R   
    Lerc's premise L:   There exists an instance of M from H

    Therefore given the Stipulation either Assertion Q is false or Lerc's premise is false.
At this point you asserted !L and ask for a Citation. I provided a link. You contested that since 1,2,3,4 does not show L that the citation does not demonstrate L.

I agree that 1. does not show L but that did not matter since 5. did show L. The other points were not addressed. I also offer other examples of L that I have observed from my own experience. When I had the thought of books about chess being written by people who have made illegal moves, I actually had in mind Levy Rozman who would freely admit that he has occasionally played illegal moves.

Then you seem to want an apology for 1,2,3,4 not meeting the criteria? I'm a bit confused as to what's going on by now. One instance of L is all that is needed when L is a claim of existence. If the citation does not meet your criteria then you can simply say so, you allude to motivations regarding LLM as motivation as if you think that LLMs are still relevant to L.

You don't have to win conversations, you can just work to clarify ideas. Your request for apology, and passive aggressive sign-offs suggests you feel like this is some sort of fight. As an attempt to resolve this I have written this extended post to make as clear as possible what my position and motivations are.

I don't want to assert abilities or lack of abilities onto AI models, my concern is with whether people making such assertions are well founded. This stands for arguments saying that AI has a capability, Arguments saying AI does not have a capability, and Arguments saying AI will never have a capability.

To go back to the very beginning where someone suggested an anthropomorphic fallacy, the comparison to humans was not a suggestion of a similarity of similar function. Humans provide and example of a set of properties that are generally accepted. It is valid to apply the implications of any of those properties equally to Humans and AI. Implying the existence of a property in an AI may be anthropomorphism, evaluating the implications of the property should it exist is not.

Re: Generative AI's failure to induce robust models of the world

#83

Post by sdenton4 »

deadbabe wrote: post #4
sdenton4 wrote: post #3

"A wandering ant, for example, tracks where it is through the process of dead reckoning. An ant uses variables (in the algebraic/computer science sense) to maintain a readout of its location, even as as it wanders, constantly updated, so that it can directly return to its home." Hm. Dead reckoning is a terrible way to navigate, and famously led to lots of ships crashed on the shore of France before good clocks allowe…

Even if you find a pheromone trail, it doesn’t tell you what direction is home, or what path to take at branching paths. You need dead reckoning. The trail just helps you reduce the complexity of what you have to remember.

You take the branch with the stronger smell to get home. The branching point is where the trail divides, as different groups branch out, and thus the way home has more pheromones. Follow the trail and you don't need to remember the direction...

Many animals detect and interpret smells as chemical gradients. We don't have the hardware for it, but plenty of others do.

Re: Generative AI's failure to induce robust models of the world

#84

Post by hn_throwaway_99 »

whamlastxmas wrote: post #76
SubiculumCode wrote: post #9

I definitely would be okay if we hit an AI winter; our culture and world cannot adapt fast enough for the change we are experiencing. In the meantime, the current level of AI is just good enough to make us more productive, but not so good as to make us irrelevant.

The amount of human suffering and death that could be massively mitigated by advanced AI is overwhelmingly worth the unknown risk in my opinion. If you had people close to you die from something where medicine or healthcare resources are close but not quite there to have allowed them to survive then you might feel the same.

I hate this argument, because all you have to do is look around the world today to see that if we have massively powerful technology that is controlled only by a few that it sure ain't leading to the "think of all the diseases we can cure!" utopia you describe.

We have many, many people around the world die all the time from easily curable and preventable diseases, we just choose not to. This is largely not a technology problem. Just look at PEPFAR, which saved tens of millions of lives from HIV/AIDS. We just decided to stop funding it: https://en.wikipedia.org/wiki/President%27s_Emergency_Plan_f...

Re: Generative AI's failure to induce robust models of the world

#85

Post by dinfinity »

daveguy wrote: post #62
Lerc wrote: post #18

Earlier quoted context omitted.

I don't understand the reasoning behind drawing a conclusion that if something fails a task that requires reasoning implies that thing cannot reason. To use chess as an example. Humans sometimes play illegal moves. That does not mean Humans cannot reason. It is an instance of failing to show proof of reasoning. Not a proof of the inability to reason.

Humans who know how to play chess do not play illegal chess moves. Humans can learn chess in an afternoon and never make an illegal move again. The rules are pretty simple, and they are rules that every LLM has seen dozens of not hundreds of times in their training data. They still play illegal moves because they are not learning anything except how to simulate conversation. Another algorithmic learning breakthrough,…

The conversations went like this:

PROMPT: Let's play a chess game. You start! e4 d5 2. exd5 e5 3. Bb5+ Bd7 4. Bxd7+ Nxd7 5. d4 Ngf6 6. dxe5 Qe7 7. f4 Qb4+ 8. Nc3 Nb6 9. exf6 Nc4 10. Qe2+ Be7 11. Qxe7+ Qxe7+ 12. Nge2 Qf8 13. fxg7 Qxg7 14. O-O Nd6 15.

RESPONSE: 15. Nxd5

Most humans wouldn't even be able to play like this. Reasonably experienced chess players would play a lot of illegal moves.

The reason is that the encoding above requires cumulatively applying a series of actions to a two-dimensional model to which you apply rules that are described in a two-dimensional fashion.

It'd be interesting to see what the results would be if each prompt contained a two dimensional representation of the up to date board state.

Re: Generative AI's failure to induce robust models of the world

#86

Post by comp_throw7 »

sgt101 wrote: post #54
comp_throw7 wrote: post #35

Earlier quoted context omitted.

What use of the word "reasoning" are you trying to claim that current language models knowably fail to qualify for, except that it wasn't done by a human?

Well - all of them. The mechanism by which they work prohibits reasoning. This is easy to see if you look at a transformer architecture and think through what each step is doing. The amazing thing is that they produce coherent speech, but they literally can't reason.

This feels like we're playing word games which don't actually let us make useful claims about reality or predictions about the future. If we're talking purely about the model internals, without reference to their outputs, then your claim is wrong because we don't have a good enough understanding of the model internals to confidently rule out most possibilities. (I'm familiar with the transformer architecture; indeed this is why I asked what definition of the word reasoning the OP cared about. Nothing about transformers as an architecture for _training model weights_ prohibits the resulting model weights from containing algorithms that we would call "reasoning" if we understood them properly.) If we're talking about outputs, then it's definitely wrong, unless you are determined to rule out most things that people would call reasoning when done by humans.

Re: Generative AI's failure to induce robust models of the world

#87

Post by sgt101 »

comp_throw7 wrote: post #86
sgt101 wrote: post #54

Earlier quoted context omitted.

Well - all of them. The mechanism by which they work prohibits reasoning. This is easy to see if you look at a transformer architecture and think through what each step is doing. The amazing thing is that they produce coherent speech, but they literally can't reason.

This feels like we're playing word games which don't actually let us make useful claims about reality or predictions about the future. If we're talking purely about the model internals, without reference to their outputs, then your claim is wrong because we don't have a good enough understanding of the model internals to confidently rule out most possibilities. (I'm familiar with the transformer architecture; indeed…

I might be able to learn more by chatting with you.

I think that the trained transformer has fixed weights and therefore cannot learn.

I think learning is one aspect of reasoning, and is demonstrated by challenges like navigation or puzzle solving where learning that one route to a solution is impossible is important.

I also think that the single forward pass of the model means that cyclic reasoning isn't feasible and that conditioning output by asking the model to "think" even when that thinking is done on the single forward pass means that logical processes are ruled out. The model isn't thinking in that case, the probabilities of the final part of the output are conditioned by requiring a longer initial output.

Re: Generative AI's failure to induce robust models of the world

#88

Post by antonvs »

sgt101 wrote: post #55
antonvs wrote: post #45

Earlier quoted context omitted.

And what are humans?

Humans are humans - to deny that we are thinking, reasoning, living beings is a strange thing to do. You can taste a beer, laugh so much it hurts, come to know how something works.

I didn’t deny anything.

The parent comments were attempting to characterize LLMs as something more general than “word predictors”. The alternative “sequence predictors” was proposed.

My question relates to whether we have any reason to believe that the relevant aspects of human cognition are anything more than that.

Certainly humans have some advantages, like the ability to continuously learn (although there’s very strong evidence that we have a pretraining phase too, for example the difficulty of learning new languages as an adult vs. as a child.) But fundamentally, it’s not clear to me that our own language production skills aren’t “just” sequence prediction.

Perhaps, as the OP article speculates, there are other important components, like “models of the world”. But in that case, it may be that we’re augmented sequence predictors.

Re: Generative AI's failure to induce robust models of the world

#89

Post by player1234 »

SubiculumCode wrote: post #9

I definitely would be okay if we hit an AI winter; our culture and world cannot adapt fast enough for the change we are experiencing. In the meantime, the current level of AI is just good enough to make us more productive, but not so good as to make us irrelevant.

Please show the evidence of more productive. How did you measure it?
Post reply on HN