Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Language Models Are Neurosymbolic Reasoners

arxiv.org

Post reply on HN
Print view View original

131–140 of 172 posts

Re: Large Language Models Are Neurosymbolic Reasoners

#131

Post by calf »

mjburgess wrote: post #129
FrustratedMonky wrote: post #126

Earlier quoted context omitted.

Everyone is captivated by the current hot thing, LLM/GPT's. But LLM's are not the whole of AI research. A lot of your arguments are based on 'embodied' reasoning. Humans live in the world, they need to eat and survive. LLM's just compress what humans generated in the world. Correct, current LLM's are mostly regurgitating, but they don't "speak because we are in a shared world of social intentions". I'd say game world…

It never has an "internal subjective experience" -- AlphaGO wasnt reasoning. Reasoning is a cognitive process in which propositions, which model the world, are considered in turn and subject to ecological rationality (concerns of utilty, effort, interest, preference, etc.). At no point in the flight of an aeroplane does it ever lay eggs. You are using smoke to establish fire, these ways of measuring internal mental s…

So I'm trying to understand your argument here, but why isn't "Reasoning is a cognitive process" circular logic?

AlphaGo wasn't reasoning, how? I think reasonably AlphaGo has modeled a world, and it is by design subject to the bounded rationality of a game-theoretic optimization problem. So two of your criteria are satisfied.

So - I'm just reading your definition here - AlphaGo wasn't reasoning because it is not a cognitive process. Is that your specific argument? And if so, then what is a cognitive process?

I'm just focusing on this part here and I don't see the argument clearly:

> It never has an "internal subjective experience" -- AlphaGO wasnt reasoning. > Reasoning is a cognitive process in which propositions, which model the world, are considered in turn and subject to ecological rationality (concerns of utilty, effort, interest, preference, etc.).

Re: Large Language Models Are Neurosymbolic Reasoners

#132

Post by [deleted] »

bubblyworld wrote: post #123
mjburgess wrote: post #121

Earlier quoted context omitted.

I think we teach people only what we can write in finite formula, and compute in finite time. This is imv, much like teaching people what's under a street light just because everything else is in darkness. I think, philosophically, we can build inferential telescopes that point to the vast (epistemic) blackness, inside say, a proton, or a cell, or the chaos in water. As an ameliorative, or therapeutic project, I thin…

Poetic. Perhaps you're right.

[deleted]

Re: Large Language Models Are Neurosymbolic Reasoners

#133

Post by mewpmewp2 »

neurallambda wrote: post #127
mewpmewp2 wrote: post #74

Earlier quoted context omitted.

For what it's worth, I tried it on ChatGPT and this was its response: "The color of the daytime sky is commonly blue. The common household fruit that is also blue would be blueberries. Blueberries typically grow in acidic soil. The pH of the soil they grow in is usually between 4.5 and 5.5."

It can get this simple example right if it does chain of thought. If you ask it to just output the answer without answering other bullet points, it will very likely not get it right. Chain of thought is duct tape to actual reasoning, and the errors/hallucinations compound exponentially. Try to get chatgpt to reason about concurrent state issues in programming. If it's not a well worn issue that it already memorized,…

But as a human being I couldn't ever do it without chain of thoughts. First I would have to bruteforce come up with fruits that have the required colour. Otherwise I am just random guessing.

Also this problem I first tried to solve myself and I couldn't because I imagined sky to be light blue and blueberries are very dark if blue at all to me.

Re: Large Language Models Are Neurosymbolic Reasoners

#134

Post by naasking »

mjburgess wrote: post #78
bubblyworld wrote: post #51

Earlier quoted context omitted.

No need, physicists already do this all the time - any computer simulation of quantum mechanical systems has to come to terms with the same problems (namely quantising the state space and representing the dynamics deterministically).

Physicists simulate on computers only what can be, which is almost nothing. Consider obtaining the dynamics of water by simulating all its parts: proton flow, hydrogen bonding etc. of 10^{PHYSICALLY UNCOMPUTABLE} interactions. The simulations which do exist fail to model vast amounts. This is why, say, climate change is given as a prediction on temperature -- because it can be obtained as a mean which ignores "basica…

> The assertion that the world is computable is just that: there are no research projects, no textbooks, no experiments, no formalism to replace physics or anything like it -- nothing. All the basic assumptions of physics would have to be false, and we would have to have good reasons for supposing so.

I have no idea what this means. Physics must be computable from straightforward physical arguments like the Bekenstein Bound: finite volumes must contain finite information. Any physical object has finite volume at any given time, the universe included, ergo they must contain finite information. Any system consisting of finite information can be modeled as a finite state machine.

Re: Large Language Models Are Neurosymbolic Reasoners

#135

Post by naasking »

mjburgess wrote: post #109
FrustratedMonky wrote: post #106

Earlier quoted context omitted.

My view is opposite. "inability to think outside of the switching frequencies of a silicon chip" Why can't you "think outside", and conceptualize that this 'internal subjective experience' we are having, is not unique, and could be taking place inside a silicon based NN? It helps to meditate and observe your own thoughts, and how they arise. You can begin to realize that you don't 'think'. You don't think about what…

Organisms are mechanistic, there's nothing "not mechanistic" about the mind. Its insane to suppose that somatosensory representation building, which requires organic neruoplasticity to be connected to the organically adaptive neuromotor system, etc. etc. etc. can just be instantiated in a bit of sand. This is deeply mystical, pseudoscience. You're credulously throwing away any kind of empirical analysis of the world…

> Its insane to suppose that somatosensory representation building, which requires organic neruoplasticity to be connected to the organically adaptive neuromotor system, etc. etc. etc. can just be instantiated in a bit of sand.

Why? That's just an assertion, not an argument. All of those big words and sophisticated concepts you used all reduce to field interactions that we've mostly captured mathematically. Unless you're imbuing these things with some magical essence that can't be observed, but then who's doing the pseudoscience?

Re: Large Language Models Are Neurosymbolic Reasoners

#136

Post by mjburgess »

calf wrote: post #131
mjburgess wrote: post #129

Earlier quoted context omitted.

It never has an "internal subjective experience" -- AlphaGO wasnt reasoning. Reasoning is a cognitive process in which propositions, which model the world, are considered in turn and subject to ecological rationality (concerns of utilty, effort, interest, preference, etc.). At no point in the flight of an aeroplane does it ever lay eggs. You are using smoke to establish fire, these ways of measuring internal mental s…

So I'm trying to understand your argument here, but why isn't "Reasoning is a cognitive process" circular logic? AlphaGo wasn't reasoning, how? I think reasonably AlphaGo has modeled a world, and it is by design subject to the bounded rationality of a game-theoretic optimization problem. So two of your criteria are satisfied. So - I'm just reading your definition here - AlphaGo wasn't reasoning because it is not a co…

> Reasoning is a cognitive process

You could have a non-cognitive view of reasoning, or an embodied one. NNs are a-cognitive systems, they do not engage in reasoning of any form.

Reasoning concerns inference across truth-apt propositions (eg., A->B, A thef. B). NNs have no propositions, nor are any parts truth-apt, true or false. NNs are statistical systems which select answers by weights found from optimisation. No process either in optimisation or prediction is an inferential one in the sense of cognition.

I also deny that the "world" as used by frankly just philosophically incompetent ML researchers, whose gross lack of familiarity with basically anything outside pytorch, is even relevant to the sense of "world" that propositions bare a truth relation to.

A world in the relevant sense isn't the state space of the training data -- this is an insane supposition which makes the claim "AI has world models" actually circular. The relevant sense of world is the cause of the training data. If the training data is about an abstract game then you collapse the distinction since the rules are the data.

The famous "NNs learn WMs" paper is just this: choose a system whose data is just a restatement of the system; rather than a measure of a world.

NNs do not form representational models of the cause of their measurement data because all they do is induce (ie., compress by function-fitting) across the measurement space. They model the measurements not their causes. This is only "predictive" of features of the causal origin of measurement data in rigged scenarios, and in general, fails catastrophically to be predictive.

Consider running a NN across photos of the sky: it is impossible for this process to produce newton's law of gravity. The weights are just models of the pixels, and these are not distributed according to this law.

Worse, in general, there is no function from the measurement space to properties of its causal origin -- so it is impossible to build representations by induction. (eg., there is no function Photo->Cat|Dog, the distributions of pixels in photos is ambiguous, and changes over time).

Reasoning, as in cognition, is an (logically) inferential process which considers propositions that bare a truth relation to the world which is the causal origin of concepts which the proposition comprises (created by a biogenerative process). It is the activity of an agent with an interior subjectivity and ecological rationality. Reasoning is done by an agent about something of interest to that agent, with motivation towards a goal the agent has, in the service of the agent's preferences, etc.

If reasoning is an abstract pattern, then rice falling to the ground is likewise "reasoning".

Re: Large Language Models Are Neurosymbolic Reasoners

#137

Post by mjburgess »

naasking wrote: post #134
mjburgess wrote: post #78

Earlier quoted context omitted.

Physicists simulate on computers only what can be, which is almost nothing. Consider obtaining the dynamics of water by simulating all its parts: proton flow, hydrogen bonding etc. of 10^{PHYSICALLY UNCOMPUTABLE} interactions. The simulations which do exist fail to model vast amounts. This is why, say, climate change is given as a prediction on temperature -- because it can be obtained as a mean which ignores "basica…

> The assertion that the world is computable is just that: there are no research projects, no textbooks, no experiments, no formalism to replace physics or anything like it -- nothing. All the basic assumptions of physics would have to be false, and we would have to have good reasons for supposing so. I have no idea what this means. Physics must be computable from straightforward physical arguments like the Bekenstei…

Thermodynamic entropy isnt information in the relevant sense.

There's a wide class of computational mysticism born of people going around and equivocating between "information" as it means radically different things where it is used.

thermodynamic entropy (a real number) != information theory entropy (bits) != information in csi != information in stat mech' != information in QM != information in a turing machien !=. ...

This is basically pseudoscience at this point. If you hear people talking about "information" as if its defined in a general sense (ie., equivocating across physics, computer science, etc.), they have no clue what they're talking about.

Eg., the "entropy" of real-valued quantum states as measured by integer-valued notions of entropy is 1 bit (the measured state is UP,DOWN) -- but QM requires the state be real-valued (having infinite information in the computability sense).

These kinds of information are not measuring the same thing, and largely irrelevant to each other.

Re: Large Language Models Are Neurosymbolic Reasoners

#138

Post by naasking »

mjburgess wrote: post #137
naasking wrote: post #134

Earlier quoted context omitted.

> The assertion that the world is computable is just that: there are no research projects, no textbooks, no experiments, no formalism to replace physics or anything like it -- nothing. All the basic assumptions of physics would have to be false, and we would have to have good reasons for supposing so. I have no idea what this means. Physics must be computable from straightforward physical arguments like the Bekenstei…

Thermodynamic entropy isnt information in the relevant sense. There's a wide class of computational mysticism born of people going around and equivocating between "information" as it means radically different things where it is used. thermodynamic entropy (a real number) != information theory entropy (bits) != information in csi != information in stat mech' != information in QM != information in a turing machien !=.…

> but QM requires the state be real-valued (having infinite information in the computability sense).

The unobservable state, which is merely a physical model that may have little resemblance to reality. All observable states necessarily have finite precision and beyond 60-70 digits are effectively undefined due to the uncertainty principle, which is yet another reason why people suggest physics is effectively computable.

While the types of information you mention are not strictly equivalent in some 1:1 sense that I don't think anyone has really suggested, there are formal correspondences, so your explanation ultimately just seems like a lot of special pleading, eg. you can derive a Bekenstein Bound for bits, thermodynamic entropy, information in QM, and so on.

Re: Large Language Models Are Neurosymbolic Reasoners

#139

Post by FrustratedMonky »

mjburgess wrote: post #136
calf wrote: post #131

Earlier quoted context omitted.

So I'm trying to understand your argument here, but why isn't "Reasoning is a cognitive process" circular logic? AlphaGo wasn't reasoning, how? I think reasonably AlphaGo has modeled a world, and it is by design subject to the bounded rationality of a game-theoretic optimization problem. So two of your criteria are satisfied. So - I'm just reading your definition here - AlphaGo wasn't reasoning because it is not a co…

> Reasoning is a cognitive process You could have a non-cognitive view of reasoning, or an embodied one. NNs are a-cognitive systems, they do not engage in reasoning of any form. Reasoning concerns inference across truth-apt propositions (eg., A->B, A thef. B). NNs have no propositions, nor are any parts truth-apt, true or false. NNs are statistical systems which select answers by weights found from optimisation. No…

This is kind of ignoring other NN's. You're very focused on LLM's as the example.

AlphaGo learned by playing itself. And is able to anticipate multiple moves ahead.

Then, that same 'engine', was able to be applied to Chess, and learned how to beat a master from scratch, by playing itself, in just a few hours.

There was no lookups, or zip'ing of aggregated data.

A lot of what you are postulating as cognition, humans don't do either. Humans didn't figure out gravity from photos of the sky (unless you mean tracing stars and planets, and then yes an NN probably can figure it out). Many humans go through their whole day without analyzing propositions and inferring reality and what to do next. Humans are similarly un-conscious.

A lot of AI news all the time. This link is from today. A little more in the theme of 'world building models', than this current post about LLM's.

https://news.ycombinator.com/item?id=39692387

There was another on also about an NN that could pass some international Geometry competition. It was based on propositions, and reasoning.

Re: Large Language Models Are Neurosymbolic Reasoners

#140

Post by calf »

bubblyworld wrote: post #119
mjburgess wrote: post #116

Earlier quoted context omitted.

1 yr medicine, 6 yr physics, 4 yr debating union, 20 yr c programming, 20 yr love of political and stand up comedy, 15 yr software eng, 10 yr data scientist, 15 yr python, 22 yr informal & formal philosophy, 8 yr data sci & software consult/coach to finance/defence/... and maybe soon, 4 yr PhD AI & HCI Of those, you may decide which is the most relevant to my writing style. The amount of theatrics and irony in a live…

Wild, well, I'm a lowly math PhD so that's where my interests lie =) I'm _not_ suggesting we replace R with Q. I'm suggesting that you bake in the desired accuracy of your computational approximation as an input . This is how Turing evades self-referential problems in his conception of computational reals, and also perhaps how you evade your criticisms with CM requiring infinite precision. Similarly - I think it's re…

I've read this thread exchange with interest, but what about the results that quantum computers are simulatable by classical computers? See David Deutsch 1985. This would reduce the issue of infinite Hilbert spaces to simulation using quantum computers, and in turn, Deutsch's result which says classical Turing machines can actually simulate quantum computers.
Post reply on HN