Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Language Models Are Neurosymbolic Reasoners

arxiv.org

Post reply on HN
Print view View original

161–170 of 172 posts

Re: Large Language Models Are Neurosymbolic Reasoners

#161

Post by mjburgess »

calf wrote: post #143
mjburgess wrote: post #136

Earlier quoted context omitted.

> Reasoning is a cognitive process You could have a non-cognitive view of reasoning, or an embodied one. NNs are a-cognitive systems, they do not engage in reasoning of any form. Reasoning concerns inference across truth-apt propositions (eg., A->B, A thef. B). NNs have no propositions, nor are any parts truth-apt, true or false. NNs are statistical systems which select answers by weights found from optimisation. No…

To focus on one issue, the neural machine that is chosen by optimization is one that "best" fits the photos of the sky. But those multiple optima do not preclude a neural machine whose parameter values are computationally equivalent to, say, a 3D representation of the sky projected onto a 2D perspective -- a kind of partial world theory or world model, that was picked randomly out of many optima. First, it's not impo…

> whether neural nets are purely stochastic parrots.

Well we know how they work, it isnt speculative. All gradient-based algs on empirical outcome spaces are just kernel machines (ie., they weight their training data and take averages across it using a similarity metric).

Insofar as the ooutput seems as-if to reason it is because the input was produced by reasoning (of people). If you input text documents which have not been structured by reasoning agents, then the system doesnt work.

As for the idea of AI building generative 3D models and then projecting 2D -- yes, indeed that's how we did it.. but there are very large infinitities of 3D models all producing the same 2D.

This is where the "start from known outcome spaces" strategy of all existing AI fails. You cannot scan an infinity, or even sample meaningfully from it.

In otherwords the AI has to build such "deep models" circumstantially, it has to have a very limited set of them, and these have to 'somehow' be necessarily close to reality.

How do we do this? No mystery, we are in reality and so we in an ecological interplay with our enviorments. THe environment isnt, in cartesian terms, an evil daemon -- it doesnt lie, and doesnt tell the truth. What it does do is act reliably in reaction to us.

Via these means we explain.

Re: Large Language Models Are Neurosymbolic Reasoners

#162

Post by klyrs »

still_grokking wrote: post #147
klyrs wrote: post #125

Earlier quoted context omitted.

> Is it actually possible to beat Nethack without reading up some "spoilers" upfront? If people are playing nethack without reading the source, they're needlessly hobbling themselves.

That's my point. Imho it's impossible to even come close to a realistic chance to beat this game without knowing more or less every detail of its internal mechanics. And you can't infer this knowledge just from playing. You need to look it up from some out-of-band source. So only letting an AI play this game will not reveal anything about the "reasoning" capabilities of said AI. Letting the AI read the source (or som…

> And you can't infer this knowledge just from playing. You need to look it up from some out-of-band source.

I disagree with this in principle. Given enough time, you can collect enough statistics to infer pretty much anything about the game. I've participated in efforts to reverse-engineer web-only games where neither source nor binary were available -- the server does all the compute and the client only receives information to show the user.

But in practice: how long will it take to first inscribe elbereth, and then notice its effect? Yeah, probably not happening in a few of my lifetimes.

Re: Large Language Models Are Neurosymbolic Reasoners

#163

Post by naasking »

mjburgess wrote: post #158
naasking wrote: post #135

Earlier quoted context omitted.

> Its insane to suppose that somatosensory representation building, which requires organic neruoplasticity to be connected to the organically adaptive neuromotor system, etc. etc. etc. can just be instantiated in a bit of sand. Why? That's just an assertion, not an argument. All of those big words and sophisticated concepts you used all reduce to field interactions that we've mostly captured mathematically. Unless yo…

I'm just driven mad by how people thinking physical objects with specific properties is "magic", but reducing reality down to abstract platonic mathematical forms is "science". This is the opposite. Gold isnt lead. Lead isnt wood. Cells are not bits of metal. Bits of metal do not polymerise. Bits of sand do not form weak covalent bonds... It's kinda exhausting that this mathematical superstition is so prevalent in pe…

> Gold isnt lead. Lead isnt wood. Cells are not bits of metal. Bits of metal do not polymerise. Bits of sand do not form weak covalent bonds...

That's not the argument. Both gold and lead are aggregates of fields. They don't have the same macroscopic properties but they do have the same microscopic properties (or attoscopic if you want to nitpick).

> The idea that reality is essentially mathematical and not essentially physical is pythagorean magical thinking. Science says the opposite.

Science says no such thing, and nobody is saying that reality is not physical. Those who adopt a mathematical universe hypothesis, or the like, say that reality is physical but that the physical is a subset of the mathematical.

> Physics does not study "2". It studies there being earth and the sun and a force between them, summarised as "2 masses" etc.

Mathematics is the study of structure. Physics is studying the structure of reality. There is therefore an obvious and inescapable link between mathematics and physics that you are simply not going to refute by repeatedly asserting that mathematical structures have nothing to do with physics. Of course any structures that have a formal correspondence have important equivalences, because that's literally what formal correspondence means.

Re: Large Language Models Are Neurosymbolic Reasoners

#164

Post by naasking »

mjburgess wrote: post #157
naasking wrote: post #138

Earlier quoted context omitted.

> but QM requires the state be real-valued (having infinite information in the computability sense). The unobservable state, which is merely a physical model that may have little resemblance to reality. All observable states necessarily have finite precision and beyond 60-70 digits are effectively undefined due to the uncertainty principle, which is yet another reason why people suggest physics is effectively computa…

No one here disagrees that measurement produces finite information; that is obvious and a necessary --- if it wasnt, we would never be able to know anything. Knowing requires an "early termination". The issue is that there's no evidence this property of measurement is a property of reality, and all the methods, premises, etc. of physics attribute the opposite to reality. Here, it is absolutely necessary for QM to wor…

> Here, it is absolutely necessary for QM to work that the unmeasured state is real-valued

You're just doubling down on the premise that a theory founded on a formalism based on unbounded numbers requires unbounded numbers to work. Sure, but why is that necessarily reflective of reality? Why does that entail that no other formalism that doesn't embed infinities / continuity is also not possible? I simply see no reason to accept your conclusion. The infinities you see as essential could very well simply be artifacts of our formalisms.

In fact, I'd conjecture that our continuous formalisms are at the very heart of some core problems in physics [1], and that at least some of those problems can be resolved by exploring more discrete formalisms. I suppose we'll see.

[1] https://arxiv.org/abs/1609.01421

Re: Large Language Models Are Neurosymbolic Reasoners

#165

Post by bubblyworld »

still_grokking wrote: post #146
bubblyworld wrote: post #89

Earlier quoted context omitted.

Fair enough - the mechanics are certainly random in the sense that they involve dice rolls. > So even the best players will fail almost 50% of the time which is almost as random as tossing a coin, and strictly not skill based Even moderately experienced players will fail close to 100% of the time. So getting to an almost 50% success rate, to my mind, shows a great deal of skill! The difference between this and a coin…

> the mechanics are certainly random in the sense that they involve dice rolls. Sorry, but that's still not what I've meant. Dice rolls are randomness in the usual meaning of this word. But "random" can also mean "arbitrary" and/or "illogical" things. Imho Nethack mechanics are also random in this sense. They don't make sense at all… :-) > Even moderately experienced players will fail close to 100% of the time. So ge…

> Do you have some links regarding this? I thought Poker is still one of the games where AIs don't play better than humans. OK, maybe it depends on the Poker variant. There are simpler and more difficult ones

Texas Hold'em, one of the most popular variants - have a look at deep mind's Player Of Games, and the general technique of Counterfactual Regret Minimisation. Both are recent advances, but poker is absolutely solved at a human professional level now.

I think one thing to keep in mind is that what you find illogical has very little bearing on whether a neural net can learn to do it. Us humans come prebaked with specific priors in our brain (like cognitive biases) that AIs don't necessarily share. I'd be careful making sweeping statements about what is impossible and what isn't, personally. But I guess we'll see =)

> Sure. But how do you learn from a distribution where no matter what you do you will fail in, say, 99,9% of the cases?

You can actually test this yourself if you're interested - try to train a neural net to predict outcomes that are deterministic but with a 99.9% chance of random failure. If the net learns to succeed 0.1% of the time then your premise is false - it has successfully extracted the signal!

Re: Large Language Models Are Neurosymbolic Reasoners

#166

Post by eru »

mewpmewp2 wrote: post #133
neurallambda wrote: post #127

Earlier quoted context omitted.

It can get this simple example right if it does chain of thought. If you ask it to just output the answer without answering other bullet points, it will very likely not get it right. Chain of thought is duct tape to actual reasoning, and the errors/hallucinations compound exponentially. Try to get chatgpt to reason about concurrent state issues in programming. If it's not a well worn issue that it already memorized,…

But as a human being I couldn't ever do it without chain of thoughts. First I would have to bruteforce come up with fruits that have the required colour. Otherwise I am just random guessing. Also this problem I first tried to solve myself and I couldn't because I imagined sky to be light blue and blueberries are very dark if blue at all to me.

You could also go with grey or white etc for the sky.

Re: Large Language Models Are Neurosymbolic Reasoners

#167

Post by bubblyworld »

mjburgess wrote: post #137
naasking wrote: post #134

Earlier quoted context omitted.

> The assertion that the world is computable is just that: there are no research projects, no textbooks, no experiments, no formalism to replace physics or anything like it -- nothing. All the basic assumptions of physics would have to be false, and we would have to have good reasons for supposing so. I have no idea what this means. Physics must be computable from straightforward physical arguments like the Bekenstei…

Thermodynamic entropy isnt information in the relevant sense. There's a wide class of computational mysticism born of people going around and equivocating between "information" as it means radically different things where it is used. thermodynamic entropy (a real number) != information theory entropy (bits) != information in csi != information in stat mech' != information in QM != information in a turing machien !=.…

I enjoyed the thread the other day, just want to point out there are some basic misunderstandings here - informational entropy and thermodynamic entropy in fact _do_ have a deep connection (Edwin Jaynes famously wrote about this). The key idea is that both are measures of the uncertainty of a system with respect to some distribution over it.

Saying that a real-valued state has infinite information in the computability sense is nonsense - information is a property of a _distribution_, not a _state_. You could talk about the Kolmogorov complexity of a real-valued state, but even this is generally not infinite, as anyone who's written a program to generate the digits of pi can attest.

Re: Large Language Models Are Neurosymbolic Reasoners

#168

Post by ranguna »

randysalami wrote: post #100
ranguna wrote: post #64

Earlier quoted context omitted.

This reminds me of a meme that goes something along these lines: Joins a university; Studies for their bachelor's degree; Gets their degree after 3-5+ years; Studies for their master's; Gets their master's after 2-4+ years; Studies for their PhD while working on their thesis for a few more years; Participares in intensive discussions with their peers, and investigates day and night; Sends their thesis for peer review…

Is the master’s really 2-4 years? I always thought it was closer to 1 or 2.

Maybe it depends on the master's. The ones I know are 2 years if you follow the standard plan, but I wouldn't be surprised if there were 1 year master's

Re: Large Language Models Are Neurosymbolic Reasoners

#169

Post by mjburgess »

naasking wrote: post #163
mjburgess wrote: post #158

Earlier quoted context omitted.

I'm just driven mad by how people thinking physical objects with specific properties is "magic", but reducing reality down to abstract platonic mathematical forms is "science". This is the opposite. Gold isnt lead. Lead isnt wood. Cells are not bits of metal. Bits of metal do not polymerise. Bits of sand do not form weak covalent bonds... It's kinda exhausting that this mathematical superstition is so prevalent in pe…

> Gold isnt lead. Lead isnt wood. Cells are not bits of metal. Bits of metal do not polymerise. Bits of sand do not form weak covalent bonds... That's not the argument. Both gold and lead are aggregates of fields. They don't have the same macroscopic properties but they do have the same microscopic properties (or attoscopic if you want to nitpick). > The idea that reality is essentially mathematical and not essential…

"Formal equivalence" does not have any causal significance.

> They don't have the same macroscopic properties

Exactly... see the above...

Whenever you taalk about "formal" equivalece you mean: ignoring the causal semantics for the formula describing the system.

"2 + 2 = 4" is eqv. to "2 + 2 = 4" even when talking about wars, people, cars, animals, ... sand, cells, brains, bodies...

NO, obvious not. This isnt hard, this is obvious.

It requires some extraordinary magical thinking to suppose that field configurations of sand are equivalent to arbitary systems... this is obvious nonesense

Re: Large Language Models Are Neurosymbolic Reasoners

#170

Post by FrustratedMonky »

mjburgess wrote: post #160
FrustratedMonky wrote: post #155

Earlier quoted context omitted.

sidebar "If reasoning is an abstract pattern, then rice falling to the ground is likewise "reasoning"." Technically, I think some philosophers over the ages have made that point, and argued that 'rice falling' is reasoning, and the rice is 'wanting' to be closer to the 'earth' or some such thing. It does sound wacky. Think Leibniz and monads made some argument like that. And the various takes on 'Will to Power', Scho…

schopenhaur's idealism is indeed very similar, as is all idealism to this computational mysticism. What none in this tradition considered possible is that the world exists; each reduced it down to a purely formal pattern one way or another. Thankfully today we treat mental illness, and derealisation and depersonalisation and raised to this status. I operate in a framework where there's a world and we're in it, and it…

"Before one studies Zen, mountains are mountains and waters are waters;

After one gains insight through the teachings of a master, mountains are no longer mountains and waters are no longer waters;

After enlightenment, mountains are once again mountains and waters are waters."

------

or another i like

"Before enlightenment; chop wood, carry water. After enlightenment; chop wood, carry water.”

---------

Yes. I do get where you are coming from.

guess after doing a lot of meditating on 'no-self'. I started picking apart my own mind, and realizing it is all just electrical sparks and chemicals.

But then my engineer mind jumped in and said, of course we can model this. So then I get into arguments on the internet about how, of course we can model this.

Lets say we both agree on the real world existing. I think with us living in this real world. We are just arguing over the degree to which we'll be able to model a human. Of course, a model being an approximation. And I lean pretty far in the direction that once we model 'us' sufficiently to be indistinguishable from real 'us', then it will have also generated some subjective experience. (I firmly believed this until just recently after reading 'blindsight', i'm having doubts).

I think the Idealist from back in Schopenhauer day, were doing their best to describe the real world. Sometimes it sounds mystical, but that was before (or during) scientific revolution. The terminology is all different, and they didn't know a lot we take for granted. I wouldn't say they were mystics because they don't have todays knowledge.

The whole 'noumenal' world versus 'Phenomenal' world is valuable 'concept'. Our senses and mind only form an internal 'model' of the real world, it isn't the real world. But we can agree water is wet. But by how much. There are lot of studies about how different people perceive objects as moving faster/slower, etc... based on fear or anxiety. Our inner 'perception' isn't 'accurate', it isn't the actual real world. Just like a computers wouldn't be.

Post reply on HN