Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Language Models Are Neurosymbolic Reasoners

arxiv.org

Post reply on HN
Print view View original

61–70 of 172 posts

Re: Large Language Models Are Neurosymbolic Reasoners

#61

Post by bubblyworld »

still_grokking wrote: post #54
bubblyworld wrote: post #6

The authors get LLMs to perform pretty well in a variety of IF-style text based games. Which is pretty cool, these kinds of games are played and read in natural language, which makes them pretty hard to write AIs for normally. Something I'd love to see one day is modern AI applied to other kinds of text based games like nethack. Last I checked nobody had managed to solve the problem of nethack AI without using hard c…

Is it actually possible to beat Nethack without reading up some "spoilers" upfront? I've never heard of anybody who managed to do that. Even when you read up all kinds of info about the game before you attempt a run it's extremely hard to reach higher levels, yet beat the game. (I myself never reached any later levels despite I know some tricks by now. Tricks impossible to infer from just playing the game; you need t…

I'm not sure. I've beaten it a number of times, but only using tons of spoilers as you say. That said, there are players who consistently win almost every game, which is crazy (there's an online nethack server somewhere, can't remember the name offhand, but you can search player stats and some of them are insane).

Edit: here's one, a player with a 60% win rate, not as crazy as I initially thought but if you've ever played nethack... https://alt.org/nethack/player-stats.php?player=Stroller

Re: Large Language Models Are Neurosymbolic Reasoners

#62

Post by aruss »

ben_w wrote: post #57
aruss wrote: post #55

Earlier quoted context omitted.

Where are you going to get all that time and space to build a lookup table? Are you sure you're able to measure all state at enough precision to make an accurate table?

Doesn't matter given the original statement spawning this subthread was: > People can not be reduced to lookup tables even in theory

Then I would say your theoretical model is wrong or incomplete or makes for a circular argument (it's an assumption and not proven that finite matter evolving through time reduces to a lookup table).

Re: Large Language Models Are Neurosymbolic Reasoners

#63

Post by mewpmewp2 »

clooper wrote: post #35
mewpmewp2 wrote: post #30

Earlier quoted context omitted.

You could pick a very large number depending on a reasonable processing capability a human has, which represents all the significant physical interactions on a human body over a certain amount of time. Then take the output over a certain amount of time, being all movements of the body. If you wanted to focus on thoughts alone, you might want to skip few layers/systems, to give input directly to whatever causes though…

Ok so give me some concrete number.

5.1536672454... could be approximated strength for a nervous signal of some sort in a human body in some unit of measurement.

Re: Large Language Models Are Neurosymbolic Reasoners

#64

Post by ranguna »

sharts wrote: post #4

Not really

This reminds me of a meme that goes something along these lines:

Joins a university;

Studies for their bachelor's degree;

Gets their degree after 3-5+ years;

Studies for their master's;

Gets their master's after 2-4+ years;

Studies for their PhD while working on their thesis for a few more years;

Participares in intensive discussions with their peers, and investigates day and night;

Sends their thesis for peer review;

Reworks their thesis according to the review;

Finally publishes their thesis in an academic journal;

Someone on the internet, reads their thesis title: bulsh*t

--

Would you care to elaborate on your comment?

Re: Large Language Models Are Neurosymbolic Reasoners

#65

Post by neurallambda »

I'm trying to tackle this problem more head-on, by outfitting LLMs with lambda calculus, stacks, queues, etc. directly in their internals, operating over their latent space. [1]

I'll read your paper, but, LLMs famously fail horribly at "multi jump" reasoning, which to me means they can't reason at all. They can merely output a reflection of the human reasoning that was baked into the training data, and they can also recombine it combinatorially. Eager to see if you've solved this!

[1] https://github.com/neurallambda/neurallambda

Re: Large Language Models Are Neurosymbolic Reasoners

#66

Post by ben_w »

mistermann wrote: post #59
ben_w wrote: post #57

Earlier quoted context omitted.

Doesn't matter given the original statement spawning this subthread was: > People can not be reduced to lookup tables even in theory

What theories are you using to solve for: - consciousness? - the unknown? https://en.m.wikipedia.org/wiki/Necessity_and_sufficiency - the misunderstood? https://plato.stanford.edu/Entries/perception-problem/ The Science of the Gaps will do I suppose? Culture could do it though I think.

Can you rephrase that?

It currently reads like shifting goalposts, and I'd like to guess that was not your intention…

Re: Large Language Models Are Neurosymbolic Reasoners

#67

Post by Karellen »

zeroonetwothree wrote: post #25
clooper wrote: post #21

Earlier quoted context omitted.

Then give some concrete numbers for the states of the atoms. My argument is not abstract, it is very concrete. Give me a neural network and I can generate the graph and prove the equivalence between the network and its graph representation as a table of tuples.

You said "even in theory" which is obviously wrong, since the (local) universe is finite and deterministic, hence it is itself a giant lookup table.

> the (local) universe is finite and deterministic,

Radioactive decay and spontaneous pair production say otherwise on the deterministic front.

Re: Large Language Models Are Neurosymbolic Reasoners

#68

Post by eru »

neurallambda wrote: post #65

I'm trying to tackle this problem more head-on, by outfitting LLMs with lambda calculus, stacks, queues, etc. directly in their internals, operating over their latent space. [1] I'll read your paper, but, LLMs famously fail horribly at "multi jump" reasoning, which to me means they can't reason at all. They can merely output a reflection of the human reasoning that was baked into the training data, and they can also…

Humans also fail much of the time at 'multi jump' reasoning. You have to prod them.

Re: Large Language Models Are Neurosymbolic Reasoners

#69

Post by littlestymaar »

eru wrote: post #68
neurallambda wrote: post #65

I'm trying to tackle this problem more head-on, by outfitting LLMs with lambda calculus, stacks, queues, etc. directly in their internals, operating over their latent space. [1] I'll read your paper, but, LLMs famously fail horribly at "multi jump" reasoning, which to me means they can't reason at all. They can merely output a reflection of the human reasoning that was baked into the training data, and they can also…

Humans also fail much of the time at 'multi jump' reasoning. You have to prod them.

Except no human (non-colorblind at least) past three years old thinks bananas have the same color as the sky (see the example given in the repo, that's a mistake literally no human could make)

Re: Large Language Models Are Neurosymbolic Reasoners

#70

Post by still_grokking »

bubblyworld wrote: post #61
still_grokking wrote: post #54

Earlier quoted context omitted.

Is it actually possible to beat Nethack without reading up some "spoilers" upfront? I've never heard of anybody who managed to do that. Even when you read up all kinds of info about the game before you attempt a run it's extremely hard to reach higher levels, yet beat the game. (I myself never reached any later levels despite I know some tricks by now. Tricks impossible to infer from just playing the game; you need t…

I'm not sure. I've beaten it a number of times, but only using tons of spoilers as you say. That said, there are players who consistently win almost every game, which is crazy (there's an online nethack server somewhere, can't remember the name offhand, but you can search player stats and some of them are insane). Edit: here's one, a player with a 60% win rate, not as crazy as I initially thought but if you've ever p…

Of course one can beat Nethack. Many people did. That's not the point. The question was more: Without "spoilers" (a.k.a. "hardcoded heuristics and goals")?

(I never made progress because I didn't try hard enough. It became very boring after finding out that this game is random and quite nonsensical, and one can't come up with some strategy only by playing it often enough. I'm in general not enjoying dice games. I prefer games where you can come up with some winning strategy by curious observation and logical thinking.)

Because of the nature of Nethack I don't think it's a good AI test as such.

Maybe it would be if one let the AI read spoilers / walkthoughs and than let it try playing. Such a test could than maybe probe for the AI's text comprehension, and the ability to map the gained understanding to concrete actions. But just letting it play Nethack unprepared does not give any insides into the AI's capabilities, imho. It will just fail over and over again. Because it's (imho) impossible to beat Nethack without spoilers. You just can't extract the needed knowledge from playing. Even from playing it millions of times.

Post reply on HN