Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Language Models Are Neurosymbolic Reasoners

arxiv.org

Post reply on HN
Print view View original

81–90 of 172 posts

Re: Large Language Models Are Neurosymbolic Reasoners

#81

Post by still_grokking »

bubblyworld wrote: post #75
still_grokking wrote: post #70

Earlier quoted context omitted.

Of course one can beat Nethack. Many people did. That's not the point. The question was more: Without "spoilers" (a.k.a. "hardcoded heuristics and goals")? (I never made progress because I didn't try hard enough. It became very boring after finding out that this game is random and quite nonsensical, and one can't come up with some strategy only by playing it often enough. I'm in general not enjoying dice games. I pre…

Relax, friend, I understand your question =) I'm just pointing out that it's not as random as you think. Mechanics in NetHack are quite predictable for the most part, but they are difficult to _discover_ without dying. Given that AIs can play hundreds of thousands of runs in the time it would take me to play one, I'm a bit more optimistic that they could learn the mechanics eventually. I think the fundamental problem…

I think my wording was bad. There are two kinds of "randomness" here at play and I didn't differentiate properly.

For me the game mechanics as such are "random". Because you can't discover them by just playing (imho).

At the same time the game is ruled by a dice. (So even the best players will fail almost 50% of the time which is almost as random as tossing a coin, and strictly not skill based).

> Given that AIs can play hundreds of thousands of runs in the time it would take me to play one, I'm a bit more optimistic that they could learn the mechanics eventually.

My gut feeling says the opposite. How do you infer any kind of rules from almost random events? Especially if the "logic" behind the "non random" parts is actually also quite made up and arbitrary (so in a sense also "random", even to people with reason).

An "exploration-based reward function" wouldn't be enough. Because this would assume that exploration has (more or less) deterministic outcomes. But given the dice in Nethack it actually does not! You can do "everything right" and still die in almost 50% of the cases. How to infer any meaningful "world model" from such events? Imho you can't.

(I can confirm that looking up spoilers will let you make progress in Nethack. That's why I think it's boring. I've tried hundred of times prior to looking up spoilers and didn't make any progress. But after biting the bullet and starting reading some walkthough it was actually quite easy to reach some deeper levels. Until I've hit the next invisible wall. Which would require again some out-of-band knowledge… I know that reading the next spoilers would also make this wall go away. But I've lost any interest in this game after finding out exactly this: It's impossible to play without a walkthough; and with a full walkthough it's actually considerably easy, and comes down to "just having luck". At this point I could just toss a coin to determine whether "I won". That's maximally boring. I don't like dice games; and the exploration part in Nethack leads nowhere because the world is arbitrarily made up. You can't discover the mechanics without already knowing them…)

Re: Large Language Models Are Neurosymbolic Reasoners

#82

Post by Legend2440 »

still_grokking wrote: post #60
Legend2440 wrote: post #33

Earlier quoted context omitted.

Here's a counterexample. Suppose I create a simple neural network that computes f(x) = x^2 + c (where x and c are complex numbers) and then I run it as an RNN. This RNN will compute the mandelbrot set, which can't be represented by a lookup table. You can't even know if the RNN will halt for a given input. Neural networks are stronger than lookup tables, they are programs.

Every computable function can be represented by a (possibly infinite)¹ lookup table. Computer programs can only compute computable functions. Therefore any computer program is (in theory) equivalent to a table lookup . ¹ For finite inputs, the lookup table can be finite, and for infinite inputs, the lookup table can be infinite but still countable, as the set of computable functions is countable.

This table is not computable. If you had this table, you could solve the halting problem by simply looking up whether the program produced an output.

Re: Large Language Models Are Neurosymbolic Reasoners

#83

Post by codetrotter »

still_grokking wrote: post #54
bubblyworld wrote: post #6

The authors get LLMs to perform pretty well in a variety of IF-style text based games. Which is pretty cool, these kinds of games are played and read in natural language, which makes them pretty hard to write AIs for normally. Something I'd love to see one day is modern AI applied to other kinds of text based games like nethack. Last I checked nobody had managed to solve the problem of nethack AI without using hard c…

Is it actually possible to beat Nethack without reading up some "spoilers" upfront? I've never heard of anybody who managed to do that. Even when you read up all kinds of info about the game before you attempt a run it's extremely hard to reach higher levels, yet beat the game. (I myself never reached any later levels despite I know some tricks by now. Tricks impossible to infer from just playing the game; you need t…

> Imho there is no winning strategy for Nethack. It's some random stuff "you need to know" to progress even a little bit paired with complete rule of the dice while encountering maximally nonsensical "puzzles".

In the training data for the LLMs there is probably a significant of information about Nethack already, I would think.

So with the right prompt perhaps it could play better than some people, if those people did not have any info about nethack and had not played it before.

Re: Large Language Models Are Neurosymbolic Reasoners

#84

Post by mdekkers »

still_grokking wrote: post #77
neurallambda wrote: post #65

I'm trying to tackle this problem more head-on, by outfitting LLMs with lambda calculus, stacks, queues, etc. directly in their internals, operating over their latent space. [1] I'll read your paper, but, LLMs famously fail horribly at "multi jump" reasoning, which to me means they can't reason at all. They can merely output a reflection of the human reasoning that was baked into the training data, and they can also…

May I ask why there are NNs in your project at all? Just to heat up the planet and make Nvidia share holders even more happy? :-) I mean, what I've seen in the Readme makes sense. But doing basic computer stuff with NNs just makes the resource usage go brr by astronomical factors, for imho no reason, while making the results brittle, random, and often just completely made up. Also: Do you know about the (already 40 y…

+1 for cyc, genuinely awesome and overlooked

Re: Large Language Models Are Neurosymbolic Reasoners

#85

Post by kybernetikos »

bubblyworld wrote: post #6

The authors get LLMs to perform pretty well in a variety of IF-style text based games. Which is pretty cool, these kinds of games are played and read in natural language, which makes them pretty hard to write AIs for normally. Something I'd love to see one day is modern AI applied to other kinds of text based games like nethack. Last I checked nobody had managed to solve the problem of nethack AI without using hard c…

One of the things that llms are mainly not great at is understanding spatial relationship characters to each other. Get chat gpt to convert between different forms of chess notation and you'll start seeing errors very quickly. Ascii art is generally fairly bad unless it's regurgitation. It's not too surprising, but it will make games like nethack harder.

Re: Large Language Models Are Neurosymbolic Reasoners

#86

Post by bubblyworld »

mjburgess wrote: post #78
bubblyworld wrote: post #51

Earlier quoted context omitted.

No need, physicists already do this all the time - any computer simulation of quantum mechanical systems has to come to terms with the same problems (namely quantising the state space and representing the dynamics deterministically).

Physicists simulate on computers only what can be, which is almost nothing. Consider obtaining the dynamics of water by simulating all its parts: proton flow, hydrogen bonding etc. of 10^{PHYSICALLY UNCOMPUTABLE} interactions. The simulations which do exist fail to model vast amounts. This is why, say, climate change is given as a prediction on temperature -- because it can be obtained as a mean which ignores "basica…

To be honest, I don't really understand what you mean by Hilbert spaces being computable, and what that has to do with the linearity of QM, determinism of classical mechanics, universe being geometrical and not computational etc. I'm familiar with all of those concepts, but not sure how they tie together here. If you have resources you could share I would appreciate it (I had little success with google).

Re: Large Language Models Are Neurosymbolic Reasoners

#87

Post by somewhereoutth »

zeroonetwothree wrote: post #25
clooper wrote: post #21

Earlier quoted context omitted.

Then give some concrete numbers for the states of the atoms. My argument is not abstract, it is very concrete. Give me a neural network and I can generate the graph and prove the equivalence between the network and its graph representation as a table of tuples.

You said "even in theory" which is obviously wrong, since the (local) universe is finite and deterministic, hence it is itself a giant lookup table.

Simply not true. Of course it is comforting for computer people to believe the world they live in is a giant computer, but that is not our real reality.

Re: Large Language Models Are Neurosymbolic Reasoners

#88

Post by somewhereoutth »

ben_w wrote: post #58
Legend2440 wrote: post #53

Earlier quoted context omitted.

We can give them both infinite precision, you still can't build a lookup table of the mandelbrot set. The mandelbrot set is essentially a map of the halting behavior of a specific program. You can't know whether or not the program will halt for a given input, and so cannot build the lookup table. Programs are stronger than input-output mappings.

Infinity is really hard to reason about, are you sure about that? (For all I know you're a PhD in transfinites, your profile says nothing).

Infinity (of the various kinds) is well understood (see Cantor etc).

The Halting Problem is a central result in computer science, again well understood (especially here I would think!)

Their comment is correct.

Re: Large Language Models Are Neurosymbolic Reasoners

#89

Post by bubblyworld »

still_grokking wrote: post #81
bubblyworld wrote: post #75

Earlier quoted context omitted.

Relax, friend, I understand your question =) I'm just pointing out that it's not as random as you think. Mechanics in NetHack are quite predictable for the most part, but they are difficult to _discover_ without dying. Given that AIs can play hundreds of thousands of runs in the time it would take me to play one, I'm a bit more optimistic that they could learn the mechanics eventually. I think the fundamental problem…

I think my wording was bad. There are two kinds of "randomness" here at play and I didn't differentiate properly. For me the game mechanics as such are "random". Because you can't discover them by just playing (imho). At the same time the game is ruled by a dice. (So even the best players will fail almost 50% of the time which is almost as random as tossing a coin, and strictly not skill based). > Given that AIs can…

Fair enough - the mechanics are certainly random in the sense that they involve dice rolls.

> So even the best players will fail almost 50% of the time which is almost as random as tossing a coin, and strictly not skill based

Even moderately experienced players will fail close to 100% of the time. So getting to an almost 50% success rate, to my mind, shows a great deal of skill! The difference between this and a coin toss is that two people, no matter how many times they have each respectively tossed a coin, will _still_ always get a 50% success rate.

> An "exploration-based reward function" wouldn't be enough. Because this would assume that exploration has (more or less) deterministic outcomes.

I don't think this is true any more for modern AIs such as AlphaGo and its predecessors, which learn distributions of possible outcomes rather than deterministic predictions. IIRC the latest versions can even self-play games like Poker to a superhuman level.

I think so long as you are able to sample a given mechanic enough times, you can build a decent estimate of the possible outcomes (and choose your behaviour accordingly). If there is any systematic deviation from pure randomness, enough data will reveal it!

Re: Large Language Models Are Neurosymbolic Reasoners

#90

Post by hackerlight »

bubblyworld wrote: post #6

The authors get LLMs to perform pretty well in a variety of IF-style text based games. Which is pretty cool, these kinds of games are played and read in natural language, which makes them pretty hard to write AIs for normally. Something I'd love to see one day is modern AI applied to other kinds of text based games like nethack. Last I checked nobody had managed to solve the problem of nethack AI without using hard c…

CICERO, an AI for Diplomacy, uses a LLM: https://ai.meta.com/research/cicero/diplomacy/
Post reply on HN