Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Language Models Are Neurosymbolic Reasoners

arxiv.org

Post reply on HN
Print view View original

51–60 of 172 posts

Re: Large Language Models Are Neurosymbolic Reasoners

#51

Post by bubblyworld »

imtringued wrote: post #49
bubblyworld wrote: post #37

Earlier quoted context omitted.

I strongly disagree with your last statement - physics explicitly _is_ formulated in terms that can be made into a lookup table (see phase spaces in classical mechanics, for instance). My point is that there's a finite light cone of possible causal influences over you at any moment in time, and in principle you can break those down into state variables finely enough to predict future states of a person. This is isomo…

How do you lookup quantum mechanics? Please tell the physicists about your breakthroughs.

No need, physicists already do this all the time - any computer simulation of quantum mechanical systems has to come to terms with the same problems (namely quantising the state space and representing the dynamics deterministically).

Re: Large Language Models Are Neurosymbolic Reasoners

#52

Post by thinkingemote »

lkrubner wrote: post #39

Possibly off-topic, but does anyone know where I can read up on LLMs? I've posted an "Ask HN" here, in the hopes some people can inform me about how I can keep up on what's new: https://news.ycombinator.com/item?id=39688911

Search box in the bottom can help look for introduction tutorials recommendations etc

Re: Large Language Models Are Neurosymbolic Reasoners

#53

Post by Legend2440 »

cjfd wrote: post #38
Legend2440 wrote: post #33

Earlier quoted context omitted.

Here's a counterexample. Suppose I create a simple neural network that computes f(x) = x^2 + c (where x and c are complex numbers) and then I run it as an RNN. This RNN will compute the mandelbrot set, which can't be represented by a lookup table. You can't even know if the RNN will halt for a given input. Neural networks are stronger than lookup tables, they are programs.

I am sorry to be this blunt but this is really utter and complete nonsense. The phrase that the mandelbrot set can't be represented in a lookup table is as such true but that is because nothing that you do with finite precision numbers can represent the mandelbrot set because it essentially is an inifinte object. The function f(x) = x^2 + c as an RNN can also not compute the mandelbrot set if the numbers it uses are…

We can give them both infinite precision, you still can't build a lookup table of the mandelbrot set.

The mandelbrot set is essentially a map of the halting behavior of a specific program. You can't know whether or not the program will halt for a given input, and so cannot build the lookup table. Programs are stronger than input-output mappings.

Re: Large Language Models Are Neurosymbolic Reasoners

#54

Post by still_grokking »

bubblyworld wrote: post #6

The authors get LLMs to perform pretty well in a variety of IF-style text based games. Which is pretty cool, these kinds of games are played and read in natural language, which makes them pretty hard to write AIs for normally. Something I'd love to see one day is modern AI applied to other kinds of text based games like nethack. Last I checked nobody had managed to solve the problem of nethack AI without using hard c…

Is it actually possible to beat Nethack without reading up some "spoilers" upfront? I've never heard of anybody who managed to do that.

Even when you read up all kinds of info about the game before you attempt a run it's extremely hard to reach higher levels, yet beat the game. (I myself never reached any later levels despite I know some tricks by now. Tricks impossible to infer from just playing the game; you need to read them up…)

Imho there is no winning strategy for Nethack. It's some random stuff "you need to know" to progress even a little bit paired with complete rule of the dice while encountering maximally nonsensical "puzzles".

But OK, maybe I'm just too dumb for this game and don't see the "logic" behind the things the game presents.

Re: Large Language Models Are Neurosymbolic Reasoners

#55

Post by aruss »

zeroonetwothree wrote: post #25
clooper wrote: post #21

Earlier quoted context omitted.

Then give some concrete numbers for the states of the atoms. My argument is not abstract, it is very concrete. Give me a neural network and I can generate the graph and prove the equivalence between the network and its graph representation as a table of tuples.

You said "even in theory" which is obviously wrong, since the (local) universe is finite and deterministic, hence it is itself a giant lookup table.

Where are you going to get all that time and space to build a lookup table? Are you sure you're able to measure all state at enough precision to make an accurate table?

Re: Large Language Models Are Neurosymbolic Reasoners

#56

Post by ben_w »

clooper wrote: post #21
Legend2440 wrote: post #17

Earlier quoted context omitted.

>People can not be reduced to lookup tables even in theory Sure you can. Simply enumerate all of the physical states that the atoms in your body could be in. Any finite-sized object has a finite number of possible states, and so can be represented by a finite lookup table. Your argument is so broad as to be meaningless.

Then give some concrete numbers for the states of the atoms. My argument is not abstract, it is very concrete. Give me a neural network and I can generate the graph and prove the equivalence between the network and its graph representation as a table of tuples.

"""the number of bits required to perfectly recreate the natural matter of the average-sized U.S. adult male human brain down to the quantum level on a computer is about 2.6×10^42 bits of information (see Bekenstein bound for the basis for this calculation).""" - https://en.wikipedia.org/wiki/Orders_of_magnitude_(data)

(That said, I think quantum physics makes it "all a Markov chain" rather than "all a lookup table").

Re: Large Language Models Are Neurosymbolic Reasoners

#57

Post by ben_w »

aruss wrote: post #55
zeroonetwothree wrote: post #25

Earlier quoted context omitted.

You said "even in theory" which is obviously wrong, since the (local) universe is finite and deterministic, hence it is itself a giant lookup table.

Where are you going to get all that time and space to build a lookup table? Are you sure you're able to measure all state at enough precision to make an accurate table?

Doesn't matter given the original statement spawning this subthread was:

> People can not be reduced to lookup tables even in theory

Re: Large Language Models Are Neurosymbolic Reasoners

#58

Post by ben_w »

Legend2440 wrote: post #53
cjfd wrote: post #38

Earlier quoted context omitted.

I am sorry to be this blunt but this is really utter and complete nonsense. The phrase that the mandelbrot set can't be represented in a lookup table is as such true but that is because nothing that you do with finite precision numbers can represent the mandelbrot set because it essentially is an inifinte object. The function f(x) = x^2 + c as an RNN can also not compute the mandelbrot set if the numbers it uses are…

We can give them both infinite precision, you still can't build a lookup table of the mandelbrot set. The mandelbrot set is essentially a map of the halting behavior of a specific program. You can't know whether or not the program will halt for a given input, and so cannot build the lookup table. Programs are stronger than input-output mappings.

Infinity is really hard to reason about, are you sure about that?

(For all I know you're a PhD in transfinites, your profile says nothing).

Re: Large Language Models Are Neurosymbolic Reasoners

#59

Post by mistermann »

ben_w wrote: post #57
aruss wrote: post #55

Earlier quoted context omitted.

Where are you going to get all that time and space to build a lookup table? Are you sure you're able to measure all state at enough precision to make an accurate table?

Doesn't matter given the original statement spawning this subthread was: > People can not be reduced to lookup tables even in theory

What theories are you using to solve for:

- consciousness?

- the unknown?

https://en.m.wikipedia.org/wiki/Necessity_and_sufficiency

- the misunderstood?

https://plato.stanford.edu/Entries/perception-problem/

The Science of the Gaps will do I suppose?

Culture could do it though I think.

Re: Large Language Models Are Neurosymbolic Reasoners

#60

Post by still_grokking »

Legend2440 wrote: post #33
clooper wrote: post #5

I was recently thinking how every neural network is equivalent to a lookup table where the input is all numbers up to what can be expressed within the context window and the output is the result of the arithmetic operations applied to that number. So every neural network is equivalent to T = {(i, f(i)) : i The main practical issue is the size of the table but I don't see any theoretical reasons why this is incorrect.…

Here's a counterexample. Suppose I create a simple neural network that computes f(x) = x^2 + c (where x and c are complex numbers) and then I run it as an RNN. This RNN will compute the mandelbrot set, which can't be represented by a lookup table. You can't even know if the RNN will halt for a given input. Neural networks are stronger than lookup tables, they are programs.

Every computable function can be represented by a (possibly infinite)¹ lookup table.

Computer programs can only compute computable functions. Therefore any computer program is (in theory) equivalent to a table lookup.

¹ For finite inputs, the lookup table can be finite, and for infinite inputs, the lookup table can be infinite but still countable, as the set of computable functions is countable.

Post reply on HN