Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is Mathematically Impossible 2: When Entropy Returns

philarchive.org

Post reply on HN
Print view View original

141–150 of 437 posts

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#141

Post by catoc »

vessenes wrote: post #32
ICBTheory wrote: post #2

This paper presents a theoretical proof that AGI systems will structurally collapse under certain semantic conditions — not due to lack of compute, but because of how entropy behaves in heavy-tailed decision spaces. The idea is called IOpenER: Information Opens, Entropy Rises. It builds on Shannon’s information theory to show that in specific problem classes (those with α ≤ 1), adding information doesn’t reduce uncer…

Thanks for this - Looking forward to reading the full paper. That said, the most obvious objection that comes to mind about the title is that … well, I feel that I’m generally intelligent, and therefore general intelligence of some sort is clearly not impossible. Can you give a short précis as to how you are distinguishing humans and the “A” in artificial?

That about ‘cogito ergo sums it up’ doesn’t it?

Intelligence is clearly possible. My gut feeling is our brain solves this by removing complexity. It certainly does so, continuously filtering out (ignoring) large parts of input, and generously interpolating over gaps (making stuff up). Whether this evolved to overcome this theorem I am not intelligent enough to conclude.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#142

Post by felipeerias »

daedrdev wrote: post #29

Clearly nature avoids this problem. So theoretically by replicating natural selection or something else in AI models, which arguably we already do, the theoretical entropy trap clearly can be avoided, we aren't even potentially decreasing entropy with AI training since doing so uses power generation which increases entropy

If we did that, would we be really replicating what nature does, or would we be just simulating it?

Human intelligence and consciousness are embodied. They are emerging features of complex biological systems that evolved over thousands and millions of years. The desirable intelligent behaviours that we seek to replicate are exhibited by those same biological systems only after decades of growth and training.

We can only hope to simulate these processes, not replicate them exactly. And the problem with such a simulation is that we have no idea if the stuff that we are necessarily leaving out is actually essential to the outcome that we seek.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#143

Post by kaibee »

rpcope1 wrote: post #133
ben_w wrote: post #14

Earlier quoted context omitted.

The mathematical proof, as you describe it, sounds like the "No Free Lunch theorem". Humans also can't generalise to learning such things. As you note in 2.1, there is widespread disagreement on what "AGI" means. I note that you list several definitions which are essentially "is human equivalent". As humans can be reduced to physics, and physics can be expressed as a computer program, obviously any such definition ca…

> physics can be expressed as a computer program Citation needed. If you've spent any time dynamical systems, as an example, you'd know that the computer basically only kind of crudely estimates things, and only things that are abstractly near by. You may be able to write down some PDEs or field equations that may describe things at some base level, but even statistical mechanics, which is really what governs a huge…

> Citation needed. If you've spent any time dynamical systems, as an example, you'd know that the computer basically only kind of crudely estimates things, and only things that are abstractly near by. You may be able to write down some PDEs or field equations that may describe things at some base level, but even statistical mechanics, which is really what governs a huge amount of what we see and interact with, is just a pretty good approximation. Computers (especially real ones) only generate approximate (to some value of alpha) answers; physics is not reducible to a computer program at all.

You're proving too much. The fact of the matter is that those crude estimations are routinely used to model systems.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#144

Post by donkeybeer »

ICBTheory wrote: post #52
ffwd wrote: post #10

Earlier quoted context omitted.

I think humans have some kind of algorithm for deciding what's true and consolidating information. What that is I don't know.

I guess so too... but whatever it is: it cannot possibly be something algorithmic. Therefore it doesn't matter in terms of demonstrating that AI has a boundary there, that cannot be transcended by tech, compute, training, data etc.

Explain what you mean by "algorithm" and "algorithmic". Be very precise. You are using this vague word to hinge on your entire argument and it is necessary you explain first what it means. Since from reading your replies here it is clear you are laboring under a defitnition of "algorithm" quite different from the accepted one.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#145

Post by ben_w »

bigyabai wrote: post #126
ben_w wrote: post #82

Earlier quoted context omitted.

We have the Navier–Stokes equations which fit on a matchbox, yet for the last 25 years there's been a US$1,000,000 prize on offer to the first person providing a solution for a specific statement of the problem: Prove or give a counter-example of the following statement: In three space dimensions and time, given an initial velocity field, there exists a vector velocity and a scalar pressure field, which are both smoo…

And when that prize is claimed, we'll ring the bell on AGI being found. Gentleman's agreement.

I don't see how it will convince anyone: people said as much before chess, then again about Go, and are still currently disagreeing with each other if LLMs do or don't pass the Turing test.

Irregardless, this was to demonstrate by analogy that things that seem simple can actually be really hard to fully understand.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#146

Post by int_19h »

felipeerias wrote: post #142
daedrdev wrote: post #29

Clearly nature avoids this problem. So theoretically by replicating natural selection or something else in AI models, which arguably we already do, the theoretical entropy trap clearly can be avoided, we aren't even potentially decreasing entropy with AI training since doing so uses power generation which increases entropy

If we did that, would we be really replicating what nature does, or would we be just simulating it? Human intelligence and consciousness are embodied. They are emerging features of complex biological systems that evolved over thousands and millions of years. The desirable intelligent behaviours that we seek to replicate are exhibited by those same biological systems only after decades of growth and training. We can o…

It doesn't matter wrt the claims the article makes, though. If AGI is an emergent feature of complex biological systems, then it's still fundamentally possible to simulate it given sufficient understanding of said systems (or perhaps physics if that turns out to be easier to grok in full) and sufficient compute.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#147

Post by chmod775 »

ninetyninenine wrote: post #5

Without reading the paper how the heck is agi mathematically impossible if humans are possible? Unless the paper is claiming humans are mathematically impossible? I’ll read the paper but the title comes off as out of touch with reality.

> Without reading the paper how the heck is agi mathematically impossible if humans are possible? Unless the paper is claiming humans are mathematically impossible?

Humans are provably impossible to accurately simulate using our current theoretical models which treat time as continuous. If we could prove that there's some resolution, or minimum time step, (like Planck Time) below which time does not matter and we update our models accordingly, then that might change*. For now time is continuous in every physical model we have, and thus digital computers are not able to accurately simulate the physical world using any of our models.

Right now we can't outright dismiss that there might be some special sauce to the physical world that digital computers with their finite state cannot represent.

* A theory of quantum gravitation would likely have to give an answer to that question, so hold out for that.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#148

Post by ben_w »

rpcope1 wrote: post #133
ben_w wrote: post #14

Earlier quoted context omitted.

The mathematical proof, as you describe it, sounds like the "No Free Lunch theorem". Humans also can't generalise to learning such things. As you note in 2.1, there is widespread disagreement on what "AGI" means. I note that you list several definitions which are essentially "is human equivalent". As humans can be reduced to physics, and physics can be expressed as a computer program, obviously any such definition ca…

> physics can be expressed as a computer program Citation needed. If you've spent any time dynamical systems, as an example, you'd know that the computer basically only kind of crudely estimates things, and only things that are abstractly near by. You may be able to write down some PDEs or field equations that may describe things at some base level, but even statistical mechanics, which is really what governs a huge…

> You may be able to write down some PDEs or field equations that may describe things at some base level, but even statistical mechanics, which is really what governs a huge amount of what we see and interact with, is just a pretty good approximation.

QED.

When the approximation is indistinguishable from observation over a time horizon exceeding a human lifetime, it's good enough for the purpose of "would a simulation of a human be intelligent by any definition that the real human also meets?"

Remember, this is claiming to be a mathematical proof, not a practical one, so we don't even have to bother with details like "a classical computer approximating to this degree and time horizon might collapse into a black hole if we tried to build it".

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#149

Post by [deleted] »

ICBTheory wrote: post #68
vessenes wrote: post #32

Earlier quoted context omitted.

Thanks for this - Looking forward to reading the full paper. That said, the most obvious objection that comes to mind about the title is that … well, I feel that I’m generally intelligent, and therefore general intelligence of some sort is clearly not impossible. Can you give a short précis as to how you are distinguishing humans and the “A” in artificial?

Sure I can (and thanks for writing) Well, given the specific way you asked that question I confirm your self assertion - and am quite certain that your level of Artificiality converges to zero, which would make you a GI without A... - You stated to "feel" generally intelligent (A's don't feel and don't have an "I" that can feel) - Your nuanced, subtly ironic and self referential way of formulating clearly suggests th…

[deleted]

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#150

Post by int_19h »

bigyabai wrote: post #18
ninetyninenine wrote: post #15

Earlier quoted context omitted.

We don’t even know how LLMs work. But we do know the underlying mechanisms are governed by math because we have a theory of reality that governs things down to the atomic scale and humans and LLMs are made out of atoms. So because of this we know reality is governed by maths. We just can’t fully model the high level consequence of emergent patterns due to the sheer complexity of trillions of interacting atoms. So it’…

> We don’t even know how LLMs work Speak for yourself. LLMs are a feedforward algorithm inferring static weights to create a tokenized response string. We can compare that pretty trivially to the dynamic relationship of neurons and synapses in the human brain. It's not similar, case closed. That's the extent of serious discussion that can be had comparing LLMs to human thought, with apologies to Chomsky et. al. It's…

The fact that it doesn't operate identically or even similarly on the physical layer doesn't mean that similar processes cannot emerge on higher levels of abstraction.
Post reply on HN