Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is Mathematically Impossible 2: When Entropy Returns

philarchive.org

Post reply on HN
Print view View original

51–60 of 437 posts

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#51

Post by ICBTheory »

like_any_other wrote: post #3

So does the human brain transcend math, or are humans not generally intelligent?

Hi and thanks for engaging :-)

Well, it in fact depends on what intelligence is to your understanding:

-If it intelligence = IQ, i.e. the rational ability to infer, to detect/recognize and extrapolate patterns etc, then AI is or will soon be more intelligent than us, while we humans are just muddling through or simply lucky having found relativity theory and other innovations just at the convenient moment in time ... So then, AI will soon also stumble over all kind of innovations. None of both will be able to deliberately think beyond what is thinkable at the respective present.

- But If intelligence is not only a level of pure rational cognition, but maybe an ability to somehow overcome these frame-limits, then humans obviously exert some sort of abilities that are beyond rational inference. Abilities that algorithms can impossibly reach, as all they can is compute.

- Or: intelligence = IQ, but it turns out to be useless in big, pivotal situations where you’re supposed to choose the “best” option — yet the set of possible options isn’t finite, knowable, or probabilistically definable. There’s no way to defer to probability, to optimize, or even to define what “best” means in a stable way. The whole logic of decision collapses — and IQ has nothing left to grab onto.

The main point is: neither algorithms nor rationality can point beyond itself.

In other words: You cannot think out of the box - thinking IS the box.

(maybe have a quick look at my first proof -last chapter before conclusion- - you will find a historical timeline on that IQ-Thing)

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#52

Post by ICBTheory »

ffwd wrote: post #10
like_any_other wrote: post #3

So does the human brain transcend math, or are humans not generally intelligent?

I think humans have some kind of algorithm for deciding what's true and consolidating information. What that is I don't know.

I guess so too... but whatever it is: it cannot possibly be something algorithmic. Therefore it doesn't matter in terms of demonstrating that AI has a boundary there, that cannot be transcended by tech, compute, training, data etc.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#53

Post by proc0 »

The paper is skipping over the definition of AI. It jumps right into AGI, and that depends on what AI means. It could be LLMs, deep neural networks, or any possible implementation on a Turing machine. The latter I suspect would be extremely difficult to prove. So far almost everything can be simulated by Turing machines and there's no reason it couldn't also simulate human brains, and therefore AGI. Even if the claim is that human brains are not enough for GI (and that our bodies are also part of the intelligence equation), we could still simulate an entire human being down to every cell, in theory (although in practice it wouldn't happen anytime soon, unless maybe quantum computers, but I digress).

Still an interesting take and will need to dive in more, but already if we assume the brain is doing information processing then the immediate question is how can the brain avoid this problem, as others are pointing out. Is biological computation/intelligence special?

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#54

Post by Workaccount2 »

IAmGraydon wrote: post #24
ninetyninenine wrote: post #15

Earlier quoted context omitted.

We don’t even know how LLMs work. But we do know the underlying mechanisms are governed by math because we have a theory of reality that governs things down to the atomic scale and humans and LLMs are made out of atoms. So because of this we know reality is governed by maths. We just can’t fully model the high level consequence of emergent patterns due to the sheer complexity of trillions of interacting atoms. So it’…

>We don’t even know how LLMs work. Care to elaborate? Because that is utter nonsense.

We understand and build the trellis that the LLMs "grow" on. We don't have good insight into how a fully grown LLM actually turns any specific input into any specific output. We can follow it through the network, but it's a totally senseless noisy mess.

"Cat" lights up a certain set of neurons, but then "cat" looks completely different. That is what we don't really understand.

(This is an illustrative example made for easy understanding, not something I specifically went and compared)

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#55

Post by JdeBP »

pvg wrote: post #49
JdeBP wrote: post #36

This has a single author; is not peer-reviewed; is not published in a journal; and was self-submitted both to PhilArchive and here on Hacker News.

There's nothing wrong with any of that, for an HN submission. The paper itself could be bad but that's what the discussion thread is for - discussing the thing presented rather than its meta attributes.

And no-one said that there was anything wrong, the inference being yours. But it's important to bear provenance in mind, and not get carried away by something like this more than one would be carried away by, say, an article on Medium propounding the same thing, as the bars to be cleared are about the same height.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#56

Post by alganet »

mort96 wrote: post #43
alganet wrote: post #16

Earlier quoted context omitted.

What makes you think that human intelligence is based on mathematics?

I will answer under the metaphysical assumption that there is no immaterial "soul", and that the entirety of the human experience arises from material things governed by the laws of physics. If you disagree with this assumption, there is no conversation to be had. The laws of physics can, as far as I can tell, be described using mathematics. That doesn't mean that we have a perfect mathematical model of the laws of p…

I'm not talking about soul.

I'm just saying you're mistaking the thing for the the tool we use to describe the thing.

I'm also not talking about simulations.

Epistemologically, I'm talking about unknown unknowns. There are things we don't know, and we still don't know we don't know yet. Math and physics deal with known unknowns (we know we don't know) and known knowns (we know we know) only. Math and physics do not address unknown unknowns up until they become known unknowns (we did not tackle quantum up until we discover quantum).

We don't know how humans think. It is a known unknown, tackled by many sciences, but so far, incomplete in its description. We think we have a good description, but we don't know how good it is.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#57

Post by geoka9 »

like_any_other wrote: post #3

So does the human brain transcend math, or are humans not generally intelligent?

Humans are fallible in a way computers are not. One could argue any creative process is an exercise in fallibility.

More interestingly, humans are capable of assessing the results of their "neural misfires" ("hmm, there's something to this"), whereas even if we could make a computer do such mistakes, it wouldn't know its Penny Lane from its Daddy's Car[0], even if it managed to come up with one.

[0]https://www.youtube.com/watch?v=LSHZ_b05W7o

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#58

Post by mort96 »

alganet wrote: post #56
mort96 wrote: post #43

Earlier quoted context omitted.

I will answer under the metaphysical assumption that there is no immaterial "soul", and that the entirety of the human experience arises from material things governed by the laws of physics. If you disagree with this assumption, there is no conversation to be had. The laws of physics can, as far as I can tell, be described using mathematics. That doesn't mean that we have a perfect mathematical model of the laws of p…

I'm not talking about soul. I'm just saying you're mistaking the thing for the the tool we use to describe the thing. I'm also not talking about simulations. Epistemologically, I'm talking about unknown unknowns. There are things we don't know, and we still don't know we don't know yet. Math and physics deal with known unknowns (we know we don't know) and known knowns (we know we know) only. Math and physics do not a…

If a human body is intelligent, and we could in principle set up a computer-simulated universe which has a human body in it and simulate it forward with sufficient accuracy to make the body operate as a real-world human body has, we would have an artificial general intelligence simulated by a computer (i.e using mathematics).

If you think there are potential flaws in this line of reasoning other than the ones I already covered, I'm interested to hear.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#59

Post by tim333 »

This sounds rather silly. Given the usual definition of AGI as being human like intelligence with some variation on how smart the humans are, and the fact that humans use a network of neurons that can largely be simulated by an artificial network of neurons, it's probably twaddle largely.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#60

Post by gremlinsinc »

ICBTheory wrote: post #2

This paper presents a theoretical proof that AGI systems will structurally collapse under certain semantic conditions — not due to lack of compute, but because of how entropy behaves in heavy-tailed decision spaces. The idea is called IOpenER: Information Opens, Entropy Rises. It builds on Shannon’s information theory to show that in specific problem classes (those with α ≤ 1), adding information doesn’t reduce uncer…

does this include if the AI can devise new components and use drones and things essentially to build a new iteration of itself more capable to compute a thing and keep repeating this going out into the universe as needed for resources and using von Neumann probes.. etc?
Post reply on HN