Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is Mathematically Impossible 2: When Entropy Returns

philarchive.org

Post reply on HN
Print view View original

281–290 of 437 posts

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#281

Post by viraptor »

usrbinbash wrote: post #271
bubblyworld wrote: post #235

I find the mathematics in this paper a little incoherent so it's hard to criticise it on those grounds - but on a charitable read, something that sticks out to me is the assumption that AGI is some fixed total computable function from the fixed decision domain to a policy. AIs these days autonomously seek information themselves. Much like living things, they are recycling entropy and information to/from their environ…

> Much like living things, they are recycling entropy and information to/from their environment (the internet) at runtime. 3 Problems with that assumption: a) Unlike living things, that information doesn't allow them to change. When a human touches a hotplate for the first time, it will (in addition to probably yelling and cursing a lot), learn that hotplates are dangerous and change its internal state to reflect tha…

> Unlike living things, that information doesn't allow them to change.

The paper is talking about whole systems for AGI not the current isolated idea of pure LLM. Systems can store memories without issues. I'm using that for my planning system and the memories and graph triplets get filled out automatically, the get incorporated in future operations.

> It can produce some sequence that may or may not cause some external entity to feed it back some more data

That's exactly what people do while they do research.

> The representation of the information has nothing to do with what it represents.

That whole point implies that the situation is different in our brains. I've not seen anyone describe exactly how our thinking works, so saying this is a limitation for intelligence is not a great point.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#282

Post by Sporktacular »

This doesn't make sense. If we can form logic circuits from biological matter we can create functionally equivalent circuits from other technologies - in hardware or software. They might have quirks but the way we know AGI is possible is because GI is possible. It may not come from LLMs or other current technologies but claiming there is a mathematical bound, and such a contestable one at that, is dubious.

Unless you want to claim some non-material basis for biological intelligence, in which case you should start by proving that.

This whole thing is fishy - "I do you the favor and leave out the middle part (although it's insightful). And we come to the end" - who publishes that? The foreword about Apple's paper is pretty clearly tacked on in a bid for relevance. Not sure why people should take this more seriously than the author takes it himself.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#283

Post by ICBTheory »

vidarh wrote: post #263
ICBTheory wrote: post #259

Earlier quoted context omitted.

No problem here is you proof - although a bit long: 1. THEOREM: Let a semantic frame be defined as Ω = (Σ, R), where Σ is a finite symbol set and R is a finite set of inference rules. Let Ω′ = (Σ′, R′) be a candidate successor frame. Define a frame jump as: Frame Jump Condition: Ω′ extends Ω if Σ′\Σ ≠ ∅ or R′\R ≠ ∅ Let P be a deterministic Turing machine (TM) operating entirely within Ω. Then: Lemma 1 (Symbol Contain…

None of this is relevant to what I wrote. If anything, they sugget that you don't understand the argument. If anything, your argument is begging the question - it's a logical fallacy - because your argument rests on humans exceeding the Turing computable, to use human abilities as evidence. But if humans do not exceed the Turing computable, then everything humans can do is evidence that something is Turing computable…

You’re flipping the logic.

I’m not assuming humans are beyond Turing-computable and then using that to prove that AGI can’t be. I’m saying: here is a provable formal limit for algorithmic systems ->symbolic containment. That’s theorem-level logic.

Then I look at real-world examples (Einstein is just one) where new symbols, concepts, and transformation rules appear that were not derivable within the predecessor frame. You can claim, philosophically (!), that “well, humans must be computable, so Einstein’s leap must be too.” Fine. But now you’re asserting that the uncomputable must be computable because humans did it. That’s your circularity, not mine. I don’t claim humans are “super-Turing.” I claim that frame-jumping is not computation. You can still be physical, messy, and bounded .. and generate outside your rational model. That’s all the proof needs.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#284

Post by vidarh »

ICBTheory wrote: post #283
vidarh wrote: post #263

Earlier quoted context omitted.

None of this is relevant to what I wrote. If anything, they sugget that you don't understand the argument. If anything, your argument is begging the question - it's a logical fallacy - because your argument rests on humans exceeding the Turing computable, to use human abilities as evidence. But if humans do not exceed the Turing computable, then everything humans can do is evidence that something is Turing computable…

You’re flipping the logic. I’m not assuming humans are beyond Turing-computable and then using that to prove that AGI can’t be. I’m saying: here is a provable formal limit for algorithmic systems ->symbolic containment. That’s theorem-level logic. Then I look at real-world examples (Einstein is just one) where new symbols, concepts, and transformation rules appear that were not derivable within the predecessor frame.…

No, I'm not flipping the logic.

> I’m not assuming humans are beyond Turing-computable and then using that to prove that AGI can’t be. I’m saying: here is a provable formal limit for algorithmic systems ->symbolic containment. That’s theorem-level logic.

Any such "proof" is irrelevant unless you can prove that humans can exceed the Turing computable. If humans can't exceed the Turing computable, then any "proof" that shows limits for algoritmic systems that somehow don't apply to humans must inherently be incorrect.

And so you're sidestepping the issue.

> But now you’re asserting that the uncomputable must be computable because humans did it.

No, you're here demonstrating you failed to understand the argument.

I'm asserting that you cannot use the fact that humans can do something as proof that humans exceed the Turing computable, because if humans do not exceed the Turing computable said "proof" would still give the same result. As such it does not prove anything.

And proving that humans exceed the Turing computable is a necessary precondition for proving AGI impossible.

> I don’t claim humans are “super-Turing.”

Then your claim to prove AGI can't exist is trivially false. For it to be true, you would need to make that claim, and prove it.

That you don't seem to understand this tells me you don't understand the subject.

(See also my edit above; your proof also contains elmentary failures to understand Turing machines)

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#285

Post by viraptor »

cess11 wrote: post #273
bubblyworld wrote: post #252

Earlier quoted context omitted.

I think it would turn off, no shocker there. I'm not sure what you mean, can you elaborate? When I say autonomous I don't mean some high-falutin philosophical concept, I just mean it does stuff on it's own.

Right, but it doesn't. It stops once you stop forcing it to do stuff.

Because that's what they're created to do. You can make a system which runs continuously. It's not a tech limitation, just how we preferred things to work so far.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#287

Post by harimau777 »

vidarh wrote: post #230
haneul wrote: post #193

Earlier quoted context omitted.

> In plain language: > No matter how sophisticated, the system MUST fail on some inputs. Well, no person is immune to propaganda and stupididty, so I don't see it as a huge issue.

I have no idea how you believe this relates to the comment you replied to.

If I'm understanding correctly, they are arguing that the paper only requires that an intelligent system will fail for some inputs and suggest that things like propaganda are inputs for which the human intelligent system fails. Therefore, they are suggesting that the human intelligent system does not necessarily refute the paper's argument.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#288

Post by Agraillo »

yodon wrote: post #72
ICBTheory wrote: post #2

This paper presents a theoretical proof that AGI systems will structurally collapse under certain semantic conditions — not due to lack of compute, but because of how entropy behaves in heavy-tailed decision spaces. The idea is called IOpenER: Information Opens, Entropy Rises. It builds on Shannon’s information theory to show that in specific problem classes (those with α ≤ 1), adding information doesn’t reduce uncer…

I'm wondering if you may have rediscovered the concept of "Wicked Problems", which have been studied in system analysis and sociology since the 1970's (I'd cite the Wikipedia page, but I've never been particularly fond of Wikipedia's write up on them). They may be worth reading up on if you're not familiar with them.

It's interesting. The question from the paper "Darling, please be honest: have I gained weight?" assumes that the "socially acceptability" of the answer should be taken into account. In this case the problem fits the "Wickedness" (Wikipedia's quote is "Classic examples of wicked problems include economic, environmental, and political issues"). But taken formally, and with the ability for LLM to ask questions in return to decrease formal uncertainty ("Please, give me several full photos of yourself from the past year to evaluate"), it is not "wicked" at all. This example alone makes the topic very uncertain in itself

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#289

Post by daqhris »

usrbinbash wrote: post #271
bubblyworld wrote: post #235

I find the mathematics in this paper a little incoherent so it's hard to criticise it on those grounds - but on a charitable read, something that sticks out to me is the assumption that AGI is some fixed total computable function from the fixed decision domain to a policy. AIs these days autonomously seek information themselves. Much like living things, they are recycling entropy and information to/from their environ…

> Much like living things, they are recycling entropy and information to/from their environment (the internet) at runtime. 3 Problems with that assumption: a) Unlike living things, that information doesn't allow them to change. When a human touches a hotplate for the first time, it will (in addition to probably yelling and cursing a lot), learn that hotplates are dangerous and change its internal state to reflect tha…

The original assumption remains valid to me based on a nearly-one year-long coding collaboration with Devin AI.

Your assertions also make some sense, especially on a technical level. I'd add only that human minds are no longer the only minds utilizing digital tools. There is almost no protective gears or powerful barrier that would likely stand in the way of sentient AIs or AGI trying to "run" and function well on bio cells, like what makes up humans or animals, for the sake of their computational needs and self-interests.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#290

Post by somewhereoutth »

vidarh wrote: post #276
somewhereoutth wrote: post #268

Earlier quoted context omitted.

> then the only way of showing that an artificial intelligence can not theoretically be constructed to at least meet the same bar is by showing that humans can compute more than the Turing computable. I would reframe: the only way of showing that artificial intelligence can be constructed is by showing that humans cannot compute more than the Turing computable. Given that Turing computable functions are a vanishingly…

Given that we know of no computable function that isn't Turing computable, and the set of Turing computable functions is known to be equivalent to the lambda calculus and equivalent to the set of general recursive functions, what is an immensely large hurdle would be to show even a single example of a computable function that is not Turing computable. If you can do so, you'd have proven Turing, Kleen, Church, Goedel…

Function != Computable Function / general recursive function.

That's my point - computable functions are a [vanishingly] small subset of all functions.

For example (and close to our hearts!), the Halting Problem. There is a function from valid programs to halt/not-halt. This is clearly a function, as it has a well defined domain and co-domain, and produces the same output for the same input. However it is not computable!

For sure a finite alphabet can describe an infinity as you show - but not all infinity. For example almost all Real numbers cannot be defined/described with a finite string in a finite alphabet (they can of course be defined with countably infinite strings in a finite alphabet).

Post reply on HN