Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is Mathematically Impossible 2: When Entropy Returns

philarchive.org

Post reply on HN
Print view View original

361–370 of 437 posts

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#361

Post by int_19h »

alganet wrote: post #255
int_19h wrote: post #155

Earlier quoted context omitted.

So far, we have found no need for this hypothesis. (Aside from "explaining" why AI couldn't ever possibly be "really intelligent" for those who find this notion existentially offensive.)

"emergent superintelligent AI" is as much superstition as believing in imaterial souls. One company literally used the term "people spirits" to refer to how LLMs behave in their official communications. It's a cult. Like many cults, it tries to latch on science to give itself legitimacy. In this case, mathematics. It has happened before many times. You're trying to say that, because it's computers and stuff, it's sci…

I didn't say anything about "emergent superintelligent AI".

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#362

Post by kbelder »

ben_w wrote: post #275
cma wrote: post #187

Earlier quoted context omitted.

If you rarely got to see letters and just saw fragments of words as something like Chinese characters (tokens), could you count the R's in arbitrary words well? The bigger issue is LLMs still need way way more data than humans get tons what they do. But they also have many less parameters than the human brain.

> If you rarely got to see letters and just saw fragments of words as something like Chinese characters (tokens), could you count the R's in arbitrary words well? While this seems correct, I'm sure I tried this when it was novel and observed that it could split the word into separate letters and then still count them wrong , which suggested something weird is happening internally. I just now tried to repeat this, and…

Counting letters is such a dull test. LLMs generally have a hard time with this question because letters are tokenized before they receive them, and they have to go through an involved reasoning process to figure it out. It's like asking a color blind person what color the street light is, and declaring him unintelligent because he sometimes gets the answer wrong.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#363

Post by adastra22 »

mrguyorama wrote: post #350
adastra22 wrote: post #160

Earlier quoted context omitted.

Penrose’s views on consciousness is largely considered quackery by other physicists.

Nobody should care what ANY physicists say about consciousness. I mean seriously, what? I don't go asking my car mechanic about which solvents are best for extracting a polar molecule, or asking my software developer about psychology.

Yet somehow quantum woo is constantly evoked to explain consciousness.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#364

Post by bigyabai »

ben_w wrote: post #145
bigyabai wrote: post #126

Earlier quoted context omitted.

And when that prize is claimed, we'll ring the bell on AGI being found. Gentleman's agreement.

I don't see how it will convince anyone: people said as much before chess, then again about Go, and are still currently disagreeing with each other if LLMs do or don't pass the Turing test. Irregardless, this was to demonstrate by analogy that things that seem simple can actually be really hard to fully understand.

I have never once heard someone describe Stockfish as potentially AGI. Honestly I don't remember anyone making the argument with AlphaGo or even IBM Watson, either.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#365

Post by bonoboTP »

vidarh wrote: post #300
harimau777 wrote: post #291

Earlier quoted context omitted.

I think that may depend on how someone defines intelligence. For example, if intelligence includes the ability to feel emotion or appreciate art, then I think it becomes much more plausible that intelligence is not the same as computation. Of course, simply stating that isn't in of itself a philisophically rigorous argument. However, given that not everyone has training in philosophy and it may not even be possible t…

I think if they define intelligence that way, it isn't a very interesting discussion, because we're back to Church-Turing: Either they can show that this actually has an effect on the ability to reason and the possible outputs of the system that somehow exceeds the Turing computable, or those aspects are irrelevant to an outside observer of said entity because the entity would still be able to act in exactly the same…

Church-Turing doesn't touch upon intelligence nor consciousness. It talks about "effective procedures". It claims that every effectively computable thing is Turing computable. And effective procedures are such that "Its instructions need only to be followed rigorously to succeed. In other words, it requires no ingenuity to succeed."

Church-Turing explicitly doesn't touch upon ingenuity. It's very well compatible with Church-Turing that humans are capable of some weird decision making that is not modelable with the Turing machine.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#366

Post by marcosdumay »

galangalalgol wrote: post #138
marcosdumay wrote: post #122

Earlier quoted context omitted.

> Are we treating an arbitrary ontological assertion as if it’s a formal argument that needs to be heroically refuted? If you are claiming that human intelligence is not "general", you'd better put a huge disclaimer on your text. You are free to redefine words to mean whatever you want, but if you use something so different from the way the entire world uses it, the onus is on you to make it very clear. And the alter…

I mean, I think it is becoming increasingly obvious humans aren't doing as much as we thought they were. So yes, this seems like an overly ambitious definition of what we would in practice call agi. Can someone eli5 the requirement this paper puts on something to be considered a gi?

I'm not sure I got the details right, but the paper seems to define "general" as in capable of making a decision rationally following a set of values in any computable problem-space.

If I got that right, yeah, humans absolutely don't qualify. It's not much of a jump to discover it's impossible.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#367

Post by bonoboTP »

vidarh wrote: post #302
harimau777 wrote: post #293

Earlier quoted context omitted.

> Assuming the Church-Turing thesis is true, the existence of any brain now or in the past capable of proving it is proof that such a program may exist. Doesn't that assume that the brain is a Turing machine or equivalent to one? My understanding is that the exact nature of the brain and how it relates to the mind is still an open question.

That is exactly the point. If the Church-Turing thesis is true, then the brain is a Turing machine / Turing equivalent. And so, assuming Church-Turing is true, then the existence of the brain is proof of the possibility of AGI, because any Turing machine can simulate any other Turing machine (possibly too slowly to be practical, but it denies its impossibility). And so, any proof that AGI is "mathematically impossibl…

> If the Church-Turing thesis is true, then the brain is a Turing machine / Turing equivalent

This is simply technically not true. You can look up the https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis and it does not talk about brains, or intelligence.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#369

Post by giardini »

catoc wrote: post #141
vessenes wrote: post #32

Earlier quoted context omitted.

Thanks for this - Looking forward to reading the full paper. That said, the most obvious objection that comes to mind about the title is that … well, I feel that I’m generally intelligent, and therefore general intelligence of some sort is clearly not impossible. Can you give a short précis as to how you are distinguishing humans and the “A” in artificial?

That about ‘cogito ergo sums it up’ doesn’t it? Intelligence is clearly possible. My gut feeling is our brain solves this by removing complexity. It certainly does so, continuously filtering out (ignoring) large parts of input, and generously interpolating over gaps (making stuff up). Whether this evolved to overcome this theorem I am not intelligent enough to conclude.

catoc states, amongst other things, that: >"Intelligence is clearly possible."

Perhaps not a citation but a proof is required here!

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#370

Post by Mr_Minderbinder »

woolion wrote: post #341
anal_reactor wrote: post #323

Earlier quoted context omitted.

What's wrong with that? Most likely, the discussion coming from various people has more value than any single article, unless it's something truly phenomenal.

I never said it was wrong, nor right. In fact, you might even read that as an excuse for "counter-hypists", as it's a pretty bad look to upvote such a low-quality submission. And I've made my own fun of AGI hype, but with knowledge of the fact that brevity is the fool of wit.

> ...as it's a pretty bad look to upvote such a low-quality submission.

I had already just about dismissed HN as a place for any serious discussion of AI for a multitude of reasons. After seeing this I think I will be hammering in the final nail.

It has already been known for decades that arbitrarily precise approximations of mathematical formulations of AGI are computable. I was expecting nothing less than a refutation of that work from this based on the title. Unfortunately the first page alone makes it apparent that it is not, nor likely even a serious work of mathematics.

Post reply on HN