Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is Mathematically Impossible 2: When Entropy Returns

philarchive.org

Post reply on HN
Print view View original

91–100 of 437 posts

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#91

Post by bigyabai »

ninetyninenine wrote: post #78
bigyabai wrote: post #18

Earlier quoted context omitted.

> We don’t even know how LLMs work Speak for yourself. LLMs are a feedforward algorithm inferring static weights to create a tokenized response string. We can compare that pretty trivially to the dynamic relationship of neurons and synapses in the human brain. It's not similar, case closed. That's the extent of serious discussion that can be had comparing LLMs to human thought, with apologies to Chomsky et. al. It's…

George Hinton the person largely responsible about the AI revolution has this to say: https://www.reddit.com/r/singularity/comments/1lbbg0x/geoffr... https://youtu.be/qrvK_KuIeJk?t=284 In that video above George Hinton, directly says we don't understand how it works. So I don't speak just for myself. I speak for the person who ushered in the AI revolution, I speak for Experts in the field who know what they're talkin…

Hinton invented the neural network, which is not the same as the transformer architecture used in LLMs. Asking him about LLM architectures is like asking Henry Ford if he can build a car from a bunch of scrap metal; of course he can't. He might understand the engine or the bodywork, but it's not his job to know the whole process. Nor is it Hinton's.

And that's okay - his humility isn't holding anyone back here. I'm not claiming to have memorized every model weight ever published, either. But saying that we don't know how AI works is empirically false; AI genuinely wouldn't exist if we weren't able to interpret and improve upon the transformer architecture. Your statement here is a dangerous extrapolation.

> you cannot tell me what an LLM would say nor tell me why an LLM said something for a given prompt showing that we can't fully control an LLM because we don't fully understand it.

You'd think this, but it's actually wrong. If you remove all of the seeded RNG during inference (meaning; no random seeds, no temps, just weights/tokenizer), you can actually create an equation that deterministically gives you the same string of text every time. It's a lot of math, but it's wholly possible to compute exactly what AI would say ahead of time if you can solve for the non-deterministic seeded entropy, or remove it entirely.

LLM weights and tokenizer are both always idempotent, the inference software often introduces variability for more varied responses. Just so we're on the same page here.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#92

Post by ICBTheory »

kelseyfrog wrote: post #73

> And - as wonderfully remarkable as such a system might be - it would, for our investigation, be neither appropriate nor fair to overburden AGI by an operational definition whose implicit metaphysics and its latent ontological worldviews lead to the epistemology of what we might call a “total isomorphic a priori” that produces an algorithmic world-formula that is identical with the world itself (which would then mak…

Just to make sure I understand:

–Are we treating an arbitrary ontological assertion as if it’s a formal argument that needs to be heroically refuted? Or better: is that metaphysical setup an argument?

If that’s the game, fine. Here we go:

– The claim that one can build a true, perfectly detailed, exact map of reality is… well... ambitious. It sits remarkably far from anything resembling science , since it’s conveniently untouched by that nitpicky empirical thing called evidence. But sure: freed from falsifiability, it can dream big and give birth to its omnicartographic offspring.

– oh, quick follow-up: does that “perfect map” include itself? If so... say hi to Alan Turing. If not... well, greetings to Herr Goedel.

– Also: if the world only shows itself through perception and cognition, how exactly do you map it “as it truly is”? What are you comparing your map to — other observations? Another map?

– How many properties, relations, transformations, and dimensions does the world have? Over time? Across domains? Under multiple perspectives? Go ahead, I’ll wait... (oh, and: hi too.. you know who)

And btw the true detailed map of the world exists.... It’s the world.

It’s just sort of hard to get a copy of it. Not enough material available ... and/or not enough compute....

P.S. Sorry if that came off sharp — bit of a spur-of-the-moment reply. If you want to actually dig into this seriously, I’d be happy to.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#93

Post by ben_w »

bigyabai wrote: post #86
ben_w wrote: post #38

Earlier quoted context omitted.

> Speak for yourself. LLMs are a feedforward algorithm inferring static weights to create a tokenized response string. If we're OK with descriptions so lossy that they fit in a sentence, we also understand the human brain: A electrochemical network with external inputs and some feedback loops, pumping ions around to trigger voltage cascades to create muscle contractions as outputs.

Yes. As long as we're confident in our definitions, that makes the questions easy. Is that the same as a feedforward algorithm inferring static weights to create a tokenized response string? Do you necessarily need an electrochemical network with external stimuli and feedback to generate legible text? No. The answer is already solved; AI is not a brain, we can prove this by characteristically defining them both and u…

> The answer is already solved; AI is not a brain, we can prove this by characteristically defining them both and using heuristic reasoning.

That "can" should be "could", else it presumes too much.

For both human brains and surprisingly small ANNs, far smaller than LLMs, humanity collectively does not yet know the defining characteristics of the aspects we care about.

I mean, humanity don't agree with itself what any of the three initials of AGI mean, there's 40 definitions of the word "consciousness", there are arguments about if there is either exactly one or many independent G-factors in human IQ scores, and also if those scores mean anything beyond correlating with school grades, and human nerodivergence covers various real states of existance that many of us find incomprehensible (sonetimes mutually, see e.g. most discussions where aphantasia comes up).

The main reason I expect little from an AI is that we don't know what we're doing. The main reason I can't just assume the least is because neither did evolution when we popped out.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#94

Post by alganet »

mort96 wrote: post #70
alganet wrote: post #66

Earlier quoted context omitted.

We currently can't simulate the universe. Not only in capability, but also knowledge. For example, we don't know where or when life started. Can't "simulate forward" from an event we don't understand. Also, a simulation is not the thing. It's a simulation of the thing. See? The same issue. You're mistaking the thing for the tool we use to simulate the thing. You could argue that the universe _is_ a simulation, or com…

Of course we can't simulate the universe (or, well, a slice of a universe which obeys the same laws as ours) right now , but we're discussing whether it's possible in principle or not. I don't understand what fundamental difference you see between a thing governed by a set of mathematical laws and an implementation of a simulation which follows the same mathematical laws. Why would intelligence be possible in the for…

> a thing governed by a set of mathematical laws

Again, you're mistaking the thing for the tool we use to describe the thing.

> aside from precision limitations

It's not only about precision. There are things we don't know.

--

I think the universe always obeys rules for everything, but it's an educated guess. There could be rules we don't yet understand and are outside of what mathematics and physics can know. Again, there are many things we don't know. "We'll get there" is only good enough when we get there.

The difference is subtle. I require proof, you seem to be ok with not having it.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#95

Post by kevin42 »

ICBTheory wrote: post #68
vessenes wrote: post #32

Earlier quoted context omitted.

Thanks for this - Looking forward to reading the full paper. That said, the most obvious objection that comes to mind about the title is that … well, I feel that I’m generally intelligent, and therefore general intelligence of some sort is clearly not impossible. Can you give a short précis as to how you are distinguishing humans and the “A” in artificial?

Sure I can (and thanks for writing) Well, given the specific way you asked that question I confirm your self assertion - and am quite certain that your level of Artificiality converges to zero, which would make you a GI without A... - You stated to "feel" generally intelligent (A's don't feel and don't have an "I" that can feel) - Your nuanced, subtly ironic and self referential way of formulating clearly suggests th…

>but obviously, there seems to be more than that.

I don't see how that's obvious. I'm not trying to be argumentative here, but it seems like these arguments always come down to a qualia, or the insistence that humans have some sort of 'spark' that machines don't have, therefore: AGI is not possible since machines don't have it.

I also don't understand the argument that "Your nuanced, subtly ironic and self referential way of formulating clearly suggests that you are not a purely algorithmic entity". How does that follow?

What scientific evidence is there that we are anything other than a biochemical machine? And if we are a biochemical machine, how is that inherently capable of more than a silicon based machine is capable of?

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#96

Post by ninetyninenine »

bigyabai wrote: post #91
ninetyninenine wrote: post #78

Earlier quoted context omitted.

George Hinton the person largely responsible about the AI revolution has this to say: https://www.reddit.com/r/singularity/comments/1lbbg0x/geoffr... https://youtu.be/qrvK_KuIeJk?t=284 In that video above George Hinton, directly says we don't understand how it works. So I don't speak just for myself. I speak for the person who ushered in the AI revolution, I speak for Experts in the field who know what they're talkin…

Hinton invented the neural network, which is not the same as the transformer architecture used in LLMs. Asking him about LLM architectures is like asking Henry Ford if he can build a car from a bunch of scrap metal; of course he can't. He might understand the engine or the bodywork, but it's not his job to know the whole process. Nor is it Hinton's. And that's okay - his humility isn't holding anyone back here. I'm n…

> But saying that we don't know how AI works is empirically false;

Your statement completely contradicts hintons statement. You didn’t even address his point. Basically you’re saying Hinton is wrong and you know better than him. If so, counter his argument don’t restate your argument in the form of an analogy.

> You'd think this, but it's actually wrong.

No you’re just trying to twist what I’m saying into something that’s wrong. First I never said it’s not deterministic. All computers are deterministic, even RNGs. I’m saying we have no theory about it. A plane for example you can predict its motion via a theory. The theory allows us to understand and control an airplane and predict its motion. We have nothing for an LLM. No theory that helps us predict, no theory that helps us fully control and no theory that helps us understand it beyond the high level abstraction of a best fit curve in multidimensional space. All we have is an algorithm that allows an LLM to self assemble as a side effect from emergent effects.

Rest assured I understand the transformer as much as you do (which is to say humanity has limited understanding of it) you don’t need to assume I’m just going off hintons statements. He and I knows and understands LLMs as much as you even though we didnt invent it. Please address what I said and what he said with a counter argument and not an analogy that just reiterates an identical point.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#97

Post by kevin42 »

rusk wrote: post #47
daedrdev wrote: post #29

Clearly nature avoids this problem. So theoretically by replicating natural selection or something else in AI models, which arguably we already do, the theoretical entropy trap clearly can be avoided, we aren't even potentially decreasing entropy with AI training since doing so uses power generation which increases entropy

It can be avoided certainly, but can it be avoided with the current or near term technology about which many are saying “it’s only a matter of time”

I like the distinction you made there. My observation that when it comes to AGI, there are those who are saying "Not possible with the current technology." and "Not possible at all, because humans have [insert some characteristic here about self awareness, true creativity, etc] and machines don't.

I can respect the first argument. I personally don't see any reason to believe AGI is impossible, but I also don't see evidence that it is possible with the current (very impressive) technology. We may never build an AGI in my lifetime, maybe not ever, but that doesn't mean it's not possible.

But the second argument, that humans do something machines aren't capable of always falls flat to me for lack of evidence. If we're going to dismiss the possibility of something, we shouldn't do it without evidence. We don't have a full model of human intelligence, so I think it's premature to assume we know what isn't possible. All the evidence we have is that humans are biological machines, everything follows the laws of physics, and yet, here we are. There isn't evidence that anything else is going on other than physical phenomenon, and there isn't any physical evidence that a biological machine can't be emulated.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#98

Post by jemmyw »

vessenes wrote: post #32
ICBTheory wrote: post #2

This paper presents a theoretical proof that AGI systems will structurally collapse under certain semantic conditions — not due to lack of compute, but because of how entropy behaves in heavy-tailed decision spaces. The idea is called IOpenER: Information Opens, Entropy Rises. It builds on Shannon’s information theory to show that in specific problem classes (those with α ≤ 1), adding information doesn’t reduce uncer…

Thanks for this - Looking forward to reading the full paper. That said, the most obvious objection that comes to mind about the title is that … well, I feel that I’m generally intelligent, and therefore general intelligence of some sort is clearly not impossible. Can you give a short précis as to how you are distinguishing humans and the “A” in artificial?

I would argue that you are not a general intelligence. Humans have quite a specific intelligence. It might be the broadest, most general, among animal species, but it is not general. That manifests in that we each need to spend a significant amount of time training ourselves for specific areas of capability. You can't then switch instantly to another area without further training, even though all the context materials are available to you.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#99

Post by ninetyninenine »

IAmGraydon wrote: post #24
ninetyninenine wrote: post #15

Earlier quoted context omitted.

We don’t even know how LLMs work. But we do know the underlying mechanisms are governed by math because we have a theory of reality that governs things down to the atomic scale and humans and LLMs are made out of atoms. So because of this we know reality is governed by maths. We just can’t fully model the high level consequence of emergent patterns due to the sheer complexity of trillions of interacting atoms. So it’…

>We don’t even know how LLMs work. Care to elaborate? Because that is utter nonsense.

https://youtu.be/qrvK_KuIeJk?t=284

The above is a video clip of Hinton basically contradicting what you’re saying.

So thats my elaboration. Picture that you just said what you said to me to hintons face. I think it’s better this way because I noticed peoples responding to me are rude and completely dismiss me and I don’t get good faith responses and intelligent discussion. I find if people realize that there statements are contradictory to the statements of the industry and established experts they tend to respond more charitably.

So please respond to me as if you just said to hintons face that what he said is utter nonsense because what I said is based off of what he said. Thank you.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#100

Post by Tadpole9181 »

jemmyw wrote: post #98
vessenes wrote: post #32

Earlier quoted context omitted.

Thanks for this - Looking forward to reading the full paper. That said, the most obvious objection that comes to mind about the title is that … well, I feel that I’m generally intelligent, and therefore general intelligence of some sort is clearly not impossible. Can you give a short précis as to how you are distinguishing humans and the “A” in artificial?

I would argue that you are not a general intelligence. Humans have quite a specific intelligence. It might be the broadest, most general, among animal species, but it is not general. That manifests in that we each need to spend a significant amount of time training ourselves for specific areas of capability. You can't then switch instantly to another area without further training, even though all the context material…

This seems like a meaningless distinction in context. When people say AGI, they clearly mean "effectively human intelligence". Not an infallible, completely deterministic, omniscient god-machine.
Post reply on HN