Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is Mathematically Impossible 2: When Entropy Returns

philarchive.org

Post reply on HN
Print view View original

301–310 of 437 posts

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#301

Post by ninetyninenine »

hnfong wrote: post #139
ninetyninenine wrote: post #77

Earlier quoted context omitted.

>That's not really true. You have a theory, and let's presume so far it's consistent with observations. But it doesn't mean it's 100% correct, and doesn't mean at some point in the future you won't observe something that invalidates the theory. In short, you don't know whether the theory is absolutely true and you can never know. Let me repharse it. As far as we know all of reality is governed by the principles of lo…

I guess it's kinda hubris on my part to question your ability to know things with such high certainty about things that philosophers have been struggling to prove for millenia... What you said is only true for the bits of humanity you have decided to focus upon -- capitalist, technology-driven modern societies. If you looked beyond that, there are cultures that build society upon other assumptions. You might think th…

I never made a claim for absolute truth. I said it’s the most likely truth given the fact that you get up every morning and drive a car or turn on your computer and assume everything will work. Because we all assume it, we assume all of logic behind it to be true as well.

Whatever probability is, whatever philosophers say about it any of this it doesn’t matter. You act like all of it is true including the usage of the web technology that allows you to post your idea here. You are acting as if all the logic, science and technology that was involved in the creation of that web technology is real and thus I am simply saying because the entire world claims this assumption by action then my claim is inline with the entire world.

You can make a philosophical argument but your actions aren’t inline with that. You may say no one can prove math or probability to be real but you certainly don’t live your life that way. You don’t think that science logic and technology will suddenly fall apart and not work when oh turn on your computer. In fact you live your life as if those things are fundamentally true. Yet you talk as if they might not be.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#302

Post by vidarh »

harimau777 wrote: post #293
vidarh wrote: post #274

Earlier quoted context omitted.

Assuming the Church-Turing thesis is true, the existence of any brain now or in the past capable of proving it is proof that such a program may exist. If the Church-Turing thesis can be proven false, conversely, then it may be possible that such a program can't exist - it is a necessary but not sufficient condition for the Church-Turing thesis to be false. Given we have no evidence to suggest the Church-Turing thesis…

> Assuming the Church-Turing thesis is true, the existence of any brain now or in the past capable of proving it is proof that such a program may exist. Doesn't that assume that the brain is a Turing machine or equivalent to one? My understanding is that the exact nature of the brain and how it relates to the mind is still an open question.

That is exactly the point.

If the Church-Turing thesis is true, then the brain is a Turing machine / Turing equivalent.

And so, assuming Church-Turing is true, then the existence of the brain is proof of the possibility of AGI, because any Turing machine can simulate any other Turing machine (possibly too slowly to be practical, but it denies its impossibility).

And so, any proof that AGI is "mathematically impossible" as the title claims, is inherently going to contain within it a proof that the Church-Turing thesis is false.

In which case there should be at least one example of a function a human brain can compute that a Turing machine can't.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#303

Post by vidarh »

somewhereoutth wrote: post #290
vidarh wrote: post #276

Earlier quoted context omitted.

Given that we know of no computable function that isn't Turing computable, and the set of Turing computable functions is known to be equivalent to the lambda calculus and equivalent to the set of general recursive functions, what is an immensely large hurdle would be to show even a single example of a computable function that is not Turing computable. If you can do so, you'd have proven Turing, Kleen, Church, Goedel…

Function != Computable Function / general recursive function. That's my point - computable functions are a [vanishingly] small subset of all functions. For example (and close to our hearts!), the Halting Problem. There is a function from valid programs to halt/not-halt. This is clearly a function, as it has a well defined domain and co-domain, and produces the same output for the same input. However it is not computa…

Non-computable functions are not relevant to this discussion, though, because humans can't compute them either, and so inherently an AGI need not be able to compute them.

The point remains that we know of no function that is computable to humans that is not in the Turing computable / general recursive function / lambda calculus set, and absent any indication that any such function is even possible, much less an example, it is no more reasonable to believe humans exceed the Turing computable than that we're surrounded by invisible pink unicorns, and the evidence would need to be equally extraordinary for there to be any reason to entertain the idea.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#304

Post by Agraillo »

viralsink wrote: post #69

If I understood correctly, this is about finding solutions to problems that have an infinite solution space, where new information does not constrain it. Humans don't have the processing power to traverse such vast spaces. We use heuristics, in the same way a chess player does not iterate over all possible moves. It's a valid point to make, however I'd say this just points to any AGI-like system having the same epist…

I find Wolfram's computational irreducibility is a very important aspect when dealing with modern LLMs, because for them it can be reduced (here it can) to "some questions shouldn't be inferred, but computed". From recent tests, I played with a question when models had to find cities and countries that can be connected with a common vowel in the middle (like Oslo + Norway = Oslorway). Every "non-thinking" LLMs answered mostly wrong, but wrote a perfect html/js ready to use copy/paste script, that when run found all the correct results from the world. Recent "thinking" ones managed to make do with the prompt thinking but it was a long process ending up with one or two results. We just can't avoid computations for plenty of tasks

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#305

Post by vixen99 »

triknomeister wrote: post #254
somedude222 wrote: post #200

[flagged]

https://philpeople.org/profiles/max-m-schlereth Crackpot.

i.e., IDC - information-free_comment. Thankfully comments on HN are mostly not in this category. You may well be correct but it would interesting to see where in your reading you made this decision.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#306

Post by cess11 »

viraptor wrote: post #285
cess11 wrote: post #273

Earlier quoted context omitted.

Right, but it doesn't. It stops once you stop forcing it to do stuff.

Because that's what they're created to do. You can make a system which runs continuously. It's not a tech limitation, just how we preferred things to work so far.

Maybe, but that's not the case here so it is lost on me why you bring it up.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#307

Post by viraptor »

cess11 wrote: post #306
viraptor wrote: post #285

Earlier quoted context omitted.

Because that's what they're created to do. You can make a system which runs continuously. It's not a tech limitation, just how we preferred things to work so far.

Maybe, but that's not the case here so it is lost on me why you bring it up.

You're making claims about those systems not being autonomous. When we want to, we create them to be autonomous. It's got nothing to do with agency or survival instincts. Experiments like that have been done for years now - for example https://techcrunch.com/2023/04/10/researchers-populated-a-ti...

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#308

Post by usrbinbash »

bubblyworld wrote: post #279
usrbinbash wrote: post #271

Earlier quoted context omitted.

> Much like living things, they are recycling entropy and information to/from their environment (the internet) at runtime. 3 Problems with that assumption: a) Unlike living things, that information doesn't allow them to change. When a human touches a hotplate for the first time, it will (in addition to probably yelling and cursing a lot), learn that hotplates are dangerous and change its internal state to reflect tha…

It's not really an assumption, it's an observation. Run an agentic tool and you'll see it do this kind of thing all the time. It's pretty clear that they use the information to guide themselves (i.e. there's an entropy reduction there in the space of future policies, if you want to use the language of the OP). > Unlike living things, that information doesn't allow them to change. It absolutely does. Their behaviour c…

> It absolutely does.

Go run an agentic workflow using RAG on a local model. Do an md5 checksum of the model before and after usage. The result will be the same.

> I agree that somewhere down the line there is a fixed set of tensors but that is not the algorithm.

And for our current tools, that is fine. They are not the algorithm, the LLM is just a part of a large machine that involves countless other things. And that is fine.

For an AGI, that would very much not be fine. An AGI has to be able to learn. Learning doesn't just involve gathering information, it also involves changing how information is used. New things from the information it ingests, have to be able to change what is currently a static thing, or it is not an AGI.

When a human reads a book twice, hes not encountering the information in the same way both times, because the first time he reads it, he alters his internal state. That's how we have things such as favorite books or movies.

> I really don't know how to engage on this. It certainly isn't me collecting the information.

And it certainly isn't the "AI" doing it either. I should know, because I implemented my own agentic AI frameworks. Information is provided by external systems.

And again, this is fine for LLMs playing their role in an "agentic" workflow. But an AGI that is limited to that, again, wouldn't be an AGI. It would just be a somewhat better LLM, as limited to the same constraints.

> I'm interested in their observable behaviour,

As am I. And that observable behavior includes hallucinations, a tendency to be repettive, falling for leading questions, regurgitating statistically correct (because it appears in the training set) but flawed (because it is obviosuly wrong to do so) information such as dumping API secrets into frontend code and many more problems.

All of which, in the end, boil down to the fact that a language model doesn't really "understand" the information it is dealing with. It just understands statistical relationships between tokens.

And if an AGI suffers from that same flaw, then it, again, isn't an AGI.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#309

Post by bubblyworld »

cess11 wrote: post #273
bubblyworld wrote: post #252

Earlier quoted context omitted.

I think it would turn off, no shocker there. I'm not sure what you mean, can you elaborate? When I say autonomous I don't mean some high-falutin philosophical concept, I just mean it does stuff on it's own.

Right, but it doesn't. It stops once you stop forcing it to do stuff.

I still don't understand your point, sorry. If it's a semantic nitpick about the meaning of "autonomous", I'm not interested - I've made my definition quite clear, and it has nothing to do with when agents stop doing things or what happens when they get turned off.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#310

Post by bubblyworld »

usrbinbash wrote: post #308
bubblyworld wrote: post #279

Earlier quoted context omitted.

It's not really an assumption, it's an observation. Run an agentic tool and you'll see it do this kind of thing all the time. It's pretty clear that they use the information to guide themselves (i.e. there's an entropy reduction there in the space of future policies, if you want to use the language of the OP). > Unlike living things, that information doesn't allow them to change. It absolutely does. Their behaviour c…

> It absolutely does. Go run an agentic workflow using RAG on a local model. Do an md5 checksum of the model before and after usage. The result will be the same. > I agree that somewhere down the line there is a fixed set of tensors but that is not the algorithm. And for our current tools, that is fine. They are not the algorithm, the LLM is just a part of a large machine that involves countless other things. And tha…

Okay, yeah, like I said - not personally interested in debating the meaning of "AGI" or "understand". More power to you for thinking about it.

> And that observable behavior includes hallucinations, a tendency to be repettive, falling for leading questions [...]

I agree with you, obviously, these are common behaviours. You can improve the outcomes a lot with tight feedback loops for development workflows (like fast-running tests and linting/formatting for the agent to code against). In a vacuum these things go totally nuts - part of the reason I think the environment deserves just as much thought in any analysis of an AI-based system!

> Go run an agentic workflow using RAG on a local model. Do an md5 checksum of the model before and after usage. The result will be the same.

As I said in my last comment, I agree with you. The md5 checksum of the tensors won't change. If your workflow accomplished anything at all, however, there will be many changes elsewhere in the system and it's environment (like your codebase). And those changes will in turn affect the future execution of workflows. Nothing controversial here.

Post reply on HN