Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is Mathematically Impossible 2: When Entropy Returns

philarchive.org

Post reply on HN
Print view View original

331–340 of 437 posts

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#331

Post by viraptor »

usrbinbash wrote: post #324
viraptor wrote: post #320

Earlier quoted context omitted.

> and we don't need to know how exactly human thinking works to acknowledge that. Until you know how thinking works in humans, you can't say something else is different. We've got the same inputs available that we can provide to AI models. Saying we don't form our thinking based on statistics on those inputs and the state of the brain is a massive claim on its own.

> Until you know how thinking works in humans, you can't say something else is different. Yes, I very much can, because I can observe outcomes. Humans are a) alot more capable than language models, and b) humans do not rely solely on the statistical relationships of language tokens. How can I show that? Easily in fact: Language tokens require organized language. And our evolutionary closest relatives (big apes) don't…

> Humans are a) alot more capable than language models

That's a scale of capability, not architecture difference. A human kid is less capable than an adult, but you wouldn't classify them as thinking using different mechanisms.

> b) humans do not rely solely on the statistical relationships of language tokens. (...) Language tokens require organized language.

That's just how you provide data. Multimodal models can accept whole vectors describing images, sounds, smells, or whatever else - all of them can be processed and none of them are organised language.

> that our brains are somehow more closely related to silicon-based integrated circuits than they are to those of a Gorilla

That's entirely different from a question about functional equivalence and limit of capabilities.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#332

Post by moomin »

roenxi wrote: post #217
moomin wrote: post #183

Earlier quoted context omitted.

I’m beginning to feel like the tests are part of the problem. Our intelligence tests are all tests of specialisation. We’ve established LLMs are part of the problem. Plenty of people who would fail a bar exam yet still know how many Rs there are in strawberry, could learn a new game just by reading the rules, know how to put up a set of shelves.

I think the problem is that, as far as we can tell, AIs are just more generally intelligent than humans and people are trying to figure out how to assert that they are not. A specialist human in their area of competence can still outperform an AI, but there don't seem to be any fields now where a human novice can reliably out-think a computer. We're seeing a lot more papers like this one where we have to define human…

I don't really buy this. It's apparently not possible to build an Estonian LLM with a satisfactory level of performance. Does that mean Estonians are general intelligences and English-speakers aren't? Or just that our ways of assessing intelligence aren't valid?

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#333

Post by viraptor »

cess11 wrote: post #322
viraptor wrote: post #314

Earlier quoted context omitted.

I linked you an experiment with multiple autonomous agents operating continuously. It's already happened. It's really not clear what you're disagreeing with here.

No, that was a simulation, akin to Conway's cellular automata. You seem to consider being fully under someone else's control to qualify as autonomy, at least in certain casees, which to me comes across as very bizarre.

You seem to be taking about some kind of free will and perfect independence, not autonomy as normally understood. Agents can have autonomy within the environment they have access to. We talk about autonomous vehicles for example, where we want them to still stay within some action boundaries. Otherwise we'd be discussing metaphysics. It's not like we can cross physical/body boundaries just because we've got autonomy.

https://en.wikipedia.org/wiki/Autonomous_robot

> An autonomous robot is a robot that acts without recourse to human control. Historic examples include space probes. Modern examples include self-driving vacuums and cars.

The same idea is used for agents - they're autonomous because they independently choose actions with a specific or vague goal.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#334

Post by mystified5016 »

Yeah I see this headline and all I can think is "humans can never travel faster than 30mph or they will die" or "buildings over ten stories will asphyxiate people at the top" or how black holes were "mathematically impossible" for a few decades.

Math doesn't prove anything about the real universe until you go and physically prove it with testable predictions.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#335

Post by AlienRobot »

regularfry wrote: post #264
AlienRobot wrote: post #261

If a human brain works why can't AGI? I think the problem with "AGI" is that people don't want "AGI," they want Einstein as their butler. A merely generally intelligent AI might be only as intelligent as the average human.

One problem with the paper is that it defines AGI in such a way that if it fails to solve a problem that is inherently unsolvable, AGI can be written off as impossible. It tries to synthesise a definition from different sources whose own definitions don't have any particular reason to overlap in any meaningful way. I'm just not sure "AGI" is a useful term at this point. It's either something trivially reachable from…

As far as I'm concerned if it can pass the Turing test it's already AI enough. Not sure what the "G" adds.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#336

Post by mystified5016 »

cess11 wrote: post #311
bubblyworld wrote: post #309

Earlier quoted context omitted.

I still don't understand your point, sorry. If it's a semantic nitpick about the meaning of "autonomous", I'm not interested - I've made my definition quite clear, and it has nothing to do with when agents stop doing things or what happens when they get turned off.

I think you should start caring about the meaning of words.

You're the one using words incorrectly. Everybody else agrees on what these words mean and you're insisting on your own made-up definitions. And then you throw a fit like a child when someone disagrees.

You're wrong and you're behaving inappropriately.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#337

Post by somewhereoutth »

vidarh wrote: post #328
somewhereoutth wrote: post #319

Earlier quoted context omitted.

Humans do a lot of stuff that is hard to 'functionalise', computable or otherwise, so I'd say the burden of proof is on you. What's the function for creating a work of art? Or driving a car?

You clearly don't understand what a function means in this context, as the word function is not used in this thread in the way you appear to think it is used. For starters, to have any hope of having a productive discussion on this subject, you need to understand what "function" mean in the context of the Church-Turing thesis (a function on the natural numbers can be calculated by an effective method if and only if i…

My original reframing was: the only way of showing that artificial intelligence can be constructed is by showing that humans cannot compute more than the Turing computable.

I was assuming the word 'compute' to have broader meaning than Turing computable - otherwise that statement is a tautology of course.

I pointed out that Turing computable functions are a (vanishingly) small subset of all possible functions - of which some may be 'computable' outside of Turing machines even if they are not Turing computable.

An example might be the three-body problem, which has no general closed-form solution, meaning there is no equation that always solves it. However our solar system seems to be computing the positions of the planets just fine.

Could it be that human sapience exists largely or wholly in that space beyond Turing computability? (by Church-Turing thesis the same as computable by effective method, as you point out). In which case your AGI project as currently conceived is doomed.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#338

Post by somenameforme »

kevin42 wrote: post #321
somenameforme wrote: post #159

Earlier quoted context omitted.

Consciousness is an issue. If you write a program to add 2+2, you probably do not believe some entity poofs into existence, perceives itself as independently adding 2+2, and then poofs out of existence. Yet somehow, the idea of an emergent consciousness is that if you instead get it to do 100 basic operations, or perhaps 2^100 then suddenly this becomes true? The reason one might believe this is not because it's logi…

My issue is that from a scientific point of view, physicalism is all we have. Everything else is belief, or some form of faith. Your example about relativity is good. It might have sounded insane at some point, but it turns out, it is physics, which nicely falls into the physicalism concept. If there is a falsifiable scientific theory that there is something other than a physical mechanism behind consciousness and in…

I don't think science and consciousness go together quite well at this point. I'll claim consciousness doesn't exist. Try to prove me wrong. Of course I know I'm wrong because I am conscious, but that's literally impossible to prove, and it may very well be that way forever. You have no way of knowing I'm conscious - you could very well be the only conscious entity in existence. This is not the case because I can strongly assure you I'm conscious as well, but a philosophical zombie would say the same thing, so that assurance means nothing.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#339

Post by galangalalgol »

usrbinbash wrote: post #318
bubblyworld wrote: post #310

Earlier quoted context omitted.

Okay, yeah, like I said - not personally interested in debating the meaning of "AGI" or "understand". More power to you for thinking about it. > And that observable behavior includes hallucinations, a tendency to be repettive, falling for leading questions [...] I agree with you, obviously, these are common behaviours. You can improve the outcomes a lot with tight feedback loops for development workflows (like fast-r…

> In a vacuum these things go totally nuts And that is, in a nutshell, my point. An AGI has to be autonomous. It cannot "go nuts" without handholding, same as a human needs to be able to (under normal operating conditions) remain coherent, even if left to their own devices. > the environment deserves just as much thought in any analysis of an AI-based system. Couldn't agree more, and since I know how much work these…

Are we sure people don't work that way? Almost all of us operate on instincts almost all of the time. We have guardrails, people who operate them are often committed to institutions. When we choose to do things, it is based on that static hardwiring. Our meta model later comes up with reasons why we did the things. Sometimes, but rarely, it is correct. The human brain is extremely heterogenous, modular even. Some of our modules function remarkably like a memory store fed back into a context window. Adding a meta model to an llm that is updated autonomously by an additional model that analyzes outcomes to upde this predictive meta model would quite likely result in the agent's models mistaking the meta model for a self. Much like we do.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#340

Post by pixl97 »

somenameforme wrote: post #159
kevin42 wrote: post #95

Earlier quoted context omitted.

>but obviously, there seems to be more than that. I don't see how that's obvious. I'm not trying to be argumentative here, but it seems like these arguments always come down to a qualia, or the insistence that humans have some sort of 'spark' that machines don't have, therefore: AGI is not possible since machines don't have it. I also don't understand the argument that "Your nuanced, subtly ironic and self referentia…

Consciousness is an issue. If you write a program to add 2+2, you probably do not believe some entity poofs into existence, perceives itself as independently adding 2+2, and then poofs out of existence. Yet somehow, the idea of an emergent consciousness is that if you instead get it to do 100 basic operations, or perhaps 2^100 then suddenly this becomes true? The reason one might believe this is not because it's logi…

>Yet somehow, the idea of an emergent consciousness is that if you instead get it to do 100 basic operations, or perhaps 2^100 then suddenly this becomes true

Why not? You can do a simple add with assembly language in a few operations. But if you put millions and millions of operations together you can get a video game with emergent behaviors. If you're just looking at the additions, where does the game come from? Is it still a game if it's not output to a monitor but an internal screen buffer?

Post reply on HN