Earlier quoted context omitted.
A lot of people see a title that is "subject I want to discuss" and jump to the comment section without even bothering to look at the link. There has been a lot AI hype, so counter-hypists are starved from content and just jumped on the first "confirmation bias title" they could find. Thank you for the comment, "typical crackpot" feels a bit light considering how unhinged that is.
What's wrong with that? Most likely, the discussion coming from various people has more value than any single article, unless it's something truly phenomenal.
AGI is Mathematically Impossible 2: When Entropy Returns
341–350 of 437 posts
Re: AGI is Mathematically Impossible 2: When Entropy Returns
#342Earlier quoted context omitted.
Okay, yeah, like I said - not personally interested in debating the meaning of "AGI" or "understand". More power to you for thinking about it. > And that observable behavior includes hallucinations, a tendency to be repettive, falling for leading questions [...] I agree with you, obviously, these are common behaviours. You can improve the outcomes a lot with tight feedback loops for development workflows (like fast-r…
> In a vacuum these things go totally nuts And that is, in a nutshell, my point. An AGI has to be autonomous. It cannot "go nuts" without handholding, same as a human needs to be able to (under normal operating conditions) remain coherent, even if left to their own devices. > the environment deserves just as much thought in any analysis of an AI-based system. Couldn't agree more, and since I know how much work these…
> An AGI has to be autonomous. It cannot "go nuts" without handholding [...]
So I think this is where I get off your bus - regardless of what you call it, I think current agentic systems like claude code are already there. They can construct their own handholds as they go. I have a section in all my CLAUDE.md files that tells them to always develop within a feedback loop like a test, and to set it up themselves if necessary, for instance. It works remarkably well!
There are lots of aspects of human cognition they don't seem to share... like curiousity or a drive for survival (hopefully lol). And creativity is very bad right now - although even there I think there's evidence it has some ability to be creative. So if you want that in your AGI, yeah, it's got a ways to go.
Situation seems very murky for an impossibility theorem though (to me).
> in the end I think we can agree that "something like how we do thinking" is pretty close to any valid definition, no?
I agree, we aren't even close to human-level ability here. I just think that people get hung up on looking at a bunch of tensors, but to me at least the real complexity is when these things embed in an environment.
All these arguments considering pure Turing machines miss this, I think. You don't study ecology by taking organisms out individully and cutting them up. There's value in that, of course, but the interactions are where the really interesting stuff happens.
Re: AGI is Mathematically Impossible 2: When Entropy Returns
#343Earlier quoted context omitted.
I’m beginning to feel like the tests are part of the problem. Our intelligence tests are all tests of specialisation. We’ve established LLMs are part of the problem. Plenty of people who would fail a bar exam yet still know how many Rs there are in strawberry, could learn a new game just by reading the rules, know how to put up a set of shelves.
If you rarely got to see letters and just saw fragments of words as something like Chinese characters (tokens), could you count the R's in arbitrary words well? The bigger issue is LLMs still need way way more data than humans get tons what they do. But they also have many less parameters than the human brain.
"Paucity of the stimulus" is the term for what I'm talking about with the brain needing much less data, but beyond just more parameters we may have innate language processing that isn't there in other animals; Chomsky has been kind of relegated away now after LLMs but he may still have been right if it isn't just parameter count and or the innate thing different from animals isn't something like transformers. If you look at the modern language program in Chomsky's later years, it does have some remarkably similar things to transformers: permutation independent internal representation, and the merge operation being very similar to transformer's soft max. It's kind of describing something very like single head attention.
We know animals have rich innate neural abilities beyond just beating the heart and breathing etc.: a baby horse can be blind folded from birth, several days later blind fold taken off and it can immediately walk and navigate. Further development goes on, but other animals like cats have a visual system that doesn't seem to develop at all if it doesn't get natural stimulus in a critical early period. Something like that may apply to human language, it may be multiple systems missing from other apes and early hominids, but whatever it is we don't think it had many generations to evolve. Researchers have identified circuits in songbird brains that are also in humans but not apes, and something like that may be a piece of it for tracking sequences.
Re: AGI is Mathematically Impossible 2: When Entropy Returns
#344Re: AGI is Mathematically Impossible 2: When Entropy Returns
#345> And - as wonderfully remarkable as such a system might be - it would, for our investigation, be neither appropriate nor fair to overburden AGI by an operational definition whose implicit metaphysics and its latent ontological worldviews lead to the epistemology of what we might call a “total isomorphic a priori” that produces an algorithmic world-formula that is identical with the world itself (which would then mak…
Just to make sure I understand: –Are we treating an arbitrary ontological assertion as if it’s a formal argument that needs to be heroically refuted? Or better: is that metaphysical setup an argument? If that’s the game, fine. Here we go: – The claim that one can build a true, perfectly detailed, exact map of reality is… well... ambitious. It sits remarkably far from anything resembling science , since it’s convenien…
> Are we treating an arbitrary ontological assertion as if it’s a formal argument that needs to be heroically refuted?
The formality of the paper already supposes a level of rigor. The problem at its core, is that p_intelligent(x: X) where X ∈ {human, AI} is not a demonstrable scissor by just proving p_intelligent(AI) = false. Without walking us through the steps that p_intelligent(human) = true, we cannot be sure that the predicate isn't simply always false.
Without demonstrating that humans satisfy the claims we can't be sure if the results are vacuously true because nothing, in fact, can satisfy the standard.
These aren't heroic refutations, they're table stakes.
Re: AGI is Mathematically Impossible 2: When Entropy Returns
#346Earlier quoted context omitted.
I linked you an experiment with multiple autonomous agents operating continuously. It's already happened. It's really not clear what you're disagreeing with here.
No, that was a simulation, akin to Conway's cellular automata. You seem to consider being fully under someone else's control to qualify as autonomy, at least in certain casees, which to me comes across as very bizarre.
Re: AGI is Mathematically Impossible 2: When Entropy Returns
#347Earlier quoted context omitted.
I guess it's kinda hubris on my part to question your ability to know things with such high certainty about things that philosophers have been struggling to prove for millenia... What you said is only true for the bits of humanity you have decided to focus upon -- capitalist, technology-driven modern societies. If you looked beyond that, there are cultures that build society upon other assumptions. You might think th…
I never made a claim for absolute truth. I said it’s the most likely truth given the fact that you get up every morning and drive a car or turn on your computer and assume everything will work. Because we all assume it, we assume all of logic behind it to be true as well. Whatever probability is, whatever philosophers say about it any of this it doesn’t matter. You act like all of it is true including the usage of th…
That's not what you claimed and that's not what I replied to.
You said you have a theory, and because of that you know something.
The explanation or the theory does not have to be right for something to work. The fact that I'm using modern technology does not mean that whatever theory of reality in vogue is fundamentally right. It just needs to work under certain conditions.
> You may say no one can prove math or probability to be real but you certainly don’t live your life that way. You don’t think that science logic and technology will suddenly fall apart and not work when oh turn on your computer.
That's a really strong claim to make, especially with "you". You don't know how I live. It's like seeing somebody appear in Church and denigrating them for not believing in Jesus.
No, I believe the world could fall apart at any time. Most people call it death. The fact that 99.9% people believe in death and continue their lives without panicking is probably something you want to think about as well. Heck, even a sufficiently strong solar flare could bring down this entire modern technology stack. Am I wrong to continue to use the web and debate about metaphysics given this knowledge? I don't think so, and neither do I think that my presence says anything about my belief in mathematics or whatever else governing reality.
Re: AGI is Mathematically Impossible 2: When Entropy Returns
#348Earlier quoted context omitted.
My issue is that from a scientific point of view, physicalism is all we have. Everything else is belief, or some form of faith. Your example about relativity is good. It might have sounded insane at some point, but it turns out, it is physics, which nicely falls into the physicalism concept. If there is a falsifiable scientific theory that there is something other than a physical mechanism behind consciousness and in…
I don't think science and consciousness go together quite well at this point. I'll claim consciousness doesn't exist. Try to prove me wrong. Of course I know I'm wrong because I am conscious, but that's literally impossible to prove, and it may very well be that way forever. You have no way of knowing I'm conscious - you could very well be the only conscious entity in existence. This is not the case because I can str…
It may be a trick our mind plays on us. The Global Workspace Theory addresses this, and some of the predictions this theory made have been supported by multiple experiments. If GWT is correct, it's very plausible, likely even, that an artificial intelligence could have the same type of consciousness.
Re: AGI is Mathematically Impossible 2: When Entropy Returns
#349Section 3.1
This is a question on how human do we want AI's to act, which I think could just be set thru system prompts.
Section 3.2
I think this is an argument saying that AI's are fundamentally missing certain sensory inputs so its information space is limited? Bad argument cuz you can always amend sensory information. The question could also be reframed as an experiment design problem instead of treating AI as an oracle. There's no reason an autonomous reasoning system can't do this.
Section 3.3
This is probably the worst argument yet. It's basically claiming that AI can't synthesize information?! Idk why the author keeps trying to simulate AI with his own words instead of just running the systems outright.
Re: AGI is Mathematically Impossible 2: When Entropy Returns
#350Earlier quoted context omitted.
> As humans can be reduced to physics, and physics can be expressed as a computer program This is an assumption that many physicists disagree with. Roger Penrose, for example.
Penrose’s views on consciousness is largely considered quackery by other physicists.
I mean seriously, what? I don't go asking my car mechanic about which solvents are best for extracting a polar molecule, or asking my software developer about psychology.