Earlier quoted context omitted.
oh no, those people without an inner voice are now cowering in a corner...
Everyone has some introspection into their own thoughts, it just takes different forms.
[citation needed]
AGI is an engineering problem, not a model training problem
321–330 of 442 posts
Re: AGI is an engineering problem, not a model training problem
#322Earlier quoted context omitted.
We are trying to get there without a few hundred million years of trial and error. To do that we need to lower the search space, and to do that we do actually need more guiding philosophy and a better understanding of intelligence.
Lower the search space or increase the search speed
Instead what they usually do is lower the fidelity and think they've done what you said. Which results in them getting eaten. Once eaten, they can't learn from mistakes no mo. Their problem.
Because if we don't mix up "intelligence" the phenomenon of increasingly complex self-organization in living systems, with "intelligence" our experience of being able to mentally model complex phenomena in order to interact with them, then it becomes easy to see how the search speed you talk of is already growing exponentially.
In fact, that's all it does. Culture goes faster than genetic selection. Printing goes faster than writing. Democracy is faster than theocracy. Radio is faster than post. A computer is faster than a brain. LLMs are faster than trained monkeys and complain less. All across the planet, systems bootstrap themselves into more advanced systems as soon as I look at 'em, and I presume even when I don't.
OTOH, all the metaphysics stuff about "sentience" and "sapience" that people who can't tell one from the other love to talk past each other about - all that only comes into view if one were to what's happening with the search space if the search speed is increasing at a forever increasing rate.
Such as, whether the search space is finite, whether it's mutable, in what order to search, is it ethical to operate from quantized representations of it, funky sketchy scary stuff the lot of it. One's underlying assumptions about this process determine much of one's outlook on life as well as complex socially organized activities. One usually receives those through acculturation and may be unaware of what they say exactly.
Re: AGI is an engineering problem, not a model training problem
#323If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…
> all the handwavy "engineering" is better done with more data.
How long until that gets more reliable than a simple database? How long until it can execute code faster than a CPU running a program?
A lot of the stuff humans accomplish is through technology, not due to growing a bigger brain. Even something seemingly basic like a math equation benefits drastically from being written down with pen&paper instead of being juggled in the human brain itself (see Extended mind thesis). And when it comes to something like running a 3D engine, there is pretty much no hope of doing it with just your brain.
Maybe we will get AIs smart enough that they can write their own tools, but for that to happen, we still need the infrastructure that allows them writing the tools in the first place. The way they can access Python is a start, but there is still a lack of persistence that lets them keep their accomplishments for future runs, be it in the form of a digital notepad or dynamic updating of weights.
Re: AGI is an engineering problem, not a model training problem
#324If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…
Indeed. The Bitter Lesson has proved true so far. This sounds like going back to the 60s expert systems concept we're trying to get away from. The author also just describes RAG. That certainly isn't AGI, which probably isn't achievable at all.
Re: AGI is an engineering problem, not a model training problem
#325I don't buy that LLMs will get us AGI, they may be part of a system but not the whole.
Re: AGI is an engineering problem, not a model training problem
#326Earlier quoted context omitted.
You spend every waking minute for 20 years or so accumulating training data. You don't learn addition and then independently discover vector calculus.
Individual people don't but we did it as a species. Any purported AGI should be capable of doing the same.
...if you run millions of instances of it for hundreds of thousands of years.
Either the bar of general intelligence set by humans is not very high, or humans are not "generally intelligent" at all. No third option there.
Re: AGI is an engineering problem, not a model training problem
#327I don't buy that LLMs will get us AGI, they may be part of a system but not the whole.
It boils down to whether or not we can figure out how to get LLMs to reliably write code. Something it can already do, albeit still unreliability. If we get there, the industry expectations for "AGI" will be met. The humanoid-like mind that the general public is picturing won't be met by LLMs, but that isn't the bar trying to be met.
Re: AGI is an engineering problem, not a model training problem
#328Earlier quoted context omitted.
> Am I the only one who feels that Claude Code is what they would have imagined basic AGI to be like 10 years ago? That wouldn't have occurred to me, to be honest. To me, AGI is Data from Star Trek. Or at the very least, Arnold Schwarzenegger's character from The Terminator. I'm not sure that I'd make sentience a hard requirement for AGI, but I think my general mental fantasy of AGI even includes sentience. Claude Co…
I would categorize sentient AGI as artificial consciousness[1], but I don't see an obvious reason AGI inherently must be conscious or sentient. (In terms of near-term economic value, non-sentient AGI seems like a more useful invention.) For me, AGI is an AI that I could assign an arbitrarily complex project, and given sufficient compute and permissions, it would succeed at the task as reliably as a competent C-suite…
"Consciousness" and "sentience" are terms mired in philosophical bullshit. We do not have an operational definition of either.
We have no agreement on what either term really means, and we definitely don't have a test that could be administered to conclusively confirm or rule out "consciousness" or "sentience" in something inhuman. We don't even know for sure if all humans are conscious.
What we really have is task specific performance metrics. This generation of AIs is already in the valley between "average human" and "human expert" on many tasks. And the performance of frontier systems keeps improving.
Re: AGI is an engineering problem, not a model training problem
#329I always enjoy discussions that intersect between psychology and engineering.
But I feel this person falls short immediately, because they don't study neuroscience and psychology. That is the big gap in most of these discussion. People don't discuss things close to the origin.
We have to account for first principals in how intelligence works, starting from the origin of ideas and how humans process their ideas in novel ways that create amazing tech like LLM! :D
How Intelligence works
In Neuroscience, if you try to identify the origin of where and how thoughts are formed and how consciousness works. It is completely unknown. This brings up the argument, do humans have free will if we are driven by these thoughts of unknown origin? That's a topic for another thread.
Going back to intelligence. If you study psychology and what forms intelligence, there are many human needs that drive intelligence, namely intellectual curiosity (need to know), deprivation sensitivity (need to understand), aesthetic sensitivity, absorption, flow, openness to experience.
When you look at how a creative human with high intelligence uses their brain, there are 3 networks involved. Default mode network (imagination network), executive attention network and salience network.
The executive attention network controls the brains computational power. It has a working memory that can complete tasks using goal directed focus.
A person with high intelligence can alternate between their imagination and their working memory and pull novel ideas from their imagination and insert them into their working memory - frequently experimenting by testing reality. The salience network filters unnecessary content while we are using our working memory and imagination.
How LLMs work
Neural networks are quite promising in their ability to create a latent manifold within large datasets that interpolates between samples. This is the basis for generalization, where we can compress a large dataset in a lower dimensional space to a more meaningful representation that makes predictions.
The advent of attention on top of neural networks, to identify important parts of text sequences, is the huge innovation powering llms today. The innovation that emulates the executive attention network.
However, that alone is a long distance from the capabilities of human intelligence.
With current AI systems, there is the origin, which is known vocabulary with learned weights coming from neural networks, with reinforcement learning applied to enhance the responses.
Inference comes from an autoregressive sequence model that processes one token at a time. This comes with a compounding error rate with longer responses and hallucinations from counterfactuals.
Correct response must be in the training distribution.
As Andy Clark said, AI will never gain human intelligence, they have no motivation to interface with the world and conduct experiments and learn things on their own.
I think there are too many unknown and subjective components of human intelligence and motivation that cannot be replicated with the current systems.
Re: AGI is an engineering problem, not a model training problem
#330Earlier quoted context omitted.
Nah. The real philosophical headache is that we still haven’t solved the hard problem of consciousness, and we’re disappointed because we hoped in our hearts (if not out loud) that building AI would give us some shred of insight into the rich and mysterious experience of life we somehow incontrovertibly perceive but can’t explain. Instead we got a machine that can outwardly present as human, can do tasks we had thoug…
I think Metzinger nailed it, we aren't conscious at all. We confuse the map for the territory in thinking the model we build to predict our other models is us. We are a collection of models a few of which create the illusion of consciousness. Someone is going to connect a handful of already existing models in a way that gives an AI the same illusion sooner rather than later. That will be an interesting day.
> The illusion of consciousness"
So you think there is "consciousness", and the illusion of it? This is getting into heavy epistemic territory.
Attempts to hand-wave away the problem of consciousness are amusing to me. It's like an LLM that, after many unsuccessful attempts to fix code to pass tests, resorts to deleting or emasculating the tests, and declares "done"