Earlier quoted context omitted.
It strikes me that until we fully understand human consciousness, we don't stand a chance of reaching AGI. Am I incorrect?
Intelligence (solving problems) does not require consciousness.
Please elaborate.
AGI is an engineering problem, not a model training problem
81–90 of 442 posts
Re: AGI is an engineering problem, not a model training problem
#82Earlier quoted context omitted.
Would that really be a physics discovery? I mean I guess everything ultimately is. But it seems like maybe consciousness could be understood in terms of "higher level" sciences - somewhere on the chain of neurology->biology->chemistry->physics.
That sounds like you’re describing AGI as being impractical to implement in an electronic computer, not impossible in principle.
Yeah, I guess I'm not taking a stance on that above, just wondering where in that chain holds the most explanatory power for intelligence and/or consciousness.
I don't think there's any real reason to think intelligence depends on "meat" as its substrate, so AGI seems in principle possible to me.
Not that my opinion counts for much on this topic, since I don't really have any relevant education on the topic. But my half baked instinct is that LLMs in and of themselves will never constitute true AGI. The biggest thing that seems to be missing from what we currently call AI is memory - and it's very interesting to see how their behavior changes if you hook up LLMs to any of the various "memory MCP" implementations out there.
Even experimenting with those sorts of things has left me feeling there's still something (or many somethings) missing to take us from what is currently called "AI" to "AGI" or so-called super intelligence.
Re: AGI is an engineering problem, not a model training problem
#83won’t somebody please think about Mr. Godel, and the Incompleteness Theorem ?
Re: AGI is an engineering problem, not a model training problem
#84It's a training data problem, and the training data you want doesn't exist.
Re: AGI is an engineering problem, not a model training problem
#85Earlier quoted context omitted.
We would also need a definition of AGI that is provable or disprovable. We don’t even have a workable definition, never mind a machine.
We don’t need such a definition of general intelligence to conclude that biological humans have it, so I’m not sure why we’d such a definition for AGI.
Well general intelligence in humans already exists, whereas general intelligence doesn't yet exist in machines. How do we know when we have it? You can't even simply compare it to humans and ask "is it able to do the same things?" because your answer depends on what you define those things to be. Surely you wouldn't say that someone who can't remember names or navigate without GPS lacks general intelligence, so it's necessary to define what criteria are absolutely required.
Re: AGI is an engineering problem, not a model training problem
#86Am I the only one who feels that Claude Code is what they would have imagined basic AGI to be like 10 years ago? It can plan and take actions towards arbitrary goals in a wide variety of mostly text-based domains. It can maintain basic "memory" in text files. It's not smart enough to work on a long time horizon yet, it's not embodied, and it has big gaps in understanding. But this is basically what I would have expec…
Claude code is neither sentient nor sapient.
I suspect most people envision AGI as at least having sentience. To borrow from Star Trek, the Enterprise's main computer is not at the level of AGI, but Data is.
The biggest thing that is missing (IMHO) is a discrete identity and notion of self. It'll readily assume a role given in a prompt, but lacks any permanence.
Re: AGI is an engineering problem, not a model training problem
#87https://mashable.com/article/apple-research-ai-reasoning-mod...
All of our current approaches "emulate" but do not "execute" general intelligence. The damning paper above basically concludes they're incredible pattern matching machines, but thats about it.
Re: AGI is an engineering problem, not a model training problem
#88Earlier quoted context omitted.
> We don't know if AGI is even possible outside of a biological construct yet. This is key. A discovery that AGI is impossible in principle to implement in an electronic computer would require a major fundamental discovery in physics that answers the question “what is the brain doing in order to implement general intelligence?”
It is vacuously true that a Turing machine can implement human intelligence: simply solve the Schrödinger equation for every atom in the human body and local environment. Obviously this is cost-prohibitive and we don’t have even 0.1% of the data required to make the simulation. Maybe we could simulate every single neuron instead, but again it’ll take many decades to gather the data in living human brains, and it woul…
> It is vacuously true that a Turing machine can implement human intelligence
The case of simulating all known physics is stronger so I'll consider that.
But still it tells us nothing, as the Turing machine can't be built. It is a kind of tautology wherein computation is taken to "run" the universe via the formalism of quantum mechanics, which is taken to be a complete description of reality, permitting the assumption that brains do intelligence by way of unknown combinations of known factors.
For what it's worth, I think the last point might be right, but the argument is circular.
Here is a better one. We can/do design narrow boundary intelligence into machines. We can see that we are ourselves assemblies of a huge number of tiny machines which we only partially understand. Therefore it seems plausible that computation might be sufficient for biology. But until we better understand life we'll not know.
Whether we can engineer it or whether it must grow, and on what substrates, are also relevant questions.
If it appears we are forced to "go numerically", as you say, it may just indicate that we don't know how to put the pieces together yet. It might mean that a human zygote and its immediate environment is the only thing that can put the pieces together properly given energetic and material constraints. It might also mean we're missing physics, or maybe even philosophy: fundamental notions of what it means to have/be biological intelligence. Intelligence human or otherwise isn't well defined.
Re: AGI is an engineering problem, not a model training problem
#89We don't know if AGI is even possible outside of a biological construct yet. This is key. Can we land on AGI without some clear indication of possibility (aka Chappie style)? Possibly, but the likelihood is low. Quite low. It's essentially groping in the dark. A good contrast is quantum computing. We know that's possible, even feasible, and now are trying to overcome the engineering hurdles. And people still think th…
> We don't know if AGI is even possible outside of a biological construct yet. This is key. A discovery that AGI is impossible in principle to implement in an electronic computer would require a major fundamental discovery in physics that answers the question “what is the brain doing in order to implement general intelligence?”
Not necessarily. It could simply be a question of scale. Being analog and molecular means that brain could be doing enormously more than any foreseeable computer. For a simple example what if every neuron is doing trillions of calculations.
(I’m not saying it is, just that it’s possible)
Re: AGI is an engineering problem, not a model training problem
#90If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…
Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…
> there is missing philosophy
I doubt it. Human intelligence evolved from organisms much less intelligent than LLMs and no philosophy was needed. Just trial and error and competition.