161–170 of 442 posts
https://mashable.com/article/apple-research-ai-reasoning-mod... All of our current approaches "emulate" but do not "execute" general intelligence. The damning paper above basically concludes they're incredible pattern matching machines, but thats about it.
The forgone conclusion that LLMs are the key or even a major step towards AGI is frustrating. They are not, and we are fooling ourselves. They are incredible knowledge stores and statistical machines, but general intelligence is far more than these attributes.
The suggested requirements are not engineering problems. Conceiving of a model architecture that can represent all the systems described in the blog is a monumental task of computer science research.
Take memory for example: give LLM a persistent computer and ask it to jot down its long-term memory as hierarchical directories of markdown documents. Recalling a piece of memory means a bunch of `tree` and `grep` commands. It's very, very rudimentary, but it kinda works, today. We just have to think of incrementally smarter ways to query & maintain this type of memory repo, which is a pure engineering problem.
If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…
Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…
And now, I still don't know; the months go by and as far as I'm aware they're still pursuing these goals but I wonder how much conviction they still have.
Earlier quoted context omitted.
> AGI, by definition, in its name Artificial General Intelligence implies / directly states that this type of AI is not some dumb AI that requires training for all its knowledge, a general intelligence merely needs to be taught how to count, the basic rules of logic, and the basic rules of a single human language. From those basics all derivable logical human sciences will be rediscovered by that AGI That's not how n…
Are you sure? Do you require dozens, to hundreds, to thousands of examples before you understand a concept? I expect no. That is because you have comprehension that can generalize a situation to basic concepts which you apply to other situations without effort. You comprehend. AI cannot do that: get the idea from a few, under a half dozen examples if necessary. Often a human needs 1-3 examples before they can general…
"AGI needs to update beliefs when contradicted by new evidence" is a great idea, however, the article's approach of building better memory databases (basically fancier RAG) doesn't seem enable this. Beliefs and facts are built into LLMs at a very low layer during training. I wonder how they think they can force an LLM to pull from the memory bank instead of the training data.
(Also, LLMs don't have beliefs or other mental states. As for facts, it's trivially easy to get an LLM to say that it was previously wrong ... but multiple contradictory claims cannot all be facts.)
The suggested requirements are not engineering problems. Conceiving of a model architecture that can represent all the systems described in the blog is a monumental task of computer science research.
Nah,this sounds like a modern remix of Japan’s Fifth Generation Computing project. They thought that by building large databases and with Prolog they would bring upon an AI renaissance. Just hand waving some “distributed architecture” and trying to duct tape modules together won’t get us any closer to AGI. The building blocks themselves, the foundation, has to be much better. Arguably the only building block that LLM…
Probably need another cycle of similar breakthrough in model engineering before this more complex neural network gets a step function better.
Moar data ain’t gonna help. The human brain is the proof: it doesnt need the internet’s worth of data to become good (nor all that much energy).
If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…
Earlier quoted context omitted.
We don’t need such a definition of general intelligence to conclude that biological humans have it, so I’m not sure why we’d such a definition for AGI.
I disagree. We claim that biological humans have general intelligence because we are biased and arrogant, and experience hubris. I'm not saying we aren't generally intelligent, but a big part of believing we are is because not believing so would be psychologically and culturally disastrous. I fully expect that, as our attempts at AGI become more and more sophisticated, there will be a long period where there are inte…
No, we say it because - in this context - we are the definition of general intelligence.
Approximately nobody talking about AGI takes the "G" to stand for "most general possible intelligence that could ever exist." All it means is "as general as an average human." So it doesn't matter if humans are "really general intelligence" or not, we are the benchmark being discussed here.