The two guys from Google get to set the rules? How will they measure wisdom or common sense (ability to make an exception)? https://youtu.be/lA-zdh_bQBo
They are not the rules. They are some rules.
Measuring progress toward AGI: A cognitive framework
11–20 of 229 posts
Re: Measuring progress toward AGI: A cognitive framework
#12It's good to have some kind of benchmark at least to structure the ongoing, fruitless discussion around "are we there already?".
However I must admit that including the last point that is partially hinting at the emotional or rather social intelligence surprised me. It makes this list go beyond usual understanding of AGI and moves it toward something like AGI-we-actually-want. But for that purpose this last point isn't ok narrow, too specific. And so is the whole list.
To be actually useful the AGI-we-actually-want benchmark should not only include positive indicators but also a list of unwanted behaviors to ensure this thing that used to be called alignment I guess.
Re: Measuring progress toward AGI: A cognitive framework
#13Social cognition: processing and interpreting social information and responding appropriately in social situations
Is social cognition really a measure of intelligence for non-social entities?
Re: Measuring progress toward AGI: A cognitive framework
#14Social cognition: processing and interpreting social information and responding appropriately in social situations Is social cognition really a measure of intelligence for non-social entities?
It is not. Why is that relevant to social entities?
Re: Measuring progress toward AGI: A cognitive framework
#15AGI feels like a vanity project.
Who cares about AGI? Honestlky what's the gain.
Maybe Google could actually make Gemini good instead of being about 10 miles behind Claude instead of trying to make AGI because of - well some reason - cause they want to be famous.
Re: Measuring progress toward AGI: A cognitive framework
#16Social cognition: processing and interpreting social information and responding appropriately in social situations Is social cognition really a measure of intelligence for non-social entities?
It is not. Why is that relevant to social entities?
How well you interact with other members of a society increases your chances of procreation, survival, knowledge acquisition, ie. it makes sense as a measure of intelligence
Re: Measuring progress toward AGI: A cognitive framework
#17I'm sorry what even is this? Giving $10k rewards for significant advancements toward "AGI"?
What does "making a framework" even mean, it feels like a nothing post.
When I think of what real AGI would be I think:
- Passes the turing test
- Writes a New York Times Bestseller without revealing it was written by AI
- Writes journal articles that pass peer review
- Wins a Nobel Prize
- Writes a successful comedy routine
- Creates a new invention
And no, nobody is going to make an automated kaggle benchmark to verify these. Which is fine, because an LLM will never be AGI. An LLM can't even learn mid-conversation.
Re: Measuring progress toward AGI: A cognitive framework
#18I'm sorry what even is this? Giving $10k rewards for significant advancements toward "AGI"? What does "making a framework" even mean, it feels like a nothing post. When I think of what real AGI would be I think: - Passes the turing test - Writes a New York Times Bestseller without revealing it was written by AI - Writes journal articles that pass peer review - Wins a Nobel Prize - Writes a successful comedy routine -…
They’re slowly redefining AGI so they can use it for more marketing. If you showed someone from 1960 our LLMs from and told them “this is AI” I think they’d be astounded but a little confused because “artificial intelligence” definitely carried a very clear meaning in literature and media. Now it is marketing terminology and we’re no closer to having a meaningful definition for the word intelligence.
Re: Measuring progress toward AGI: A cognitive framework
#19I'm sorry what even is this? Giving $10k rewards for significant advancements toward "AGI"? What does "making a framework" even mean, it feels like a nothing post. When I think of what real AGI would be I think: - Passes the turing test - Writes a New York Times Bestseller without revealing it was written by AI - Writes journal articles that pass peer review - Wins a Nobel Prize - Writes a successful comedy routine -…
Grok recently created a cancer vaccine for a dog that reduced tumor size by 75%
Re: Measuring progress toward AGI: A cognitive framework
#20When people imagined AI/AGI, they imagined something that can reason like we can, except at the speed of a computer, which we always envisioned would lead to the singularity. In a short period of time, AI would be so far ahead of us and our existing ideas, that the world would become unrecognizable.
That's not what's happening here, and it's worth remembering: A caveman from 200K years ago would have been just as intelligent as any of us here today, despite not having language or technology, or any knowledge.
In Carolyn Porco's words: "These beings, with soaring imagination, eventually flung themselves and their machines into interplanetary space."
When you think of it that way, it should be obvious that LLMs are not AGI. And that's OK! They're a remarkable piece of technology anyway! It turns out that LLMs are actually good enough for a lot of use cases that would otherwise have required human intelligence.
And I echo ArekDymalski's sentiment that it's good to have benchmarks to structure the discussions around the "intelligence level" of LLMs. That _is_ useful, and the more progress we make, the better. But we're not on the way to AGI.