> Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do > It can be debated whether arena.ai is a suitable metric for AGI, a strong case can probably be made for why it’s not. However, that’s irrelevant, as the spirit of the self-sacrifice clause is to avoid an arms race, and we are clearly in one. No, the spirit is clearly meant for near AGI and we aren’t near AGI
The changing goalposts of AGI and timelines
21–30 of 411 posts
Re: The changing goalposts of AGI and timelines
#22You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summarizer stuffed into a context window (which is like having someone take an entire physics course, writing down everything they learn on post-it notes, then you ask a different person a physics question, and that different person has to skim all the post-it notes, and then write a new post-it note to answer you). To learn it would need RL (which requires specific novel inputs) and retraining (so that it can retain and compute answers with the learned input). This would all take too much time and careful input/engineering along with novel techniques. So AGI is too expensive, time consuming, and difficult for us to achieve without radically different designs and a whole lot more effort.
Not only are LLMs not AGI, they're still not even that great at being LLMs. Sure, they can do a lot of cool things, like write working code and tests. But tell one "don't delete files in X/", and after a while, it will delete all the files in "X/", whereas a human would likely remember it's not supposed to delete some files, and go check first. It also does fun stuff like follow arbitrary instructions from an attacker found in random documents, which most humans also wouldn't do. If they had a real memory and RL in real-time, they wouldn't have these problems. But we're a long way away from that.
LLMs are fine. They aren't AGI.
Re: The changing goalposts of AGI and timelines
#23The way Sam Altman bungled the Pentagon deal by swooping in a few hours after Anthropic was fired should be grounds for OpenAI finding another CEO.
Why? Give it a couple weeks and everybody will forget about this. They'll be earning more money than previously. Job well done.
Re: The changing goalposts of AGI and timelines
#24Re: The changing goalposts of AGI and timelines
#25The way Sam Altman bungled the Pentagon deal by swooping in a few hours after Anthropic was fired should be grounds for OpenAI finding another CEO.
the whole public debacle was planned, the tos isn't stopping the pentagon from doing anything (as we seen with openai now)
Re: The changing goalposts of AGI and timelines
#26"Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project."
I claim that currently no "value-aligned, safety-conscious project comes close to building AGI", both for the reasons
- "value-aligned, safety-conscious" and
- "close to building AGI".
So, based on this charter, OpenAI has no reason to surrender the race.
Re: The changing goalposts of AGI and timelines
#27Hah can you imagine a world where OpenAi says to all the people who have dumped billions in : "well we lost guys, sorry about that, were just gonna help Google now". I'll eat my hat after I sell you a bridge.
Gemini might be great at benchmarks, it is terrible at actual agentic coding. So Anthropic seems like a more logical choice.
Re: The changing goalposts of AGI and timelines
#28One can argue that they have already achieved this. At least for short termed tasks. Humans are still better at organization, collaboration and carrying out very long tasks like managing a project or a company.
Re: The changing goalposts of AGI and timelines
#29The way Sam Altman bungled the Pentagon deal by swooping in a few hours after Anthropic was fired should be grounds for OpenAI finding another CEO.
Re: The changing goalposts of AGI and timelines
#30> The impotence of naive idealism in the face of economic incentives. I don't think it was so much the naivety of idealism, but more an adoption of idealism and related language to help market what was actually being built: a profit-first organization that's taking its true form little by little.