I find this line of thinking to be
extremely indicative of the problem, I'd bet, OpenAI was trying to get rid of with Sam.
Here's something to ponder on: The human brain is about the size of an A100, and consumes: 12 watts of power, on average. Its capable of general intelligence and conscious thought.
One problem that companies have is: they're momentum based. Once they realize something is working, and generating profit, they become increasingly calcified toward trying fundamentally new and different things. The best case scenario for a company is to calcify at a local maxima. A few companies try to structure themselves toward avoiding this, like Google; and it turns out, they just lose the ability to execute on anything. Some will stay small, remain nimble, and accomplish little of note. The rest die. That's the destiny for every profit-focused company.
Here's three things I expect to be true: AGI/ASI won't be achieved with LLMs. A sufficiently powerful LLM may be a component of a larger AGI/ASI system, but GPT-4 is already pretty dang sufficiently powerful. And: OpenAI was becoming an extremely effective and successful B2B SaaS Big Tech LLM company. Outing Sam is a gambit; the company could implode, and with no one left AGI/ASI probably won't happen at OpenAI. But the alternative, it seems from the outside, had a higher probability of failure; because the company would become so successful and good at making LLMs that the non-profit's mission is put to the side.
Ilya's superalignment efforts were given 20% of OpenAI's compute capacity. If the foundation's goal is to produce safe AGI; and ideally, you want progress on safety before something unsafe is made; it seems to me that 51% is the totally symbolic but meaningful minimum he should be working with. That's just one example.