AI Doomers are worse than wrong – they're incompetent

A couple of OpenAI board members isn’t a “social movement”. Big strawman article with nothing really insightful to add

Openai employees had extreme leverage because they were (or at least appeared, by the number of signatures on the letter) united. They could just leave for another company (Microsoft or other) where they would not be at all controlled by AI safety people.

Openai investors had only the leverage of "we won't invest more and you're running an operating loss", they did not actually have a case in court. The investment contract they signed stated "view your investment as a donation" and "this investment confers no fiduciary duties from the board".

The board had the leverage of "you employees and investors value your stock holdings, and if the company fails, those stock holdings to to zero".

The resolution I think was (could be wrong) just to initially reduce the size of the board to three. So it isn't completely guaranteed which side won the battle for control: it's not impossible that the AI safety people might still control the board. Maybe one or both of the new board members is secretly convinced by the AI safety movement.

The battle for public opinion is a different battle, and it looks extremely likely that the AI safety people lost that battle. Maybe it was still worth it for them, if they did in fact win or at least stalemate the battle for control of openai.

I do think there's some risk of self propagation that comes when we start hooking these LLMs up to publicly open APIs, but I'm not too worried yet. And even then, I don't think that would automatically make it any more capable. Similarly, it seems like we're not too far from an LLM that could automate simple cyber attacks, which would probably lower the barrier to entry, but might not be much worse then existing similar tooling. I guess where we might get into trouble is a combination of the two, something prompted to propagate and hack wherever it can (which still wouldn't be any more intelligent). Did anyone think the Morris worm was possible until it happened?

Edit: ok, off topic, but apparently Robert Morris is friends with Paul Graham and a part of YC? I had no idea lol.

This piece would be a lot better is it didn't spend half the time piling ad-hominem attacks on "doomers". If you see this episode through an AGI-safety lense, it realized/reflects a type of risk that has been very extensively articulated/predicted by the safety community.

Artificial intelligence is for those without natural intelligence.