Before Altman’s ouster, OpenAI’s board was divided and feuding

> Non-profit corporations exist to further their mission, and are accountable to the IRS in this regard.

The IRS’s actual power in this regard is extremely limited. There’s so much wiggle room to define the “mission” that, at best, the only thing they can do is make sure you’re not secretly using your nonprofit as a tax shelter for a regular for-profit business, and even then some companies find ways around it (see IKEA).

Ordinary fraud (as opposed to tax fraud, though maybe that too) is even easier to pull off with a nonprofit than with an for-profit. Any nonprofit has a fundraising mechanism (which is usually an obnoxious set of dark patterns and outright spam reached through decades of optimization; the techniques are well understood), some overhead for managing the organization and money itself, and then the actual work involved with the mission. It’s quite simple for the overhead and fundraising parts of the organization to become most of the organization. But it gets worse: when your “mission” is vague enough, you can also turn that side of the business into a bunch of sinecures for your friends. There are well known “awareness” nonprofits that do literally nothing to actually solve the problems they’re nominally about because their job is to “spread awareness” (i.e. the top of the funnel for the fundraising part of the organization).

This is ironically also the origin story of “effective altruism”. The IRS doesn’t stop you from running a non-profit scam, so you have to do your own research to find out which non-profits are legit, and that’s what the effective altruists started off with, along with some cost-benefit modeling once you get past the outright scams.

The problem with OpenAI seems to be related to a vague mission as well. From the “an organization is what it does” perspective, OpenAI develops AI. If you have a formal mission that emphasizes safety over capability, you could certainly argue that OpenAI was closer to achieving that mission by sabotaging its own work, but if your interpretation of “safety” is to sabotage the development of AI, there are certainly better ways to do it than to develop AI and then sabotage yourself.

I think the difference is this: if a tech exec thinks something is good for the humanity and releases it as a product, I can choose not to use it. On the other hand, if Toner kills OpenAI because she thinks AI is dangerous, I won't be able to use OpenAI's product any more. And of course, I don't think someone who does not work on tech or STEM in general can have any insights on machine learning.

???

Where did I say anything about a "superintelligence"?

The extant statistical inference engines aren't going to build killer androids. No, I'm terrified of what profit-motivated humans are going to do to each other using machines designed to match patterns and generate convincing bullshit.

Think less "Terminator", more like Cambridge Analytica, Russian-style firehose-of-falsehood, 2018 YouTube algorithmic radicalization, 2014 Facebook "emotional contagion" experiment, 2023 TikTok censorship, doctored videos/evidence, crossing an inflection point in declining trust in democratic institutions, fully automated fraud and identity theft in an arms race with increasingly invasive countermeasures, scientific atrophy, artistic and cultural homogenization, etc.

FFS, the SGA and WGA just wrapped up an unprecedented strike motivated in large part by the broadly anticipated destructive impact of ML on their industry. You wanna talk about jobs? Wikipedia says that cost 45,000, many of whom were barely getting by, while you whinge about 770 in-demand professionals who already have highly paying offers from at least two Fortune 500 companies. From what I've seen, the voice acting industry, although less able to advocate for itself, is in disarray too and already directly impacted by the spread of models trained unconsensually on actors' past work. Plus major publications like CNET have already been caught canning their staff and dishonestly publishing error-ridden drivel with a misguided faith in LLMs, Google and thus access to most textual information online is basically useless because of SEO blogspam already, etc.

Framing climate change as "we didn't move fast enough on AGI" is… Bizarre. Windmills ain't new tech, and neither are hydro dams, nuclear reactors, nor LRT. Framing disease as an "AGI" problem is… Delusional, as well. Some academics may certainly find domain-specific ML models useful, but were I a bit more vindictive, I'd pay good money to see you try to get ChatGPT to fold a novel protein correctly.

> It's all a game of "how can I justify doing heinous shit to people".

Who cares what he thinks, his money is gone and he has no control over it.

In fact the most positive outcome is if Altman and the rest of the staff went to MS and did their thing and OpenAI started from scratch with the $13B they've come into. That would double the chances of something useful emerging from the OpenAI work so far.

I personally think Altman is very much less than a genius (maybe at extracting financial advantage) so all OpenAI's eggs shouldn't be placed in that particular basket.

> Pandora’s box is already open. You have to guard it. You have to use your power and influence to enforce other competitors to also do that with their own boxes.

> Self-destructing is the worst way to ensure AI safety.

If they don't believe you're prepared to destroy the company if that's what it takes, then you have zero power or influence. If they try to "call your bluff", you have to go there, otherwise they'll never respect you again.