OpenAI’s head of ethics leaves less than a year after joining • Hacker News Board

Re: OpenAI’s head of ethics leaves less than a year after joining

#431

Post by close04 » Wed, Aug 12, 2026, 7:46 AM UTC

hnlmorg wrote: post #410

BloondAndDoom wrote: post #402

Earlier quoted context omitted.

I get what you are saying but that’s exactly how security and compliance work. And I don’t think amount of people who want to be ethical in an organization ia that different than people who want to build secure software (but possibly no one gives a shit compliance other than it’s something that needs to be done)

And organisations fail at security when security and compliance is only considered important by that one teamnn Security requires the whole business to buy in. And it requires processes that allow people to get shit done without people resorting to shadow IT; thus working around that one team. So the GPs point still works.

The difference between "ethics" and "security and compliance" is that the latter is something that hits inside the company, while the former usually hits outside of the company.

Poor security practices harm your teams, your data, and usually you make moderate savings at best. Poor ethics "only" harm your customers while making bank for the company.

This is the real problem with ethics in a large corporation. You're not saying "no" to another team, you're saying no to large profits, you're saying no to the company's leadership. That is what never works.

Re: OpenAI’s head of ethics leaves less than a year after joining

#432

Post by wongarsu » Wed, Aug 12, 2026, 8:01 AM UTC

saghm wrote: post #325

teiferer wrote: post #250

Earlier quoted context omitted.

> if the model requires retraining for ethics reasons. Haha, how refreshingly naive. What do you think is more likely: this or the other way around: getting the "ethics team" to align with investor goals? Hint: Where does the money come from? > The people in AI ethics who spent years thinking their job was marketing or publishing thinkpiece papers may be having to adapt quickly or get out of the field. Or it's exactl…

Yeah, I've never found the "maximizing shareholder profit" imperative to be compelling as something that should be done, but as a predictor of how companies will act, it's not something I ever feel safe betting against.

One subtlety that is worth considering is that what a company is inherently set up to do is maximize shareholder value. Value can be quite complex, especially while the number of shareholders is small. Value can include the legacy or reputation of the shareholders. It can include the ethics of the shareholders. It can include completely different goals, like the famous "make humans a multi-planetary species".

Treating value as purely monetary profit is a perversion of that. To a certain degree it's inevitable as the number of shareholders grows, so seeing it in public companies isn't surprising. But even then, if a company focused on creating equipment for sustainable, pesticide free farming were to pivot into making Hellfire missiles that might be insanely profitable while still destroying a large part of what existing shareholders value about the company

Re: OpenAI’s head of ethics leaves less than a year after joining

#433

Post by RandomLensman » Wed, Aug 12, 2026, 8:02 AM UTC

danlitt wrote: post #74

RandomLensman wrote: post #19

Earlier quoted context omitted.

The people and procedures inside are the backstop.

How many leadership meetings have you been in where some profitable approach was refused on an ethical basis? (rather than the basis that behaving ethically is actually more profitable, which is totally different) By contrast, how many conversations have you personally had with people working at a company believing a profitable action was firmly unethical, but felt forced to behave that way due to "business realities…

I think in the end, could always reframe any moral decision as risk management but not sure that is ultimately helpful. I have certainly seen decisions that were taken because it didn't "sit" right to do something (profitable). I'd also say that I feel older (as in existing longer) businesses tend to be on average better there.

On the second point: my experience is rather that there is quite a variance in what is considered moral vs immortal and that often drives different decisions (in the same place, some do certain things, other don't and certain experise/people might not be available for X therefore). I cannot say that I have come across many situations where people did something they considered firmly immoral but felt they needed to go along with it (trying to recall some). Yes, finding justifications happens, too, but that isn't necessarily whitewashing as people do disagree to surprising levels.

(Changed to moral vs immoral, because that was the starting point.)

Re: OpenAI’s head of ethics leaves less than a year after joining

#434

Post by Planktonne » Wed, Aug 12, 2026, 8:05 AM UTC

CursedSilicon wrote: post #381

Planktonne wrote: post #232

Earlier quoted context omitted.

Why only ask for citations of one side's claims?

If someone says "the sky is blue" and someone else says "actually the sky is polka dots!" are you going to ask for citations from both?

In that case I wouldn't ask for citations from either, because I have direct evidence.

But with two competing claims, and no easy way to check ground truth, you absolutely ask for citations from both sides. There's nothing rational about uncritically accepting the narrative you heard first.

Re: OpenAI’s head of ethics leaves less than a year after joining

#435

Post by vintermann » Wed, Aug 12, 2026, 8:12 AM UTC

I'm a broken record on this, but I'd like to reiterate:

1. Academic ethics is the theoretical study of what you can get away with. (Law is the applied branch). Professors in ethics are not experts at being good, or knowing what is good, they're experts in how to get away with things. If you want an expert in being good, you need a saint, not an academic studying ethics. Saints are in short supply.

2. That said, ethics academics can still be decent. They don't have to be sanctimonious corporate shills . However, an "ethics expert" who hasn't threatened to quit and gone through with it at least once, should be assumed to be one.

I respect those who quit - or, at least, I respect them more. I wonder if it's more a reflection of the people, or of the state of the industry, that the list of (AI) ethics experts who have quit in protest is getting long.

Re: OpenAI’s head of ethics leaves less than a year after joining

#436

Post by hnlmorg » Wed, Aug 12, 2026, 8:14 AM UTC

close04 wrote: post #431

hnlmorg wrote: post #410

Earlier quoted context omitted.

And organisations fail at security when security and compliance is only considered important by that one teamnn Security requires the whole business to buy in. And it requires processes that allow people to get shit done without people resorting to shadow IT; thus working around that one team. So the GPs point still works.

The difference between "ethics" and "security and compliance" is that the latter is something that hits inside the company, while the former usually hits outside of the company. Poor security practices harm your teams, your data, and usually you make moderate savings at best. Poor ethics "only" harm your customers while making bank for the company. This is the real problem with ethics in a large corporation. You're n…

> The difference between "ethics" and "security and compliance" is that the latter is something that hits inside the company, while the former usually hits outside of the company.

I don’t agree with this. Data breaches affect customers more than businesses. If your point were true, we’d see fewer breaches. Plus not all breaches are a result of software engineering teams. For example product managers sharing customer details.

I’ve managed plenty of teams where I’ve had to instil the importance of secure best practices at all stages of development. So it’s definitely not something inherently important to all people who work in organisations.

Just like with ethics. It’s very easy to dismiss either as an inconvenience if you don’t instil the right company culture at all levels of the organisation.

This is why European financial organisations have such strict onboarding procedures to teach new hires about fraud, bribery and other financial misconduct even for issues that are ethical grey rather than outright illegal. Similarly many organisations will have onboarding procedures to teach new hires their security best practices too

Re: OpenAI’s head of ethics leaves less than a year after joining

#437

Post by chii » Wed, Aug 12, 2026, 8:15 AM UTC

close04 wrote: post #431

hnlmorg wrote: post #410

Earlier quoted context omitted.

And organisations fail at security when security and compliance is only considered important by that one teamnn Security requires the whole business to buy in. And it requires processes that allow people to get shit done without people resorting to shadow IT; thus working around that one team. So the GPs point still works.

The difference between "ethics" and "security and compliance" is that the latter is something that hits inside the company, while the former usually hits outside of the company. Poor security practices harm your teams, your data, and usually you make moderate savings at best. Poor ethics "only" harm your customers while making bank for the company. This is the real problem with ethics in a large corporation. You're n…

This is exactly why ethics doesnt work, and it is why laws work. If ethics worked, we'd have no need for laws.

Re: OpenAI’s head of ethics leaves less than a year after joining

#438

Post by torginus » Wed, Aug 12, 2026, 8:15 AM UTC

madrox wrote: post #247

In five years, I'd love to read a book about the history of AI ethics. I suspect it will read like Voltaire. My sense is that it is radically shifting from a fluffy marketing arm to a department expected to contribute meaningfully to development and justify its impact. I would expect an ethics team to build frameworks that can help train/eval the model that the company spend millions of dollars and months training is…

Imo 'AI ethics' in this context was code for 'how to make our platform advertiser friendly'

Re: OpenAI’s head of ethics leaves less than a year after joining

#439

Post by mchinen » Wed, Aug 12, 2026, 8:22 AM UTC

BloondAndDoom wrote: post #402

seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

I get what you are saying but that’s exactly how security and compliance work. And I don’t think amount of people who want to be ethical in an organization ia that different than people who want to build secure software (but possibly no one gives a shit compliance other than it’s something that needs to be done)

SRE is a 'say no to powerful people' job as well. I think for this to work the leadership needs to show support for it. The friction is still there, but in more tolerable areas. I bet for ethics this isn't the case at OAI, but everyone values security and stability.

For an SRE there can be more directed hate received from the junior employees, that want to release new features they developed. Especially because there is less accountability across orgs. Security is an interesting one because it seems to have less of this friction, maybe because it's more clear cut what is an issue.

Re: OpenAI’s head of ethics leaves less than a year after joining

#440

Post by close04 » Wed, Aug 12, 2026, 8:32 AM UTC

hnlmorg wrote: post #436

close04 wrote: post #431

Earlier quoted context omitted.

The difference between "ethics" and "security and compliance" is that the latter is something that hits inside the company, while the former usually hits outside of the company. Poor security practices harm your teams, your data, and usually you make moderate savings at best. Poor ethics "only" harm your customers while making bank for the company. This is the real problem with ethics in a large corporation. You're n…

> The difference between "ethics" and "security and compliance" is that the latter is something that hits inside the company, while the former usually hits outside of the company. I don’t agree with this. Data breaches affect customers more than businesses. If your point were true, we’d see fewer breaches. Plus not all breaches are a result of software engineering teams. For example product managers sharing customer…

I think we're both thinking of different examples and drawing wider conclusions. I'm thinking of a company losing the data that makes it money, you're thinking of losing the customer data and thus their trust. Probably equally valid points of view.

The lack of ethics hugely contributes to companies collecting more and more user data that they normally shouldn't have. This makes the data a more attractive target.

> If your point were true, we’d see fewer breaches.

But this doesn't follow. There are way more factors at play that influence the number of attacks and the number of successes. Companies hold more and more data with ever higher value (so more liability), and hacking tools and hacker determination advanced faster than defensive measures. The result is expected and the solution isn't only "more security", but also "hold less data".

While security is a double edged sword, ethics had been proven to be very single edged. History shows that for startups and large companies alike, the lack of ethics is actually a competitive advantage for the company.