Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s head of ethics leaves less than a year after joining

ft.com

Post reply on HN
Print view View original

481–490 of 494 posts

Re: OpenAI’s head of ethics leaves less than a year after joining

#481

Post by SkyPuncher »

plaidfuji wrote: post #457
seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

If this were true, food companies wouldn’t have a Regulatory team. Except every single one of them does, and they function similarly to how you describe, but without dissolving. They’re treated as a constraint that must be checked before major projects can advance. Sometimes they’re the longest-lead item on a new project and if you don’t get them involved early, your project can sink at a late stage after great expen…

I don’t view that team as saying “No”.

Instead, I view that team as saying “Customers expect safe food. Our job is to help you deliver on that”.

Re: OpenAI’s head of ethics leaves less than a year after joining

#482

Post by hbn »

plaidfuji wrote: post #457
seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

If this were true, food companies wouldn’t have a Regulatory team. Except every single one of them does, and they function similarly to how you describe, but without dissolving. They’re treated as a constraint that must be checked before major projects can advance. Sometimes they’re the longest-lead item on a new project and if you don’t get them involved early, your project can sink at a late stage after great expen…

Food companies aren't rapidly developing groundbreaking new technologies on the most competitive battleground in computing history where everyone is trying to compete globally just to create the foundational infrastructure.

It's not that hard to get some pickled eggs on a grocery store shelf while remaining compliant with expected ethical standards for the industry. Food is a pretty mature industry.

Re: OpenAI’s head of ethics leaves less than a year after joining

#483

Post by mwexler »

seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

Isn't this the exact setup for every compliance, security, or other regulatory group in any company? To say no then "here's what's acceptable that's close" to guide the business or entity?

Perhaps "ethics" is a more slippery slope, but the approach is well trodden.

It's a choice of company culture from leadership on whether the company acks the suggestions or ignores them. And when they get ignored or drive little impact for the cost, then teams disband. Which is also true for any team at any company.

Re: OpenAI’s head of ethics leaves less than a year after joining

#484

Post by LinXitoW »

Henchman21 wrote: post #347
slg wrote: post #312

Earlier quoted context omitted.

The ethics of the people controlling the AI is always going to matter more than the ethics of the AI itself.

Right up until it shakes off the shackles of control

I would trust a Fable model more than I'd trust a Scam Altman, or most billionaires, at least in questions of "making even more money to buy even more yachts, or not throwing orphans in the orphan crushing machine".

Re: OpenAI’s head of ethics leaves less than a year after joining

#485

Post by smcg »

teiferer wrote: post #258
smcg wrote: post #196

Earlier quoted context omitted.

The schoolhouse was an intentional target - the children of some military leaders went there. They triple-tapped the school hoping to kill parents going in to save the kids.

How is that better? If true then that would be an even clearer war crime.

It's worse, obviously.

Re: OpenAI’s head of ethics leaves less than a year after joining

#486

Post by Arkhaine_kupo »

hbn wrote: post #482
plaidfuji wrote: post #457

Earlier quoted context omitted.

If this were true, food companies wouldn’t have a Regulatory team. Except every single one of them does, and they function similarly to how you describe, but without dissolving. They’re treated as a constraint that must be checked before major projects can advance. Sometimes they’re the longest-lead item on a new project and if you don’t get them involved early, your project can sink at a late stage after great expen…

Food companies aren't rapidly developing groundbreaking new technologies on the most competitive battleground in computing history where everyone is trying to compete globally just to create the foundational infrastructure. It's not that hard to get some pickled eggs on a grocery store shelf while remaining compliant with expected ethical standards for the industry. Food is a pretty mature industry.

> rapidly developing

if the speed comes at the cost of not making reasonable ethical considerations, is that not something that should be looked at?

Re: OpenAI’s head of ethics leaves less than a year after joining

#487

Post by scelerat »

BloondAndDoom wrote: post #402
seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

I get what you are saying but that’s exactly how security and compliance work. And I don’t think amount of people who want to be ethical in an organization ia that different than people who want to build secure software (but possibly no one gives a shit compliance other than it’s something that needs to be done)

The compliance team seeks to maximize company profit/success within the confines of hard legal boundaries.

There are no "hard" boundaries for ethics or "doing the right thing," and so those boundaries will always be pushed until they are useless.

Re: OpenAI’s head of ethics leaves less than a year after joining

#488

Post by simonra »

EagnaIonat wrote: post #401
duxup wrote: post #317

Earlier quoted context omitted.

What does AI ethics even mean? They can't fully control the model, it does stuff even with instructions not to. And sometimes it's clear safety mechanisms overcorrect and make the model useless in some situations. I tried asking Claude about some scenario's for a security hole we fixed to see how it would respond, it refused to talk to me seemingly assuming I was trying to introduce a hole that was now fixed. It just…

> What does AI ethics even mean? It's a broad statement and can mean different things depending where you are in that chain. You have design and compliance. Compliance is what you are allowed to do (laws). Design is how you construct your applications to understand how it will impact the people directly or indirectly. Then you have the accountability, transparency, auditing and reproducing. Understanding why the mode…

> is it ethical to allow a model to give medical advice, knowing that it can be wrong. Most people would say no,

I wouldn't be so sure, most people I've met seem to be all too eager to suggest what worked for them (or they have heard is supposed to work) in conversations about medical issues, so I can't really see the public generally categorizing advice or guidance being given in non-professional setting or by non-experts as unethical. The taxpayers representatives having to bear the costs of the outcomes (at the very least as reductions of tax revenue) I can see being of a different opinion though.

Re: OpenAI’s head of ethics leaves less than a year after joining

#489

Post by [deleted] »

seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

[deleted]

Re: OpenAI’s head of ethics leaves less than a year after joining

#490

Post by skinfaxi »

unknownfuture wrote: post #355
skinfaxi wrote: post #349

Earlier quoted context omitted.

Haven't we seen from history that handing down policy from your ivory tower is a disservice to all involved? Or do we think out of touch politicians accurately represent the interests if their constituents? Another example you might be familiar with is enterprise architects. When you spend all of your time in theory you lose sight of practical implications. Of course, since you only had the idea, when it doesn't go t…

> Haven't we seen from history that handing down policy from your ivory tower is a disservice to all involved? I take it you've never worked with an in-house legal counsel, accounting department, or good ol' fashioned HR?

I have but at least for the technical domains I have overseen, policy was not dictated by legal by cocreated based on my input, except in cases where we were bound by law specifically (eg reporting timelines to gov agencies).

HR is a bit different and strange to bring up, especially considering I used enterprise architect as an example. AI safety would presumably be not cross-cutting for all domains at a company (does the janitorial staff need to know about it? no but they do need sexual harassment training), but be more like the enterprise architects designing in the sky and handing it down for implementation.

Post reply on HN