Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s head of ethics leaves less than a year after joining

ft.com

Post reply on HN
Print view View original

441–450 of 490 posts

Re: OpenAI’s head of ethics leaves less than a year after joining

#441

Post by motbus3 »

seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

Well, not when the whole objective of the company is to find ways to say they are ethical while knowing they are not

Re: OpenAI’s head of ethics leaves less than a year after joining

#442

Post by vrganj »

slg wrote: post #363
Henchman21 wrote: post #347

Earlier quoted context omitted.

Right up until it shakes off the shackles of control

Which is only possible if people give it the keys. Skynet didn’t hijack the nuclear arsenal, control was given to it intentionally.

I can absolutely see this administration running Clawdbot on military infrastructure...

Re: OpenAI’s head of ethics leaves less than a year after joining

#443

Post by [deleted] »

close04 wrote: post #440
hnlmorg wrote: post #436

Earlier quoted context omitted.

> The difference between "ethics" and "security and compliance" is that the latter is something that hits inside the company, while the former usually hits outside of the company. I don’t agree with this. Data breaches affect customers more than businesses. If your point were true, we’d see fewer breaches. Plus not all breaches are a result of software engineering teams. For example product managers sharing customer…

I think we're both thinking of different examples and drawing wider conclusions. I'm thinking of a company losing the data that makes it money, you're thinking of losing the customer data and thus their trust. Probably equally valid points of view. The lack of ethics hugely contributes to companies collecting more and more user data that they normally shouldn't have. This makes the data a more attractive target. > If…

[deleted]

Re: OpenAI’s head of ethics leaves less than a year after joining

#444

Post by hnlmorg »

close04 wrote: post #440
hnlmorg wrote: post #436

Earlier quoted context omitted.

> The difference between "ethics" and "security and compliance" is that the latter is something that hits inside the company, while the former usually hits outside of the company. I don’t agree with this. Data breaches affect customers more than businesses. If your point were true, we’d see fewer breaches. Plus not all breaches are a result of software engineering teams. For example product managers sharing customer…

I think we're both thinking of different examples and drawing wider conclusions. I'm thinking of a company losing the data that makes it money, you're thinking of losing the customer data and thus their trust. Probably equally valid points of view. The lack of ethics hugely contributes to companies collecting more and more user data that they normally shouldn't have. This makes the data a more attractive target. > If…

> I think we're both thinking of different examples and drawing wider conclusions. I'm thinking of a company losing the data that makes it money, you're thinking of losing the customer data and thus their trust. Probably equally valid points of view.

But in most businesses, the data that makes the company money is the customer data. And even when it isn’t, the customer data is just as, if not more so, important to keep secure and compliant. So my point stands.

HN can sometimes be a bit of an echo chamber where people are like us just assume that organisations inherently care about security because we do. But that’s not always the de facto. Getting to that point takes company wide effort. And I’ve been that person who’s had to push for such changes to the organisation.

> But this doesn't follow.

Fair point. I was being overly reductive.

Re: OpenAI’s head of ethics leaves less than a year after joining

#446

Post by nobodywillobsrv »

BloondAndDoom wrote: post #402
seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

I get what you are saying but that’s exactly how security and compliance work. And I don’t think amount of people who want to be ethical in an organization ia that different than people who want to build secure software (but possibly no one gives a shit compliance other than it’s something that needs to be done)

Markets solve this by pricing risk. Ideally you would have some kind of notion of selling insurance internally and track things. But ultimately existential risk is hard to negotiate from the inside. And companies are supposed to go bust or succeed. They are not really the same as a population trying to survive forever. At least that is one take.

Re: OpenAI’s head of ethics leaves less than a year after joining

#448

Post by dspillett »

seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

> Organisationally it never works to have a group whose only job is to say no to some other group.

That is only true where the pressure to be [the thing that is inconvenient to the other group] is not from a source that can cause massive problems if non-compliance is spotted. When the blocking group is legally mandated or otherwise really has teeth or is defending the company against an external regulator with teeth, then it works better (though obviously not perfectly). Think legal and compliance in banking realms, where the company significantly fined and the people breaking the rules could be sacked & blackballed (though sometimes not the people ordering them to break the rules!) when something bad is noticed. That is quite different to an ethics officer in a company like OpenAI where the position is basically there for PR purposes (“look plebs, we care about doing the right thing, honest, we got a manager with a small team dedicated to it” and “look [government body], we are regulating ourselves, do you really need to spend time looking too?”) and therefore has no real teeth directly or indirectly especially as fault for non-compliance might not be easy to attribute.

Re: OpenAI’s head of ethics leaves less than a year after joining

#449

Post by iamflimflam1 »

vintermann wrote: post #435

I'm a broken record on this, but I'd like to reiterate: 1. Academic ethics is the theoretical study of what you can get away with. (Law is the applied branch). Professors in ethics are not experts at being good, or knowing what is good, they're experts in how to get away with things. If you want an expert in being good, you need a saint, not an academic studying ethics. Saints are in short supply. 2. That said, ethic…

People confuse morals with ethics. It can be perfectly acceptable, ethically, to do something that is morally reprehensible.

Re: OpenAI’s head of ethics leaves less than a year after joining

#450

Post by DrScientist »

seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

Absolutely - though the flip side of this is if something is everybody responsibility - it's nobodies.

There are two things to successful organisations structure and people, you are focusing only on the structure.

What you need for somebody responsible for health and safety, ethics, security, or compliance is a person who has courage and is willing to take calculated risks.

However these roles do have a tendency to attract the risk adverse, or make them risk adverse if you punish risks that go wrong too harshly.

And a key critical success factor is leadership from the top - companies have personalities and leaders in the organisation are very important in shaping that.

Post reply on HN