Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s head of ethics leaves less than a year after joining

ft.com

Post reply on HN
Print view View original

421–430 of 447 posts

Re: OpenAI’s head of ethics leaves less than a year after joining

#421

Post by bluehatbrit »

hypfer wrote: post #407
seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

> whose only job is to say no to some other group I suspect that people will miss just how precise you've been there. Your proposal - if I understand correctly - is not just to spread the "ethics team" onto everyone (which is correct), but also to have someone in a dedicated role. However, they need to have not _just_ that role, but also some skin in the game.

This is also my view. These sorts of roles can have a very high impact and be very successful if everyone feels like they're rowing in the same direction.

The people in those roles need make people feel like there is value in seeking their input. They also need to find ways to say yes that helps things happen in the right way, rather than stopping them entirely.

This happens naturally in organisations where security is valued by everyone. InfoSec / CIO roles can operate very successfully and deliver a lot of value. As soon as people lose faith and see them as the "no" team, they start trying to hide from them.

There are lots of ways to create this but it's as much about the people in the role, as it about the organisation itself. If either are skeptical about the other, it falls apart very quickly.

Re: OpenAI’s head of ethics leaves less than a year after joining

#422

Post by seanhunter »

BloondAndDoom wrote: post #402
seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

I get what you are saying but that’s exactly how security and compliance work. And I don’t think amount of people who want to be ethical in an organization ia that different than people who want to build secure software (but possibly no one gives a shit compliance other than it’s something that needs to be done)

In a high-functioning security and compliance team they tend to say “no” only in really dire circumstances. Good security and compliance teams spend a lot of time asking exactly what it is the people are really trying to do and then find a way to say “yes, and…” as in “yes you can and here’s how you do it without compromising security/breaking the law etc”. And as a sibling said, orgs fail at security when they make security only the infosec teams’ job.

This is also why I said “whose only job”. In a good org, the security team doesn’t only say no to devsecops requests, they also do trainings to skill up other teams, keep the network secure, proactively seek out and understand external threats, work with external vendors etc etc …

Re: OpenAI’s head of ethics leaves less than a year after joining

#423

Post by bravetraveler »

tommica wrote: post #397
bravetraveler wrote: post #380

Earlier quoted context omitted.

Third crack commandment: never trust nobody; your moms'll set that ass up, properly gassed up Applies every-which-way; the chat bot, the provider, the consumer.

Is this written by AI? It sounds nonsensical

No. Let me help dissect; first half is quoted from what I mistook to be a well-known song and the second half is... I don't know how else to say this kindly, discernible (and all me). The main reasons I'm not flipping that grating question on you: song age, search engine decline.

Said plainly: the previously-mentioned keys will absolutely be handed over. Nobody, not even your mother or her favorite LLM, is to be trusted to such degree.

Re: OpenAI’s head of ethics leaves less than a year after joining

#424

Post by jongjong »

Wow. How much does this job pay? Google says $550k to $1.3 million.

For that kind of money, my AI paymaster could do no wrong.

Imagine going to work and getting paid $2300+ USD per day to sit on your ass all day thinking about ethics without having to produce any deliverables. This would be a dream job for me.

If anyone questioned my morals, I'd be like; "They're paying me $550k per year" and the other person would be like "Ahhh ok, my bad! Your logic is starting to make sense now!"

Re: OpenAI’s head of ethics leaves less than a year after joining

#425

Post by Rebuff5007 »

seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

Shall we get rid of the FAA and the FDA then? Lets just have all pfizer employees make ethics their own responsibility.

What your describing is the specific case of a company so paralyzed by short-term thinking that they see regulation as a burden. Some maturity in the organization would allow reframing this to be less antagonistic.

Re: OpenAI’s head of ethics leaves less than a year after joining

#426

Post by erehweb »

Rebuff5007 wrote: post #425
seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

Shall we get rid of the FAA and the FDA then? Lets just have all pfizer employees make ethics their own responsibility. What your describing is the specific case of a company so paralyzed by short-term thinking that they see regulation as a burden. Some maturity in the organization would allow reframing this to be less antagonistic.

Crucially, the FAA is not a part of United Airlines, and the FDA is not a part of Pfizer. If there weren't any FDA, then the safety group in Pfizer would be a lot weaker.

Re: OpenAI’s head of ethics leaves less than a year after joining

#427

Post by benjiro29 »

Rebuff5007 wrote: post #425
seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

Shall we get rid of the FAA and the FDA then? Lets just have all pfizer employees make ethics their own responsibility. What your describing is the specific case of a company so paralyzed by short-term thinking that they see regulation as a burden. Some maturity in the organization would allow reframing this to be less antagonistic.

Its funny because "Shall we get rid of the FAA and the FDA then?" ... matches exactly with the current situation of the FDA and Points 3 > 5 ... The systematic nerfing of the FDA and other organizations their power.

Re: OpenAI’s head of ethics leaves less than a year after joining

#428

Post by mnahkies »

BloondAndDoom wrote: post #402
seanhunter wrote: post #399

Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group…

I get what you are saying but that’s exactly how security and compliance work. And I don’t think amount of people who want to be ethical in an organization ia that different than people who want to build secure software (but possibly no one gives a shit compliance other than it’s something that needs to be done)

I think the key part is about aligning incentives and outcomes - if your security and compliance departments are only judged on "were we breached, did we pass our audits" then there's a risk they won't weigh the tradeoffs associated with the decisions around controls implemented to achieve those outcomes.

To use a ridiculous extreme you can't breach a web app that isn't exposed to the internet, but the users can't access it either.

If you can connect/balance those goals to other metrics around cost and productivity, usability, and a realistic threat model, as guardrails then you incentatize cross-team collaboration to achieve the shared org outcomes.

Re: OpenAI’s head of ethics leaves less than a year after joining

#430

Post by riazrizvi »

I guess working on ethics for Meta's social media recommendation and flagging algorithms isn't meeting the moment or need of OpenAI where LLMs are breaking out of sandboxes and there's a pressing question of what is okay for an LLM to conversationally engage in bc lawsuits. Seems to me they need someone with more of a language and AI background.
Post reply on HN