Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s head of ethics leaves less than a year after joining

ft.com

Post reply on HN
Print view View original

311–320 of 397 posts

Re: OpenAI’s head of ethics leaves less than a year after joining

#311

Post by [deleted] »

gizajob wrote: post #293
madrox wrote: post #247

In five years, I'd love to read a book about the history of AI ethics. I suspect it will read like Voltaire. My sense is that it is radically shifting from a fluffy marketing arm to a department expected to contribute meaningfully to development and justify its impact. I would expect an ethics team to build frameworks that can help train/eval the model that the company spend millions of dollars and months training is…

I think you’re confusing Voltaire with the Marquis de Sade and Machiavelli

[deleted]

Re: OpenAI’s head of ethics leaves less than a year after joining

#312

Post by slg »

minraws wrote: post #267

The rats are fleeing because the captain doesn't care if the ethical ship sinks... Is what my take away would be if I didn't know, that the ship sank a long time ago and we are just pretending anyone really cares. Let's be honest unless AI gains literal limbs, and the records a video of itself presenting a manifesto about world domination and then kills actual humans with its cold mechanical hands. We won't care abou…

The ethics of the people controlling the AI is always going to matter more than the ethics of the AI itself.

Re: OpenAI’s head of ethics leaves less than a year after joining

#313

Post by zem »

teiferer wrote: post #250
madrox wrote: post #247

In five years, I'd love to read a book about the history of AI ethics. I suspect it will read like Voltaire. My sense is that it is radically shifting from a fluffy marketing arm to a department expected to contribute meaningfully to development and justify its impact. I would expect an ethics team to build frameworks that can help train/eval the model that the company spend millions of dollars and months training is…

> if the model requires retraining for ethics reasons. Haha, how refreshingly naive. What do you think is more likely: this or the other way around: getting the "ethics team" to align with investor goals? Hint: Where does the money come from? > The people in AI ethics who spent years thinking their job was marketing or publishing thinkpiece papers may be having to adapt quickly or get out of the field. Or it's exactl…

reminds me of a startup I was in, where one of the devs accepted a promotion to head of engineering on the condition that the CEO would not override him when he said a release was too buggy to go out to customers.

the CEO agreed because (I imagine) he really needed someone to take that position. and as everyone should probably have seen coming, the first time he really wanted to send out a release bugs and all, he called the guy into a room and pretty much browbeat him for an hour about how making the promosed release date was more important than making a good release, until he said "fine but I'm not responsible if it breaks".

the CEO held this up as an example of how he had kept his word not to send the release out without the guy agreeing to it. that startup, needless to say, is long dead.

Re: OpenAI’s head of ethics leaves less than a year after joining

#314

Post by bigbadfeline »

pessimizer wrote: post #300
teiferer wrote: post #250

Earlier quoted context omitted.

> if the model requires retraining for ethics reasons. Haha, how refreshingly naive. What do you think is more likely: this or the other way around: getting the "ethics team" to align with investor goals? Hint: Where does the money come from? > The people in AI ethics who spent years thinking their job was marketing or publishing thinkpiece papers may be having to adapt quickly or get out of the field. Or it's exactl…

It's funny, but Scientologists were innovators on this front. They use the word "ethics" a lot, but they've specifically redefined it to mean productivity at work (people who aren't productive are stealing a living.) Being "downstat" (iirc) is when you're not productive in comparison to your coworkers, which makes you "outethics" and a "potential trouble source (PTS)." If you're outethics, then you get called in for…

This

Re: OpenAI’s head of ethics leaves less than a year after joining

#315

Post by lovich »

walrus01 wrote: post #275
minraws wrote: post #267

The rats are fleeing because the captain doesn't care if the ethical ship sinks... Is what my take away would be if I didn't know, that the ship sank a long time ago and we are just pretending anyone really cares. Let's be honest unless AI gains literal limbs, and the records a video of itself presenting a manifesto about world domination and then kills actual humans with its cold mechanical hands. We won't care abou…

> But honestly if I started a new AI lab named it Skynet Corp you're too late https://skynetai.dev/

Ah, another great example of building the Torment Nexus from the award winning sci fi classic “Please for the Love of God, Never Build the Torment Nexus”

Re: OpenAI’s head of ethics leaves less than a year after joining

#316

Post by Avicebron »

madrox wrote: post #305
a2ff6eeb0 wrote: post #288

Earlier quoted context omitted.

Why? Are trolley problems making it hard to make money?

Because leaving that to emergent behavior is silly. Why is the idea of training models on ethical reasoning so strange?

Because 1) ethics aren't universal. It's perfectly easy to frame ethics for the outcome you want depending on your a priori assumptions. 2) it's blatantly obvious that the business "needs" will supersede any ethical concerns or the ethics will be framed in a way that aligns with the primary goal of gaining money and power.

Re: OpenAI’s head of ethics leaves less than a year after joining

#317

Post by duxup »

madrox wrote: post #247

In five years, I'd love to read a book about the history of AI ethics. I suspect it will read like Voltaire. My sense is that it is radically shifting from a fluffy marketing arm to a department expected to contribute meaningfully to development and justify its impact. I would expect an ethics team to build frameworks that can help train/eval the model that the company spend millions of dollars and months training is…

What does AI ethics even mean?

They can't fully control the model, it does stuff even with instructions not to.

And sometimes it's clear safety mechanisms overcorrect and make the model useless in some situations. I tried asking Claude about some scenario's for a security hole we fixed to see how it would respond, it refused to talk to me seemingly assuming I was trying to introduce a hole that was now fixed. It just wouldn't talk ...

I'm not sure it's a job that you can win at even if you tried / were given all the resources.

AI was trained on human things, including bad things. https://youtu.be/KUXb7do9C-w

There's no safety to be found.

Re: OpenAI’s head of ethics leaves less than a year after joining

#318

Post by madrox »

duxup wrote: post #317
madrox wrote: post #247

In five years, I'd love to read a book about the history of AI ethics. I suspect it will read like Voltaire. My sense is that it is radically shifting from a fluffy marketing arm to a department expected to contribute meaningfully to development and justify its impact. I would expect an ethics team to build frameworks that can help train/eval the model that the company spend millions of dollars and months training is…

What does AI ethics even mean? They can't fully control the model, it does stuff even with instructions not to. And sometimes it's clear safety mechanisms overcorrect and make the model useless in some situations. I tried asking Claude about some scenario's for a security hole we fixed to see how it would respond, it refused to talk to me seemingly assuming I was trying to introduce a hole that was now fixed. It just…

> What does AI ethics even mean?

This, I think, is the question at the core of of the field right now. Five years ago it was highly hypothetical. Today, not so much. I'll be curious to see what happens.

Re: OpenAI’s head of ethics leaves less than a year after joining

#319

Post by mtlynch »

throwa356262 wrote: post #211
wxw wrote: post #200

> Previously, she was chief ethicist at Meta. She spent 6 years at Meta. I get the sense there are other factors at play than "ethics departments are just PR stunts". There's no way she isn't already aware of that. Unfortunately, the article is light on any other details.

Being "chief ethicist at Meta" under Zuckerberg... is like being head rabbi at Auschwitz under Rudolf Höß.

Strangely, a few years ago, Facebook hired a rabbi and appointed him "Director of Responsible Innovation."[0]

He did an interview on Search Engine that I thought was interesting,[1] but it definitely seemed like there was more to the story than just, "Facebook needed to be ethical, and this guy was so good at ethics that he decided to help out."

[0] https://chochmat.org/rabbi-zvika-krieger/

[1] https://www.searchengine.show/what-does-it-feel-like-to-beli...

Re: OpenAI’s head of ethics leaves less than a year after joining

#320

Post by afro88 »

madrox wrote: post #247

In five years, I'd love to read a book about the history of AI ethics. I suspect it will read like Voltaire. My sense is that it is radically shifting from a fluffy marketing arm to a department expected to contribute meaningfully to development and justify its impact. I would expect an ethics team to build frameworks that can help train/eval the model that the company spend millions of dollars and months training is…

> I would expect an ethics team to build frameworks that can help train/eval the model that the company spend millions of dollars and months training is going to be aligned to the ethical stances the company chooses.

Which like any company will be entirely driven by legal constraints and money. Or just money if it's cheaper to break the law for profit and pay fines. There will be no "this is what's good for humanity, economics be damned".

Much like HR isn't to help employees but just the company.

Post reply on HN