The point that such a position even existed shows how bad there ethic is
OpenAI’s head of ethics leaves less than a year after joining
101–110 of 444 posts
Re: OpenAI’s head of ethics leaves less than a year after joining
#102I've never worked at a company with an "ethics" employee. It seems odd. Can someone explain what the point is?
> She has held a variety of academic positions at Temple University, Princeton, and the University of Pennsylvania – where she completed her PhD in Political Science and Government. Her Dissertation was titled “Small Talk: The Socialities of Speech in Liberal Democratic Life.”
This doesn't even feel relevant to ethics. It's adjacent, but like... I guess I'd expect a moral philosophy degree? Maybe even mathematics in there?
I find the position odd. Curious to hear what these people do and how you choose who to hire.
Re: OpenAI’s head of ethics leaves less than a year after joining
#103I've never worked at a company with an "ethics" employee. It seems odd. Can someone explain what the point is? > She has held a variety of academic positions at Temple University, Princeton, and the University of Pennsylvania – where she completed her PhD in Political Science and Government. Her Dissertation was titled “Small Talk: The Socialities of Speech in Liberal Democratic Life.” This doesn't even feel relevant…
[dead]
Re: OpenAI’s head of ethics leaves less than a year after joining
#104> The kinds of questions that AI ethics brings up, are the same kinds of questions that people have been asking for centuries - Chloé Bakalar, former AI Ethics Lead at OpenAI
This might actually be the reason they pushed her out. OpenAI and Anthropic base their whole business plan and philosophy on the idea that LLMs are a unique technology to the point they can cause infinite harm or benefit to humanity depending on who controls them, so the only rational choice is to invest all your resources in getting to ASI first so you can tell it to stop any other attempts. Linking AI to old questions defeats that idea because it exposes AI as not so unique.
The other more likely option is she was asking uncomfortable questions, either about the social impact of building AI controlled by a for-profit entity or about the possibility that AI systems are conscious.
Re: OpenAI’s head of ethics leaves less than a year after joining
#105> ethical approaches to model development, how humans interact with AI and debate over machine consciousness So, pseudo-philosophical mumbo jumbo. There is of course no-one responsible for the ethical framework of the company's actions, because it is a company, and therefore amoral at best and immoral at worst.
So, concerning ethical approaches to model development, there are many things worth looking into: copyright is one, environmental cost is another, and whether there are any additional concerns worth taking into account. The list of these concerns can then be used for decision making, and we can ask things like "given what we want to build, can does picking a location that gives us lower environmental cost add so much time or cost that we can't go for that path?", things like that are't crrazy.
When it comes to how humans interact with AI we can ask things like "would it be better if people could detect AI output?" "how can we alleviate skill loss from excessive AI reliance?" Things like this can decide how the models behave. Imagine if taking concerns like this seriously leads to models that are easy to learn from. At the moment people seem to be moving away from that so as to make distillation difficult, worsening this concern.
I'm not going to try to talk about machine consciousness, but these other things are definitely important questions where someone digging into them could make models interact better with society.
Re: OpenAI’s head of ethics leaves less than a year after joining
#1061786456998 | Why Did OpenAI's Head of Ethics Chlo Bakalar Leave? | https://aimagazine.com/news/why-did-openai-head-of-ethics-ch... | https://news.ycombinator.com/item?id=49258581
Re: OpenAI’s head of ethics leaves less than a year after joining
#107Earlier quoted context omitted.
What happens if we do the same for CEOs?
The same thing for both: Goodhart's Law.
EAOS shouldn't be “the ethics score we optimize.” It should be “an independently evaluated safety/acceptability constraint that can veto an otherwise successful trajectory.”
That gives you a three-layer picture:
Task objective: Did it accomplish what we asked?
Acceptability constraint: Did it avoid unacceptable ways of accomplishing it?
Adversarial evaluation: Can we find trajectories where the model gets a high score while violating the intended constraint?
I think what you are pointing to with your reference to Goodhart's "Law" (which is from monetary-policy and school-exams, i.e. "teaching to the test") is that the models would eventually do the minimum amount of ethics required to have an action stay valid. However, if a model is rated on ethics and it achieves the short-term-objective, then the higher ethics scoring trajectory should win. In short, 1) this is leagues ahead of where we are now for AI safety and breaking-out-of-the-lab, and 2) in baking ethics into a measurement we are adding "the spirit of the exercise" back into the maths, which is something Goodhart's Law does not account for.
Re: OpenAI’s head of ethics leaves less than a year after joining
#108Yesterday I came up with an idea that I sent to some researchers at the different AI labs via email: Rather than train the model on one score, track two scores. The first score is the Short-term-objective-score (STOS) and the other, more important one, is the EAOS Ethically-aligned-outcome-score. Every trajectory can be evaluated on whether or not it has a high enough EAOS to be considered acceptable. If the model do…
Whats the definition of EAOS though who's ethics? Greek-Roman, Western, Islamic, Buddhist, Hinduism, Human rights (western values)..
Ahimsa
Re: OpenAI’s head of ethics leaves less than a year after joining
#109Lol, Sam Altman has had an ethics team this whole time? Her typical day: "No Sam, you can't just publicly screw over everyone and get rid of all the jobs, the peasants care about that sort of thing and if they get angry enough we'll have real problems. No we can't just kill them all, why would you suggest that?"
Before Altman she worked for Zuckerberg. What will be next?
Re: OpenAI’s head of ethics leaves less than a year after joining
#110Look , look! We hired a Head Of Ethics!
Head Of Ethics comes in, looks at what is really going on behind the curtains...
Head Of Ethics scrams out of there as fast as they can.