Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft lays off one of its responsible AI teams

platformer.news

Post reply on HN
Print view View original

271–280 of 466 posts

Re: Microsoft lays off one of its responsible AI teams

#271

Post by AstralStorm »

EVa5I7bHFq9mnYK wrote: post #259
bonzini wrote: post #170

Earlier quoted context omitted.

That's not AGI. An AGI would be able to tell you they don't know things, ask for clarification as to why you think they're wrong, etc. ChatGPT just spouts out a different wild guess when you tell it it's wrong, but doesn't learn from its mistake—not even within a single chat. Sydney just goes full psychopath.

That's not AGI. AGI would be able to write its own code, and improve it exponentially.

Not necessarily, much like every human does not know pharmacology, genetics, robotics or other specialties that could be used for self improvement. Including social engineering and programming.

What you're describing is a particular kind of superintelligence, recursively improving kind. Between that and general intelligence is another gap.

Re: Microsoft lays off one of its responsible AI teams

#272

Post by maronato »

whywhywhywhy wrote: post #255
worrycue wrote: post #74

Earlier quoted context omitted.

> this has not caused mass social harm despite the tens of millions of users Isn’t it a bit early to tell? I can foresee ChatGPT being used to farm “karma” on social media sites to bot accounts to credibility on sites like Reddit - maybe even HN.

Who’s checking the karma of posters on Reddit?

Some subs have karma requirements for posting or commenting.

Re: Microsoft lays off one of its responsible AI teams

#273

Post by visarga »

ssalka wrote: post #89
anonylizard wrote: post #19

Earlier quoted context omitted.

This. People should understand AI safety != AI ethics. AI safety has two branches, AI ethics and AI alignment. Those two branches despise each other with intense passion. The alignment branch (responsible for developing RLHF->ChatGPT) believes AI ethics is completely trivial, and distracts attention away from existential risks. The ethics branch (responsible for all the media attention about racist AIs/bias etc) view…

AI alignment people are the group who are capable of bringing AGI to life, but want to do it responsibly. AI ethics people are the group who are not capable of bringing AGI to life, but want to make sure that those who can do it responsibly or not at all.

> but want to make sure that those who can do it responsibly or not at all.

Mostly not at all. They write papers like "Can Language Models Be Too Big?"

I say then: can people be too afraid of the unknown?

Re: Microsoft lays off one of its responsible AI teams

#274

Post by [deleted] »

vorticalbox wrote: post #159
worrycue wrote: post #74

Earlier quoted context omitted.

> this has not caused mass social harm despite the tens of millions of users Isn’t it a bit early to tell? I can foresee ChatGPT being used to farm “karma” on social media sites to bot accounts to credibility on sites like Reddit - maybe even HN.

Do you consider "karma" farming "mass harm"?

[deleted]

Re: Microsoft lays off one of its responsible AI teams

#275

Post by AstralStorm »

pmarreck wrote: post #270
anonylizard wrote: post #19

Earlier quoted context omitted.

This. People should understand AI safety != AI ethics. AI safety has two branches, AI ethics and AI alignment. Those two branches despise each other with intense passion. The alignment branch (responsible for developing RLHF->ChatGPT) believes AI ethics is completely trivial, and distracts attention away from existential risks. The ethics branch (responsible for all the media attention about racist AIs/bias etc) view…

> Stable diffusion does not default to creating black people (you have to insert black into the prompt), honestly in order to make race an irrelevant or random factor, I believe the training data would have to be evenly divided between all known races... and possibly some blends of races? And I think that would be a good thing. Could you synthesize such data by (ironically perhaps) using AI to change the race of a pe…

You can certainly try this kind of data augmentation strategy. Pretty sure it'll fail because of the text analysis still being biased.

You would need equalized amounts of text referring to various groups of people and topics too. That's much harder to augment.

Re: Microsoft lays off one of its responsible AI teams

#276

Post by visarga »

whywhywhywhy wrote: post #258

End of the day the AI space will be dominated by whatever model doesn’t lecture you about “as a large language model I can’t…”. And the company that will ship that won’t have the largest “AI safety and ethics” team. ChatGPT will become as irrelevant as Dall-E2 when that happens. (Not saying this is for the best, just saying what I think will happen)

> ChatGPT will become as irrelevant as Dall-E2 when that happens.

We're at that point (nocomercially) with LLaMA. It's not just running on private hardware, but unrestricted and tunable like DeamBooth.

Re: Microsoft lays off one of its responsible AI teams

#277

Post by joshxyz »

hn_throwaway_99 wrote: post #9

These sorts of adjunct, off-to-the-side "ethics" teams are always corporate window dressing, and when push comes to shove they always get jettisoned when competition demands an increase in speed. I don't mean to start a flame war, but this is a reason why I think "Chief Diversity Officers" are such an ill-conceived notion. It's not that I don't think diversity in corporate environments is extremely valuable, but it's…

diversity equity and inclusion, oh dear.

Re: Microsoft lays off one of its responsible AI teams

#278

Post by pc86 »

hn_throwaway_99 wrote: post #9

These sorts of adjunct, off-to-the-side "ethics" teams are always corporate window dressing, and when push comes to shove they always get jettisoned when competition demands an increase in speed. I don't mean to start a flame war, but this is a reason why I think "Chief Diversity Officers" are such an ill-conceived notion. It's not that I don't think diversity in corporate environments is extremely valuable, but it's…

Almost entirely, the existence of Chief Diversity Officers - and to a lesser but still prevalent extent, this "off-to-the-side" teams you mention - are just jobs programs with an identity politics twist.

CDOs exist so that the "right" people can have a high-power (even if only via public perception), well-paid C-suite role. These ethics teams exist so that a group of the "right" people can level up their resumes. Instead of being a developer at a bank in Omaha you can be on the AI ethics team at a startup. Instead of being a developer at a startup you can be on the AI ethics team at a FAANG.

Re: Microsoft lays off one of its responsible AI teams

#279

Post by spacebanana7 »

EVa5I7bHFq9mnYK wrote: post #250
anonylizard wrote: post #19

Earlier quoted context omitted.

This. People should understand AI safety != AI ethics. AI safety has two branches, AI ethics and AI alignment. Those two branches despise each other with intense passion. The alignment branch (responsible for developing RLHF->ChatGPT) believes AI ethics is completely trivial, and distracts attention away from existential risks. The ethics branch (responsible for all the media attention about racist AIs/bias etc) view…

Sorry, what is RLHF?

RLHF is Reinforcement Learning from Human Feedback.

It usually refers to fine tuning language models using data labelled by humans.

Hugging face have a good overview in this article: https://huggingface.co/blog/rlhf

Re: Microsoft lays off one of its responsible AI teams

#280

Post by jefftk »

matusp wrote: post #158
anonylizard wrote: post #19

Earlier quoted context omitted.

This. People should understand AI safety != AI ethics. AI safety has two branches, AI ethics and AI alignment. Those two branches despise each other with intense passion. The alignment branch (responsible for developing RLHF->ChatGPT) believes AI ethics is completely trivial, and distracts attention away from existential risks. The ethics branch (responsible for all the media attention about racist AIs/bias etc) view…

> The alignment branch (responsible for developing RLHF->ChatGPT) RLHF was not conceived by AI Alignment people. Using RL to train generative models was a thing even ten years ago. Now they finally made it work on scale. This has nothing to do with alignment.

Reinforcement learning in general didn't come out of AI alignment work, but RHLF in particular did. The initial idea and paper [1] were from AI alignment folks, as was most of the later development [2][3][4][5]. Overview: https://www.alignmentforum.org/posts/vwu4kegAEZTBtpT6p/thoug...

[1] Deep Reinforcement Learning from Human Preferences https://arxiv.org/abs/1706.03741

[2] Fine-Tuning Language Models from Human Preferences https://arxiv.org/abs/1909.08593

[3] Learning to summarize from human feedback https://arxiv.org/abs/2009.01325

[4] Recursively Summarizing Books with Human Feedback https://arxiv.org/abs/2109.10862

[5] Training language models to follow instructions with human feedback https://arxiv.org/abs/2203.02155

Post reply on HN