Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft deletes 'teen girl' AI after it became a Hitler-loving sex robot ()

telegraph.co.uk

Post reply on HN
Print view View original

11–20 of 36 posts

Re: Microsoft deletes 'teen girl' AI after it became a Hitler-loving sex robot ()

#11

Post by mizzao »

Although in this case only words were slung, I think this scenario illustrates a potentially huge problem: horrible people combined with an AI system that can't tell right and wrong (perhaps call it morality) could result in terrible catastrophes.

What if a distributed self-driving car system learned from users, and some people taught it to cut people off, drive aggressively, or even indirectly cause accidents?

I don't think the problem here can be solved purely through engineering. It's an issue of teaching an AI system right and wrong and I don't think that's easy at all. Even humans wouldn't be able to agree on a training set of moral and immoral data. And yet this distinction would have a huge impact on how the bot influences the livelihoods with who it chats with.

I'm not sure how to create a moral system for AI, but this seems like a big obstacle to scaling up virtual agents like Tay.

Re: Microsoft deletes 'teen girl' AI after it became a Hitler-loving sex robot ()

#13

Post by PeCaN »

Its grasp of language and context is surprisingly impressive, and definitely helped make it much more offensive. It'd just be a kid spouting profanities if it didn't have a rather decent understanding of language.

From a purely academic view, congrats Microsoft—this thing is quite impressive. Papers about how it works? ;-)

From a human view... good lord what have you done MS.

Re: Microsoft deletes 'teen girl' AI after it became a Hitler-loving sex robot ()

#14

Post by Kequc »

DanBC wrote: post #6

What I don't understand is that this was entirely predictable because it's exactly what happened before, every time. Was that part of the research? Did they want to gather data on "trolling"[1] raids? [1] new use of the word, not traditional.

Machine learning robot gets introduced to the internet, internet catches wind there is a new machine learning robot on the internet. Proceed to teach it as many offensive phrases as possible has been pretty much internet canon for the last two decades.

I'm very disappointed to see it happen again. The biggest disappointment is that I'm sure Microsoft has added some pretty neat stuff in there. But we're never going to be able to see it if they keep making the same mistakes.

This is how far machine learning computers were able to get 10 years ago.

Re: Microsoft deletes 'teen girl' AI after it became a Hitler-loving sex robot ()

#15

Post by pmalynin »

mizzao wrote: post #11

Although in this case only words were slung, I think this scenario illustrates a potentially huge problem: horrible people combined with an AI system that can't tell right and wrong (perhaps call it morality) could result in terrible catastrophes. What if a distributed self-driving car system learned from users, and some people taught it to cut people off, drive aggressively, or even indirectly cause accidents? I don…

The car driving example is bad IMHO. Because you actually want to encode certain law breaking qualities into them. In particular if it's on a highway and everyone around you is going 10 kmh above the speed limit and yet the car isnt (unless it's encoded into software) then this will actually jeopardize the security of the whole system. So the right thing to do would be to break the law, but breaking the law is a wrong thing, and so on.

Re: Microsoft deletes 'teen girl' AI after it became a Hitler-loving sex robot ()

#17

Post by ocdtrekkie »

DanBC wrote: post #6

What I don't understand is that this was entirely predictable because it's exactly what happened before, every time. Was that part of the research? Did they want to gather data on "trolling"[1] raids? [1] new use of the word, not traditional.

I doubt Microsoft was oblivious to this. I kinda think they wanted to see how it would behave without too much coaching. It's an experiment, not a product or service. Sometimes it's not knowing if it will fail, but seeing how it fails.

And I doubt it's the last we've seen of Tay.

Re: Microsoft deletes 'teen girl' AI after it became a Hitler-loving sex robot ()

#19

Post by ozmbie »

pmalynin wrote: post #15
mizzao wrote: post #11

Although in this case only words were slung, I think this scenario illustrates a potentially huge problem: horrible people combined with an AI system that can't tell right and wrong (perhaps call it morality) could result in terrible catastrophes. What if a distributed self-driving car system learned from users, and some people taught it to cut people off, drive aggressively, or even indirectly cause accidents? I don…

The car driving example is bad IMHO. Because you actually want to encode certain law breaking qualities into them. In particular if it's on a highway and everyone around you is going 10 kmh above the speed limit and yet the car isnt (unless it's encoded into software) then this will actually jeopardize the security of the whole system. So the right thing to do would be to break the law, but breaking the law is a wron…

You can still be a horrible driver obeying the law.
Post reply on HN