Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Poor Software QA Is Root Cause of TAY-Fail (Microsoft's AI Twitter Bot)

exploringpossibilityspace.blogspot.com

Post reply on HN
Print view View original

21–30 of 57 posts

Re: Poor Software QA Is Root Cause of TAY-Fail (Microsoft's AI Twitter Bot)

#21

Post by stcredzero »

Mikeb85 wrote: post #14

'Learning from the internet' was destined to fail. The internet is filled with so much sarcasm, so many memes and trolling that this was inevitable. Their whole premise just sucked. And of course, once people realized they could influence the bot, everyone just upped the level of trolling... On the plus side, I was entertained. I'm sure many people were (maybe not Microsoft investors). It was pretty damn hilarious.

And of course, once people realized they could influence the bot, everyone just upped the level of trolling...

Science fiction premise: We create a truly sentient AI. The Internet's immediate knee-jerk reaction is massive trolling. AI decides to destroy humanity because the vast majority of the data we've supplied to it indicates we're massive assholes. (Also an addendum to the category, "This is why we can't have nice things.")

Re: Poor Software QA Is Root Cause of TAY-Fail (Microsoft's AI Twitter Bot)

#22

Post by colllectorof »

hyperion2010 wrote: post #18

Garbage in garbage out. Blaming anyone or anything other than the environment the algorithm was put in is absurd.

Blaming environment for algorithm's failure only makes sense if you're prepared to credit it for algorithm's success. Otherwise your evaluation is entirely one-sided.

Re: Poor Software QA Is Root Cause of TAY-Fail (Microsoft's AI Twitter Bot)

#23

Post by jdp23 »

It doesn't sound like the author's done much root cause analysis. QA typically doesn't cause defects (it can only prevent them), so there's almost always a deeper cause. And, QA's not necessarily the best way to find bugs like this.

In this case, it sounds like to me like it was most likely a requirements problem -- a missing requirement for being 4chan-resilient, if you will. A different way to look at it is that it's a design flaw that they system is manipulable by default.

True, once you've identified the root cause of the defect, it's also important to look at why it didn't get detected. So yeah, QA missed it. Then again, so did design and code reviews, developer testing, threat modeling, and the social-manipulation equivalent of penetration testing. Of course these all complementary approaches to "quality" but typically they are not the responsibility of QA.

I wonder if the author also thinks the root causes of the errors in his original post are also QA failures?

Re: Poor Software QA Is Root Cause of TAY-Fail (Microsoft's AI Twitter Bot)

#26

Post by 13of40 »

One thing we're probably not considering here is that the veteran QA person might have said "that's a ludicrous idea, people are just going to troll it mercilously" and the PM with college plus a year and a half of industry experience told him to shut the hell up.

Re: Poor Software QA Is Root Cause of TAY-Fail (Microsoft's AI Twitter Bot)

#28

Post by galistoca »

It's arrogant to think that any kind of QA could prevent how an AI would turn out. That's probably why the terminators took over humans in the movie--because humans were arrogant enough to think that doing enough QA would prevent everything. What should happen instead is you need to be humble and assume that things won't go the way you designed them to, that's the "safe" way to build AIs in the long term. The OP uses the "repeat after me" feature as the QA fail, but he's overlooking how it could have gone wrong in many other ways even if they didn't have that problem. No matter how robust system you build there always will be hackers who try to manipulate it (in fact it's more satisfying to hack a robust system than brittle one)

Re: Poor Software QA Is Root Cause of TAY-Fail (Microsoft's AI Twitter Bot)

#29

Post by Pxtl »

stcredzero wrote: post #21
Mikeb85 wrote: post #14

'Learning from the internet' was destined to fail. The internet is filled with so much sarcasm, so many memes and trolling that this was inevitable. Their whole premise just sucked. And of course, once people realized they could influence the bot, everyone just upped the level of trolling... On the plus side, I was entertained. I'm sure many people were (maybe not Microsoft investors). It was pretty damn hilarious.

And of course, once people realized they could influence the bot, everyone just upped the level of trolling... Science fiction premise: We create a truly sentient AI. The Internet's immediate knee-jerk reaction is massive trolling. AI decides to destroy humanity because the vast majority of the data we've supplied to it indicates we're massive assholes. (Also an addendum to the category, "This is why we can't have ni…

That sounds like an SMBC comic. Or the plot of Age of Ultron, if it had been slightly better-written so Ultron could have an actual motivation.

Re: Poor Software QA Is Root Cause of TAY-Fail (Microsoft's AI Twitter Bot)

#30

Post by 13of40 »

apalmer wrote: post #27

I am not understanding the spin that is being displayed in general with regards to Tay. Microsoft created a chat bot, the chat bot chatted. It wasnt a failure on any technical level as far as i have seen.

It's a failure of censorship. Like someone else pointed out, they should have had a simple blacklist of certain words.
Post reply on HN