Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Poor Software QA Is Root Cause of TAY-Fail (Microsoft's AI Twitter Bot)

exploringpossibilityspace.blogspot.com

Post reply on HN
Print view View original

51–57 of 57 posts

Re: Poor Software QA Is Root Cause of TAY-Fail (Microsoft's AI Twitter Bot)

#51

Post by DonaldFisk »

galistoca wrote: post #49
DonaldFisk wrote: post #42

Earlier quoted context omitted.

I do have an idea. I first got involved in AI in the early 1980s, and worked on it on-and-off (mostly on) well into the 1990s. Some of the non-AI stuff I did back then is now, for reasons I can't quote fathom, classified as AI. A certain amount of common-sense, and basic political knowledge, should have been included in the Tay. I don't think there's any way of avoiding it. I'm skeptical of general application of the…

If you really are experienced as you say you are in AI, you wouldn't talk about these things as if they were easy to implement. I won't assume things but I can say I'm probably not so less experienced than yourself, and I know basically everything in this field belongs to "easier said than done" category. Most researchers just run experiments in closed environments just like you said and that's what makes them useles…

I never implied it was easy to implement. I also said I was skeptical of the recent "free lunch" approach to AI.

It was a failure. It didn't understand the remarks it was making, unlike a real Holocaust denier, a 4chan user, or a stand-up comic whose joke just fell flat.

Re: Poor Software QA Is Root Cause of TAY-Fail (Microsoft's AI Twitter Bot)

#52

Post by jdp23 »

MrMeritology wrote: post #44
jdp23 wrote: post #23

It doesn't sound like the author's done much root cause analysis. QA typically doesn't cause defects (it can only prevent them), so there's almost always a deeper cause. And, QA's not necessarily the best way to find bugs like this. In this case, it sounds like to me like it was most likely a requirements problem -- a missing requirement for being 4chan-resilient, if you will. A different way to look at it is that it…

I just wrote a follow-up post giving more evidence: http://exploringpossibilityspace.blogspot.com/2016/03/micros... I call it "poor software QA" because, generally, the software QA process is supposed to detect and prevent defects from 1) being introduced in the first place; and 2) from being propagated into "production" versions. As my most recent post shows, the "repeat after me" rule was a legacy of a software lib…

Thanks for the reply, and interesting followup with ALICE.

If you're using "software QA failure" in the very general sense of "a defect got introduced and then propagated to production", then yes that's what happened here. But then you're essentially saying "the root cause of this defect is that a defect got introduced and then propagated". This isn't useful for process improvement (it's true for every defect, so doesn't give any insight into what happened).

If you're right about them reusing ALICE, then a more useful way of looking at the root cause of the repeat-after-me bug is "component reuse without considering the attack model". That highlights other situations where there are risks of similar defects, and points to ways to prevent or detect similar defects.

Since there were other bugs as well, the requirement and/or design issues might still be a better candidate for root cause for the whole Tay-fail. One of the things you discover doing root cause analysis is that there are almost always multiple contributors, and you typically want to make process changes at multiple levels.

Re: Poor Software QA Is Root Cause of TAY-Fail (Microsoft's AI Twitter Bot)

#53

Post by galistoca »

DonaldFisk wrote: post #51
galistoca wrote: post #49

Earlier quoted context omitted.

If you really are experienced as you say you are in AI, you wouldn't talk about these things as if they were easy to implement. I won't assume things but I can say I'm probably not so less experienced than yourself, and I know basically everything in this field belongs to "easier said than done" category. Most researchers just run experiments in closed environments just like you said and that's what makes them useles…

I never implied it was easy to implement. I also said I was skeptical of the recent "free lunch" approach to AI. It was a failure. It didn't understand the remarks it was making, unlike a real Holocaust denier, a 4chan user, or a stand-up comic whose joke just fell flat.

Kids pick up words from adults. They too don't completely understand the words they use when they first start picking up and using new vocabulary but the usage tend to get calibrated based on social feedback.

Re: Poor Software QA Is Root Cause of TAY-Fail (Microsoft's AI Twitter Bot)

#55

Post by MrMeritology »

jdp23 wrote: post #52
MrMeritology wrote: post #44

Earlier quoted context omitted.

I just wrote a follow-up post giving more evidence: http://exploringpossibilityspace.blogspot.com/2016/03/micros... I call it "poor software QA" because, generally, the software QA process is supposed to detect and prevent defects from 1) being introduced in the first place; and 2) from being propagated into "production" versions. As my most recent post shows, the "repeat after me" rule was a legacy of a software lib…

Thanks for the reply, and interesting followup with ALICE. If you're using "software QA failure" in the very general sense of "a defect got introduced and then propagated to production", then yes that's what happened here. But then you're essentially saying "the root cause of this defect is that a defect got introduced and then propagated". This isn't useful for process improvement (it's true for every defect, so doe…

I'm not using "software QA" in the very broad way you describe in the first sentence. Also, "defect" as I use it does not mean every short-coming of the product. It means "something doesn't work (or failed) as designed or required."

Context: my blog posts are meant to contrast with "experts" who claimed that poisoning social AI was just in the nature of learning systems, even when they worked as designed (i.e. had no defects). They are claiming that Tay learned to be foul mouthed and racist, and thus had become foul mouthed and racist.

If that were true, then this undesirable behavior would not be a software QA problem. The AI would be working as designed. No QA process would change things.

In contrast, I'm claiming (from evidence) that the main problem in Tay is due to a hidden feature in a reused library + rule set that should have been detected and removed in a QA process that considered various attacks. BTW, this attack (getting bot to repeat naughty words) has been around since ELIZA in the 60s.

The other failings of Tay (esp. no black list) are design and requirement failures, not QA failures.

Re: Poor Software QA Is Root Cause of TAY-Fail (Microsoft's AI Twitter Bot)

#56

Post by jdp23 »

MrMeritology wrote: post #55
jdp23 wrote: post #52

Earlier quoted context omitted.

Thanks for the reply, and interesting followup with ALICE. If you're using "software QA failure" in the very general sense of "a defect got introduced and then propagated to production", then yes that's what happened here. But then you're essentially saying "the root cause of this defect is that a defect got introduced and then propagated". This isn't useful for process improvement (it's true for every defect, so doe…

I'm not using "software QA" in the very broad way you describe in the first sentence. Also, "defect" as I use it does not mean every short-coming of the product. It means "something doesn't work (or failed) as designed or required." Context: my blog posts are meant to contrast with "experts" who claimed that poisoning social AI was just in the nature of learning systems, even when they worked as designed (i.e. had no…

> I'm not using "software QA" in the very broad way you describe in the first sentence.

I took this description from your earlier comment "I call it "poor software QA" because 1) and 2)" so yes, you are using it that way at least sometimes :)

Anyhow we obviously see things differently on the root cause side (and both of us are on the outside so there's a lot we don't know). That said I certainly agree that it's a defect, and that it's an attack that could reasonably have been anticipated.

Post reply on HN