I would have liked to see a more realistic discussion of when QA helps the bottom line and when it doesn't. Plenty of software is sold and tolerated by users despite having bugs. If Microsoft waited until the bug count in Windows was 0, they would never ship, and would go out of business. In other words, this blog post is telling us that the correct amount of QA is higher than "no QA", but it is not helping us determine what the right amount of QA is.
QA? We don't need no stinking QA
11–20 of 62 posts
Re: QA? We don't need no stinking QA
#12One financial services company I worked at a long time ago was getting caught up in the Lean hype. We did process maps of nearly everything including our dev process. They put a big red circle around the QA piece noting that it was a candidate for elimination since it was not value-add. That is parts like coding and deployment were necessary because they actually were important and helped move business ideas to the c…
If the team considers automated testing waste then they truly are missing the point of Lean. I disagree with you on the fact that having a QA step could be considered waste. Note that this isn't the same as calling QA waste.
IMO QA is something that should be included throughout the task lifecycle. Sure, after a task is marked as complete there should be some sort of usability testing, but that isn't the sole job of QA. They should be working before tasks are even started by defining the exit criteria and writing integration tests. Giving QA its own step is wasteful because its effectively pigeonholing the job.
Re: QA? We don't need no stinking QA
#13I would have liked to see a more realistic discussion of when QA helps the bottom line and when it doesn't. Plenty of software is sold and tolerated by users despite having bugs. If Microsoft waited until the bug count in Windows was 0, they would never ship, and would go out of business. In other words, this blog post is telling us that the correct amount of QA is higher than "no QA", but it is not helping us determ…
As a former QA myself, we never insisted on the bug count being 0, instead we focused on the "critical" bug count being 0 and the "major" bug count being as close to 0 as possible before shipping the product. Often it comes down to a negotiation between QA, dev and product but a good company will give QA a great deal of input in when a product is ready to ship.
Re: QA? We don't need no stinking QA
#14quoting Linus: "There really are only two acceptable models of development: 'think and analyze' or 'years and years of testing on thousands of machines'"[1]
i would never say that "we don't need no stinking QA", but i do believe that teams who live by "think and analyse" need QA a lot less than teams who don't.
[1] http://www.dustingetz.com/linus-think-and-analyze-motherfuck...
Re: QA? We don't need no stinking QA
#15I would have liked to see a more realistic discussion of when QA helps the bottom line and when it doesn't. Plenty of software is sold and tolerated by users despite having bugs. If Microsoft waited until the bug count in Windows was 0, they would never ship, and would go out of business. In other words, this blog post is telling us that the correct amount of QA is higher than "no QA", but it is not helping us determ…
As a former QA myself, we never insisted on the bug count being 0, instead we focused on the "critical" bug count being 0 and the "major" bug count being as close to 0 as possible before shipping the product. Often it comes down to a negotiation between QA, dev and product but a good company will give QA a great deal of input in when a product is ready to ship.
Agreed. At thing point, any software company with a modicum of experience knows that reaching a 0 bug count is not only impossible, but a waste of money. You get out the new features that you want and make sure it works well, and without causing regressions. Any bugs that inhibit functionality that 80% of the users would use needs to be corrected, as well as any bugs that make us look bad (UI bugs like spelling mistakes, etc).
Re: QA? We don't need no stinking QA
#16I worked at a QA-heavy financial organisation.
The only reason they needed QA is the software was a giant toilet-clogging turd. There were no formal unit or integration test cases so it was used to catch regressions against massive test plans knocked up in Excel, Word and god-knows what else.
Re: QA? We don't need no stinking QA
#17One thing also needs to be mentioned, is that supportability of features also needs to be something that has to be addressed early on. The unintended consequence of continuing to support every single combination of every single option makes your support matrix, as well as your QA effort, enormous, and your code incredible unmaintainable.
I work at a company where they have a 5 year old product, and the mentality at the time was "Yes I know it doesn't make sense but if the user does X, then it could cause the server to crash, so let's do Y." This grew to a series of matrices that has now turned the act of adding 1 single feature into a 3 pages worth of "If A then B, otherwise C". This has made the QA effort in terms of testing incredibly complex, and even developers get confused as to what is supported and what isn't.
For example, there is an instance where copying a directory takes a different code path than copying a single file. Why? Because they wanted to "special case" this situation. It makes the functionality and behavior utterly unpredictable.
The takeaway that comes with experience is don't spend too much time trying to prevent your users from doing stupid things. If they want to do stupid things and the system crashes, it's okay to say "Don't do it, otherwise it will crash." This keeps your feature set, your code, and your QA effort much cleaner and more maintainable over the long haul.
Re: QA? We don't need no stinking QA
#18I mostly agree. That MMO that is getting away with using customers as QA, and only fixing bugs that NEED fixing? I bet they're making a nice profit. I'm sure the customers are annoyed, but in my experience, gamers won't quit playing over bugs. Bad customer service, price hikes, etc... Sure. But not bugs. Even if they stick around for years. However, business applications are a different story. Businesses can't afford…
Lately my interest has been piqued by analytical code verification techniques and/or fuzzing (going beyond simple static verification or "checking the code style"). I think a lot of bugs made in day-to-day coding could be found given such methods. It would be best to integrate something like that into the compiler (and IDE) as it has the necessary information and structures. I believe the clang team is working on thi…
Hell yes you are spot on right there.
I've got into a habit of defining pre and post-call checks on just about everything. It's knocked down my regression and defect rate quite considerably. Assumptions are dangerous - so always check them.
Microsoft are actually leading this if you ask me with PEX:
Try it here: http://www.pexforfun.com/
Re: QA? We don't need no stinking QA
#19I have been working in QA for a few years now and have made several observations.
* The people who are good at QA usually have the skills to work as a developer. QA is generally not as desirable as development to a young developer, so why do it?
* Many QA people are almost completely useless and exist just to generate paperwork so that the company can cover its ass. Warning signs for bad QA people seem to be certifications and fear of anything that does not strictly adhere to classic waterfall development. The bulk of all defects are found by a small number of people on my team.
* Most developers are horrible at testing and writing unit tests. We have thousands of unit tests and the main thing that they catch is that someone did not update the test after refactoring code.
Re: QA? We don't need no stinking QA
#20quoting Linus: "There really are only two acceptable models of development: 'think and analyze' or 'years and years of testing on thousands of machines'"[1] i would never say that "we don't need no stinking QA", but i do believe that teams who live by "think and analyse" need QA a lot less than teams who don't. [1] http://www.dustingetz.com/linus-think-and-analyze-motherfuck...
Linus is in no way representative of the average developer, nor are most people on HN. Most of the developers in the industry are barely competent.