Live data from Hacker News

Should QA exist?

rubick.com

61–70 of 160 posts

Re: Should QA exist?

#61

Earlier quoted context omitted.

More like certified products. New ISO standard may require professional liability for software products, which will be adopted as requirement by big consumers and will pull the industry into certification loop, because insurers will ask for it. This will obviously put a high entry barrier to many product categories, slowing down innovation.

Yes, but slowing down to avoid hazards is sometimes important. Medical devices and such are the only places I’d expect to see the need for certified products. By extension, in the new era, we really ought only expect certified software where we expect a duty to care from the software system (or any other assigned duty)

In development of medical devices existing quality controls are already working well, right?

Re: Should QA exist?

#62

Earlier quoted context omitted.

Yes, but slowing down to avoid hazards is sometimes important. Medical devices and such are the only places I’d expect to see the need for certified products. By extension, in the new era, we really ought only expect certified software where we expect a duty to care from the software system (or any other assigned duty)

In development of medical devices existing quality controls are already working well, right?

My point exactly, embedded devices are the closest software gets to actually being built by licensed engineers. The expectation can often be that you are an electrical engineer by training, where licensure is a viable path, unlike in software engineering.

Re: Should QA exist?

#63

If your product is used by humans, then it needs to be tested by humans - this cannot be automated. Those humans can be your QA people, or your customers. Perhaps your customers are happy to be testers, perhaps not. Unit tests are very expensive and return little value. Conversely, a (manual?) 'smoke test' is very cheap and returns great value - the first thing you do when updating a server for example is to check it…

> Unit tests are very expensive and return little value Why are unit tests very expensive? This goes against everything I know.

Unit tests very roughly double the amount of effort required to make any meaningful change to your codebase. They are also require maintenance same as ordinary code - but the customer does not care in the slightest whether or not they pass. On the other side, they can only really tell you about low level bugs that you already expected, they cannot surface system level bugs - the actual hard bugs that cause problems for you and your customers.

Then there is the danger of thinking that green=all good, an example of 'automation bias' where we learn to trust the automation even as things go wrong.

As makers, it is also tempting to believe that [all] problems can be solved by making something (i.e. code), but actually many problems are not of that nature, and cannot be solved in that way.

Re: Should QA exist?

#64
So many of these articles talk about why a particular role or type of role within an org should be there or not, but they fail to touch on the 'theory' of why or why not. This article has that same lack of foundation, and so meanders around a bit, IMHO.

Any process in an organization of size will have indicators that measure output. Those indicators should typically be paired with indicators that measure the quality of the output, to ensure product or service levels. That's the theory, and the genesis of 'quality management': whether you're measuring output code or breakfasts [1] or chemicals or widgets or medicine, you need to measure the quality of the output if there are any client specifications or expectations around the output. And there are very few cases where your customers will not have any specs or expectations around your product or service.

How you manage quality follows from those basics; it matters where you measure quality but it is so process dependent - earlier in the process lowers costs, but may not suffice to guarantee final quality - that quality management has to be designed around the specific process; balancing cost with benefit and requirements. How deep or specialized quality management becomes depends on the needs of the org, the size of the org, and the needs of the particular process.

This is why I'm skeptical about whether broad articles like this are beneficial overall. Why and how matter, and where's the foundational discussion behind why and how? Do folks not think at the organizational/business level? Maybe not everyone is a Sheryl Sandberg :-)

[1] https://en.wikipedia.org/wiki/High_Output_Management

Re: Should QA exist?

#65

Is this the real life? Is this just fantasy? If engineering owns quality, then engineering own all, up the chain. No need for anything and anybody. Which is the AI pipe dream, really. You are, in fact, with using AI, QA or coding or otherwise, externalizing services in hope the services will improve and costs will drop. Let me know how that goes without HITLs.

> Is this the real life? Is this just fantasy?

Caught in a landslide, no escape from reality ...

:-)

Re: Should QA exist?

#66
I ship a very visible product which, when it breaks, generates a lot of social media angst (it's in the gaming adjacent space). So we try not to break things to the best of our ability. We have very few QA people and have whittled down that team over the past few years (DevOps was eliminated during the first round of layoffs ~2023).

This was painful at first but I do think it's the way to go. We found that too much manual QA incentivizes devs writing features to throw it over the fence - why should they test more if someone else is paid to do it? Devs need to feel the pain of writing tests if their code is hard to test, and they need to be held accountable when their code blows up in production. This feedback loop is valuable in the long run.

Same thing for test automation. Previously we shipped this over to our in-team DevOps people and they built complicated CI/CD setups. Losing them meant we needed to simplify our stack. Took a while and it slowed down feature development, but it was worth it. Of course you need leadership who understands this and dedicates time to building this out.

In defense of DevOps, I think the landscape for automation was poor a few years back. Jenkins and Teamcity are way too complex. Github Actions (for all its warts, and there are many) is much simpler. Our pipelines are also in their own CI/CD (CDK, CodeBuild) - infrastructure as code is the key to scaling.

We still have manual QA people to test things we can't automate. Usually this is for weird surfaces such as smart TVs, or for perceptual tests. I don't see this going away any time soon, but high levels of automation elsewhere drive down the need for "catch-all" manual testing.

Re: Should QA exist?

#68

100% and I’m a software developer and have been for ~30 years. Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering” and that it should exist. One of the QA people I work with currently is one of my favorite people. They don’t always make me happy (in the moment) with their bugs or with how they decide to break the software,…

Exactly. I spent 20 years split between MS and Apple. Some of the best people I ever worked with were in QA. One guy in particular was an extremely talented engineer who simply didn't enjoy the canonical "coding" role; what he did enjoy was finding bugs and breaking things. ;-)

Really? The best people I worked with were never QA.

Moreover, the best QAs would almost always try to be not QA - to shift into a better respected and better paid field.

I wish it werent so (hence my username) but there is a definite class divide between devs and QA and it shows up not just in terms of the pay packets but also who gets the boot in down times and who gets listened to. This definitely affects the quality of people.

I think it's overdue an overhaul much like the sysadmin->devops transition.

Re: Should QA exist?

#69

Earlier quoted context omitted.

Exactly. I spent 20 years split between MS and Apple. Some of the best people I ever worked with were in QA. One guy in particular was an extremely talented engineer who simply didn't enjoy the canonical "coding" role; what he did enjoy was finding bugs and breaking things. ;-)

Really? The best people I worked with were never QA. Moreover, the best QAs would almost always try to be not QA - to shift into a better respected and better paid field. I wish it werent so (hence my username) but there is a definite class divide between devs and QA and it shows up not just in terms of the pay packets but also who gets the boot in down times and who gets listened to. This definitely affects the qual…

We have differing experiences, which shouldn't be surprising. My example explicitly referred to someone who was a good engineer who enjoyed the QA role.

This might have been an Apple/MS thing, but we always had very technical QA people on the dev tools team. For example, the QA lead for the C++ compiler had written their own compiler from scratch and was an amazing contributor.

Re: Should QA exist?

#70

100% and I’m a software developer and have been for ~30 years. Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering” and that it should exist. One of the QA people I work with currently is one of my favorite people. They don’t always make me happy (in the moment) with their bugs or with how they decide to break the software,…

Agreed. QA specialists are there to think about what the engineer didn't think about. Unless the engineer is incompetent or the organization is broken, the engineer has already written tests for everything they could think of, but they can't think of everything. More importantly, it is almost impossible for engineers to be as well incentivized to spend extra time exploring edge cases in something they already believe…

Ive almost never worked on a project where there was the right number of QAs who were doing the right thing.

Usually there either arent any in which case bugs get missed or there are 5 very cheap ones running mindless scripts who are standing in for the devs' inability or unwillingness to write decent automated tests but dont catch the really deep level thorny stuff.

Post reply on HN