Live data from Hacker News

Maybe getting rid of your QA team was bad

davidkcaudill.medium.com

41–50 of 269 posts

Re: Maybe getting rid of your QA team was bad

#41

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

I was surprised by the opposite of this (after entering my first real job at Google, after startup founder => seller.)

People wrote off QA completely unless it meant they didn't have to write tests, but, it didn't track from my (rather naive) perspective that tests are _always_ part of coding.

From that perspective, it seemed QA should A) manage go/nogo and manual testing of releases B) keep the CI green and tasks assigned for red (bonus points if they had capacity to try fixing red) C) longer term infra investments, ex. what can we do to migrate manual testing to integration testing, what can we do to make integration testing not-finicky in the age of mobile

I really enjoyed this article because it also indicates the slippery slide I saw there: we had a product that had a _60% success rate_ on setup. And the product was $200+ dollars to buy. In retrospect, the TL was into status games, not technical stuff, and when I made several breakthroughs that allowed us to automate testing of setup, they pulled me aside to warn me that I should avoid getting into it because people don't care.

It didn't compute to me back then, because leadership _incessantly_ talked about this being a #1 or #2 problem in quarterly team meetings.

But they were right. All that happened was my TL got mad because I kept going with it, my skip manager smiled and got a bottle of wine to celebrate with, I got shuffled off to work with QA for next 18 months, and no one ever really mentioned it again.

Re: Maybe getting rid of your QA team was bad

#43
post #23

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

From what I've seen, the value in QA is product familiarity. Good QA'ers know more about how the product actually works than anybody else. More than PM's, more than sales, and more than most dev teams. They have a holistic knowledge of the entire user-facing system and can tell you exactly what to expect when any button gets pushed. Bad QA'ers are indeed a source of noise. But so are bad devs, sysadmins, T1/2 support…

> Good QA'ers know more about how the product actually works than anybody else. More than PM's, more than sales, and more than most dev teams.

Not disagreeing with this, but there's one thing they won't always be aware of. They won't always know what code a dev touched underneath the hood and what they might need to recheck (short of a full regression test every single time) to verify everything is still working.

I know that the component I adjusted for this feature might have also affected the component over in spots X, Y, and Z, because I looked at that code, and probably did a code search or a 'find references' check at some point to see where else it's getting called, and also I usually retest those other places as well (not every dev does, though. I've met some devs that think it's a waste of time and money for them to test anything and that's entirely QA's job).

A good QA person might also intuit other places that might be affected if it's a visible component that looks the same (but either I haven't worked with too many good QA people or that intuition is pretty rare, I'm guessing it's the latter because I believe I have worked with people who were good at QA). Because of that, I do my best to be proactive and go "oh by the way this code might have affected these other places, please include those in your tests".

Re: Maybe getting rid of your QA team was bad

#44
QA has always been about risk management. There are multiple ways to manage risk, and some of those ways can be more cost effective to a business. As software shifted towards SaaS offerings, deployments (and rollbacks) became quicker, customer feedback loops also got lightning fast. Team's can manage the risk of a bug more efficiently by optimizing for mean-time-to-recovery. This muscle is not one that QA teams are particularly optimized for, thus their effectiveness in this new model was reduced. I've found that holding on to QA function in this environment can severely dilute the ownership of quality as a requirement from engineers.

QA is still extremely valuable in any software that has long deployment lead times. Mobile apps, On-Prem solutions, anything that cannot be deployed or rolled back within minutes can benefit from a dedicated QA team that can manage the risk appropriately.

Re: Maybe getting rid of your QA team was bad

#45
post #22

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

the engineering team are usually great at writing tests that test their code, a good QA can test alongside them to find cases they've missed and issues that automated code tests can't find. The QA person doesn't have to spend time checking that the app basically works, they can be confident in that and spend their time testing for other 'qualities' But yes, I've known QA teams that will only find bugs that no one car…

It seems so obvious to me that your typical engineer, who spent hours / days / whatever working on a feature, is never going to test the edge cases that they didn’t conceive of during implementation. And if they didn’t think of it, I bet they’re not handling it correctly.

Sometimes that’ll get caught in code review, if your reviewer is thinking about the implementation.

I’ve worked in payroll and finance software. I don’t like it when users are the ones finding the bugs for us.

Re: Maybe getting rid of your QA team was bad

#46

I completely agree with the sentiment of this article: It is a big problem that being a "software tester" is not at all as prestigious as being a software engineer. Having someone who really understands how the users interact with the software and systematically covers all behavior in test cases is very valuable. I experienced both worlds: I worked in an organization where 4 QA engineers tested each release that was…

what industry is that?

Re: Maybe getting rid of your QA team was bad

#49

Agree with the sentiment of this article but the disturbing ai generated images every paragraph were definitely not necessary - do people actually need to see these?

It did strike me as ironic that the article is about ruthlessly automating to avoid paying QA engineers, and it uses AI to avoid paying illustrators.

Re: Maybe getting rid of your QA team was bad

#50
post #36

Earlier quoted context omitted.

Frameworks like playright can record as code user actions and you can replay them in a test. So you can make your QA teams create plenty of tests if you give them the right tools.

In my experience such tests are brittle as hell.

You're not wrong, but a good, well resourced QA org can both help write or develop more flexible tests, and also help fix brittle tests when they do break. The idea of brittle tests that break often being a blocker is predicated on practices like running every type of test on every commit that exist to deal with a lack of QA effort in the first place.

Maybe recorded integration tests are run on every release instead of every commit? Maybe the QA team uses them less to pass/fail work and more to easily note which parts of the product have changed and need attention for the next release? There's lots of possibilities.

Post reply on HN