Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Maybe getting rid of your QA team was bad

davidkcaudill.medium.com

Post reply on HN
Print view View original

51–60 of 269 posts

Re: Maybe getting rid of your QA team was bad

#51

Post by spookie »

wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

At least from my knowledge of the gaming world there are QA devs who do find the issues and fix them if they've the ability to do so, point out code that should be taken a look at, and all of that. I find it extremely valuable to have another set of eyes in the code with a very more focused perspective, sometimes different from the dev.

Re: Maybe getting rid of your QA team was bad

#52

Post by yungporko »

wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

yeah my experience is basically the same, usually if a place has qa at all, it's one person in a team who doesnt have an adequate environment or data set to test with and they effectively end up just watching the developer qa their own work and i end up screaming into a pillow every time i see "Tester: hi" pop up on my screen.

the one exception to this was when i was qa (never again) and i made sure we only ever did automated tests. unfortunately management was nonexistent, devs made zero effort to work with us, and naturally we were soon replaced by a cheap offshore indian team who couldn't tell you the difference between a computer and a fridge anyway.

i think a lot of it just stems from companies not caring about qa, not knowing who to hire, and not knowing what they want the people they hire to achieve. "qa" is just like "agile", where nobody can be bothered to actually learn anything about it, so they make something up and then pat themselves on the back for having it.

Re: Maybe getting rid of your QA team was bad

#53

Post by bbarn »

yabones wrote: post #23
wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

From what I've seen, the value in QA is product familiarity. Good QA'ers know more about how the product actually works than anybody else. More than PM's, more than sales, and more than most dev teams. They have a holistic knowledge of the entire user-facing system and can tell you exactly what to expect when any button gets pushed. Bad QA'ers are indeed a source of noise. But so are bad devs, sysadmins, T1/2 support…

This is a great model, until those people so familiar with the business needs end up.. doing business things instead. It's really hard to keep people like that in a QA role once the business recognizes their value. Kind of the same problem with QA automation people - once they become really good at test automation, they are effectively software developers, and want to go there.

Re: Maybe getting rid of your QA team was bad

#54

Post by amtamt »

> The most conscientious employees in your organization are the most bitter. They see the quality issues, they often address them, and they get no recognition for doing so. When they speak up about quality concerns, they get treated like mouthbreathers who want to slow down. They watch the “move fast and break things” crowd get rewarded time after time, while they run around angrily cleaning up their messes. To these folks, it feels like giving a damn is a huge career liability in your organization. Because it is.

This is the bitter truth, no one wants to acknowledge.

DBAs and Infra, are in the same boat as QAs. Pendulam will swing back in not so long time frame i hope.

Re: Maybe getting rid of your QA team was bad

#55

Post by karmakaze »

Having a QA team is like having an Ops team with stuff being 'thrown over the wall' to the downstream.

There's two kinds of tests. Regression testing, that should be automated and written and maintained by devs. New feature or change testing should be done by those that defined them, namely Product people. In the best case it's an iterative and collaborative process, where things can be evaluated in dev/test environments, staging environments, or production for beta flag enabled test users.

Re: Maybe getting rid of your QA team was bad

#56

Post by Zelphyr »

I worked at two companies 15-20 years ago that invested in top-tier QA teams. They were worth their weight in gold. The products were world class because the QA team were fantastic at finding bugs we developers didn't think of looking for because we were too close to the problem. We are too used to looking at the happy path.

One key attribute to both companies is that it was dictated from on high that the QA team had final say whether the release went to production or not.

These days companies think having the developers write automated tests and spend an inordinate amount of time worrying over code coverage is better. I can't count how many products I've seen with 100% code coverage that objectively, quantifiably doesn't work.

I'm not saying automated testing is bad. I'm saying, just as the author does, that doing away with human QA testers is.

Re: Maybe getting rid of your QA team was bad

#57

Post by spinningD20 »

evilantnie wrote: post #44

QA has always been about risk management. There are multiple ways to manage risk, and some of those ways can be more cost effective to a business. As software shifted towards SaaS offerings, deployments (and rollbacks) became quicker, customer feedback loops also got lightning fast. Team's can manage the risk of a bug more efficiently by optimizing for mean-time-to-recovery. This muscle is not one that QA teams are p…

There are so so many instances where "rolling back" is just not a feasible solution. Working for a SaaS company with mobile/web/api and huge db's, migrations, payroll uses in the product, rolling back is and should always be a LAST RESORT. In 99% of the cases, something significant enough to want to roll back usually results in a "hot patch" workflow instead because rolling back or etc has its own risk.

> QA has always been about risk management.

100%.

QA should be related to identifying risk, likelihood of failure, impact of failure to user, client and company. The earlier this is done in the varying processes, the better. ("shift left" but I've seen a ton of differences with how people describe this, but generally QA should start getting involved in the "design phase")

Another example from my own first-hand experience:

A company I worked for made a product that plugged into machines that were manufacturing parts, and based on your parameters it would tell you whether or not the part was "good" or "bad".

When interviewing the leadership of the company, as well as the majority of the engineering group, "what is the biggest risk with this product" they all said "if the product locks up!". Upon further discussion, I pulled out a much larger, insidious risk; "what if our product tells the client that the part is 'good' when it is not?"

In this example, the part could be involved in a medical device that keeps someone alive.

You're not going to be able to roll that back.

Re: Maybe getting rid of your QA team was bad

#58

Post by mattgreenrocks »

godelski wrote: post #32

The main problem with QA teams is the same problem with IT teams or even management. If they are doing their jobs well they appear to be doing nothing. This often creates a situation where people need to "justify" their jobs. Usually this happens due to an over reliance upon metrics (see Goodhart's Law) rather than understanding what the metrics are proxying and what the actual purpose of the job is. A bad QA team is…

> Shouldn't businesses be operating like the latter? Constantly training?

There's a rampant cultural mind-virus that argues that 95%th percentile is somehow tons of work (rather than a lack of unforced mistakes), so everyone just writes it off. It's on full display at this very site. Just look on any post involving software quality, and read a bunch of comments suggesting widespread apathy from engineers.

Obviously every situation is different, but people seem to be pretty okay with relinquishing agency on these things and just going along with whatever local maxima their org operates in. It's not totally their fault, but they're not blameless either.

Re: Maybe getting rid of your QA team was bad

#59

Post by marcelr »

wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

Weird, I have had the opposite experience that most shit slips through the cracks of automated testing & manual testing by an experienced QA is 10x more effective.

Re: Maybe getting rid of your QA team was bad

#60

Post by tootie »

wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

I think the thing missing from a lot of these conversations is what problem domain you're working in. The more "technical" your problem domain is the more valuable automated testing will be over manual. For almost anything based on user experience and especially mass-market customer-facing products, human QA is far more necessary.

In either case, the optimal operating model is that QA is embedded in your product team. They participate in writing tickets, in setting test criteria and understanding the value of the work being done. "Finding bugs" is a low value task that anyone can do. Checking for product correctness requires a lot more insight and intuition. Automated test writing can really go either direction, but typically I'd expect engineers to write unit tests and QA to write e2e tests and only as much or as little as it actually saves time and can satisfactorily indicated success or failure of a user journey.

Post reply on HN