Maybe getting rid of your QA team was bad • Hacker News Board

Re: Maybe getting rid of your QA team was bad

#81

Post by g051051 » Thu, Dec 14, 2023, 8:59 PM UTC

wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

The last two organizations I worked for had full QA teams with people who wrote the tests, not just test plans. The devs sometimes provided features to facilitate it, but the QA teams were the ones that constructed the tests, ran them, and decided if the software was ready to be released. Some things had manual tests, but a large percentage was fully automated.

Re: Maybe getting rid of your QA team was bad

#82

Post by kgermino » Thu, Dec 14, 2023, 9:00 PM UTC

taylodl wrote: post #63

yabones wrote: post #23

Earlier quoted context omitted.

From what I've seen, the value in QA is product familiarity. Good QA'ers know more about how the product actually works than anybody else. More than PM's, more than sales, and more than most dev teams. They have a holistic knowledge of the entire user-facing system and can tell you exactly what to expect when any button gets pushed. Bad QA'ers are indeed a source of noise. But so are bad devs, sysadmins, T1/2 support…

To your point, the QA team is the customer's advocate. As you say, they know the product, from the customer's perspective , better than anyone else in the development organization. Where I've seen QA teams most effective is providing more function than "just" QA. I've seen them used for 2nd tier support. I've seen them used to support sales engineers. I've also seen QA teams that take their manual test plans and auto…

That also means they test from a different perspective than the dev does. If I get a requirement my build is based on my understanding of the requirement, and so is my testing.

A separate QA person coming at it from the customer's perspective will do a test that's much more likely to reflect reality.

Re: Maybe getting rid of your QA team was bad

#83

Post by solardev » Thu, Dec 14, 2023, 9:01 PM UTC

wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

I've only ever had an official QA team in one job, at a Fortune 1000. When I started we didn't have anyone yet, but eventually they hired an mid-manager from India and brought him over (as in relocated his whole family). He then brought on a QA person he had worked with previously.

I did not work well with the mid-manager, who was both my new boss and the QA person's (not too relevant here). However, I do give him credit for the person he hired.

That QA person, a young Indian woman with some experience, was actually phenomenal at her job, catching many mistakes of ours both in the frontend and in the APIs.

She not only did a bunch of manual testing (and thus discovered many user-facing edge cases the devs missed), she wrote all the test cases (exhaustively documented them in Excel, etc. for the higher-ups), AND the unit tests in Jest, AND all the end-to-end tests with Playwright. It drastically improved our coverage and added way more polish to our frontend than we otherwise would've had.

Did she know everything? No, there was some stuff she wasn't yet familiar with (namely DOM/CSS selectors and Xpath), and it took some back-and-forth to figure out a system of test IDs that worked well enough for everyone. She also wasn't super fluent with the many nuances of Javascript (but really, who is). There was also a bit of a language barrier (not bad, but noticeable). Overall, though, I thought she was incredible at her job, very bright, and ridiculously hard-working. I would often stay a little late, but she would usually be there for hours after the end of the day. She had to juggle both the technical dev/test tasks, the cultural barriers, and managing both up and across (as in producing useless test case reports in Excel for the higher ups, even though she was also writing the actual tests in code), dealing with complex inter-team dynamics, etc.

I would work with her again any day, and if I were in management, I'd have promoted the heck out of her, trained her in whatever systems/languages she was interested in learning, or at least given her a raise if she wanted to stay in QA. To my knowledge the company didn't have a defined promotion system though, so for as long as I was there, she remained QA :( I think it was still better than the opportunities she would've had in India, but man, she deserved so much more... if she had the opportunities I did as an American man, she'd probably be a CTO by now.

Re: Maybe getting rid of your QA team was bad

#84

Post by mbb70 » Thu, Dec 14, 2023, 9:01 PM UTC

wmichelin wrote: post #27

yabones wrote: post #23

Earlier quoted context omitted.

From what I've seen, the value in QA is product familiarity. Good QA'ers know more about how the product actually works than anybody else. More than PM's, more than sales, and more than most dev teams. They have a holistic knowledge of the entire user-facing system and can tell you exactly what to expect when any button gets pushed. Bad QA'ers are indeed a source of noise. But so are bad devs, sysadmins, T1/2 support…

Agreed! I did have some good experiences at my last job with the QA team, but it was definitely a unique model. They were really a "Customer Success" team, it was a mix of QA, sales, and customer support. These "Customer Support" reps, when functioning as QA, knew the product better than product or eng, exactly how you're describing. I did enjoy that model, but they also did not write tests for us. They primarily exe…

We had no dedicated QA, but would consistently poach "Customer Success" team members for critical QA work for the exact reasons your listed. Worked quite well for us.

Especially for complex products that are based on users chaining many building blocks together to create something useful, devs generally have no visibility into how users work and how to test.

Re: Maybe getting rid of your QA team was bad

#85

Post by ThalesX » Thu, Dec 14, 2023, 9:02 PM UTC

I am currently working with a startup that spends a lot of time on building tests that need to be refactored every sprint because it's early stage and the assumptions change. I am shocked at the amount of developer-hours spent on tests that need to be disabled / deleted instead of just hiring 1 - 2 super cheap manual testers that just go through the flows days in and out.

For me it's a no brainer, if I were CEO / CTO, until product-market-fit is achieved and exponential growth is visible, I'd just outsource Q&A and that's that.

Re: Maybe getting rid of your QA team was bad

#86

Post by spinningD20 » Thu, Dec 14, 2023, 9:02 PM UTC

natbennett wrote: post #80

Strongly disagree with the literal premise of this post. The idea of having a separate team with the mandate to “ensure quality” was always ridiculous. Occasionally it was helpful by accident but it was always ridiculous. “Quality” isn’t something you can bake in afterwards by testing a bunch. Getting rid of everyone with testing expertise, and treating testing expertise as inherently less valuable that implementatio…

Doing every quality activity "after the fact" I agree is the issue. That's the root of the problem you're seeing, not that there was a separate quality team.

Re: Maybe getting rid of your QA team was bad

#87

Post by me_smith » Thu, Dec 14, 2023, 9:03 PM UTC

wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

Hello. I am QA that writes tests for engineering. Technically, my title is a Software Development Engineer in Test (SDET). Not only do I write "test plans", I work on the test framework, infrastructure and the automation of those test plans.

Every company is different on how they implement the QA function. Whether it be left to customer, developers, customer support, manual only QA, or SDET. It really comes down to how much leadership values quality or how leadership perceives QA.

If a company has a QA team, I think the most success comes when QA get involved early in the process. If it is a good QA team, they should be finding bugs before any code is written. The later they are involved, the later you find bugs (whether the bugs are just "noise" or not) and then the tighter they get squeezed between "code complete" and release. I think that the QA team should have automation skills so more time is spent on new test cases instead of re-executing manual test cases.

Anyways, from my vantage point, the article really hits hard. QA are sometimes treated as second class citizens and left out of many discussions that can give them the context to actually do their job well. And it gets worse as the good ones leave for development or product management. So the downward spiral is real.

Re: Maybe getting rid of your QA team was bad

#88

Post by rwmj » Thu, Dec 14, 2023, 9:03 PM UTC

wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

That's only your personal experience because our QE team at Red Hat spend a very large amount of their time coding new tests or automating existing ones. They use this framework: https://avocado-vt.readthedocs.io/en/latest/Introduction.htm...

Re: Maybe getting rid of your QA team was bad

#89

Post by trealira » Thu, Dec 14, 2023, 9:03 PM UTC

JohnFen wrote: post #70

wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

The type of testing QA should be doing is different from the type of testing that devs should be doing. One doesn't substitute for the other.

I remember Steve Maguire saying this in Writing Solid Code (that they're both necessary, and both types of testing complement the other). He criticized Microsoft employees who relied on QA to find their bugs. He compared QA testing to a psychologist sitting down with a person and judging whether the person is insane after a conversation. The programmer can test from the inside out, whereas QA has to treat the program like a black box, with outputs and effects resulting from certain inputs.

Re: Maybe getting rid of your QA team was bad

#90

Post by giantrobot » Thu, Dec 14, 2023, 9:05 PM UTC

cableshaft wrote: post #43

yabones wrote: post #23

Earlier quoted context omitted.

From what I've seen, the value in QA is product familiarity. Good QA'ers know more about how the product actually works than anybody else. More than PM's, more than sales, and more than most dev teams. They have a holistic knowledge of the entire user-facing system and can tell you exactly what to expect when any button gets pushed. Bad QA'ers are indeed a source of noise. But so are bad devs, sysadmins, T1/2 support…

> Good QA'ers know more about how the product actually works than anybody else. More than PM's, more than sales, and more than most dev teams. Not disagreeing with this, but there's one thing they won't always be aware of. They won't always know what code a dev touched underneath the hood and what they might need to recheck (short of a full regression test every single time) to verify everything is still working. I k…

> Not disagreeing with this, but there's one thing they won't always be aware of. They won't always know what code a dev touched underneath the hood and what they might need to recheck (short of a full regression test every single time) to verify everything is still working.

It doesn't necessarily matter what code was changed, a change in code in Module A can cause a bug in Module B that hasn't been changed in a year. A QA test plan should cover the surface area of the product as used by consumers whoever they might be. While knowing some module had fixes can inform the test plan or focus areas when the test schedule is constrained, only testing changes is the road to tears.