Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Maybe getting rid of your QA team was bad

davidkcaudill.medium.com

Post reply on HN
Print view View original

31–40 of 269 posts

Re: Maybe getting rid of your QA team was bad

#31

Post by cratermoon »

convolvatron wrote: post #13
wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

in the classic model, most QA orgs were a useless appendage. partially by construction, but largely because QA gets squeezed out when dev is late (when does that happen?). they aren't folded in early, so they twiddle their thumbs doing 'test infrastructure' and 'test plans', until they finally get a code drop and a 48 hr schedule to sign off, which they are under extreme pressure to do. but every once and a while you…

Saying "useless appendage" sounds to me like it's the QA team that's the problem, when what you're really saying is that it's the organization and process that pushed QA teams into irrelevance. I agree with your assessment overall, and those issues were one of the driving forces behind companies dispensing with QA and putting it all on the developers.

Re: Maybe getting rid of your QA team was bad

#32

Post by godelski »

The main problem with QA teams is the same problem with IT teams or even management. If they are doing their jobs well they appear to be doing nothing.

This often creates a situation where people need to "justify" their jobs. Usually this happens due to an over reliance upon metrics (see Goodhart's Law) rather than understanding what the metrics are proxying and what the actual purpose of the job is. A bad QA team is one who is overly nitpicky, looking to ensure they have something to say. A good QA team simultaneous checks for quality as well as trains employees to produce higher quality.

I do feel like there is a lack of training going on in the workforce. We had the "95%-ile isn't that good"[0] post on the front page not long ago and literally it is saying "It's easy to get to the top 5% of performers in any field because most performers don't actively train or have active feedback." It's like the difference between being on an amateur sports team vs a professional. Shouldn't businesses be operating like the latter? Constantly training? Should make hiring be viewed differently too, as in "can we turn this person into a top performer" rather than "are they already" because the latter isn't as meaningful as it appears when your environment is vastly different than the one where success was demonstrated.

[0] https://news.ycombinator.com/item?id=38560345

Re: Maybe getting rid of your QA team was bad

#33

Post by shados »

wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

> but I've never encountered a QA team that actually writes the tests for engineering.

I have a few times. But the only common thing in the QA industry, is that every company does it differently and think they're doing it the "normal way".

Re: Maybe getting rid of your QA team was bad

#34

Post by hasoleju »

yabones wrote: post #23
wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

From what I've seen, the value in QA is product familiarity. Good QA'ers know more about how the product actually works than anybody else. More than PM's, more than sales, and more than most dev teams. They have a holistic knowledge of the entire user-facing system and can tell you exactly what to expect when any button gets pushed. Bad QA'ers are indeed a source of noise. But so are bad devs, sysadmins, T1/2 support…

I completely agree with that. It really comes down to having the right skills as a QA person. If you don't know how the product is used and only click on some buttons, you will never reach the states in the software that real users reach and therefor you will also not be able to reproduce them.

Re: Maybe getting rid of your QA team was bad

#35

Post by closeparen »

wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

In my company the engineering team mostly writes unit tests. Then there was a weekly manual QA exercise where the oncall engineer followed a checklist with our actual mobile app on an actual phone before it went to the store. When this started to take almost the entire day, we hired a contract workforce for it. The contract workforce is in the process of automating those tests, but the most important ones still get human eyes on.

Re: Maybe getting rid of your QA team was bad

#36

Post by chopin »

BiteCode_dev wrote: post #25
wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

Frameworks like playright can record as code user actions and you can replay them in a test. So you can make your QA teams create plenty of tests if you give them the right tools.

In my experience such tests are brittle as hell.

Re: Maybe getting rid of your QA team was bad

#37

Post by danesparza »

EvanAnderson wrote: post #12
robotnikman wrote: post #4

Microsoft is probably the one case where this sticks out the most, at least for me anyways. Noticeably more bugs in updates since they dropped their QA team, in Windows as well as cloud products.

The revenue keeps rolling in so, clearly, they made the right business decision... >sigh I use a lot of MSFT software and services in the "day job". I wish there was some kind of consequence to them for their declining quality.

Almost like you should be able to file a 'bug report' or something. Maybe they should build a team to make sure their code quality is up to snuff...

Re: Maybe getting rid of your QA team was bad

#38

Post by philk10 »

godelski wrote: post #32

The main problem with QA teams is the same problem with IT teams or even management. If they are doing their jobs well they appear to be doing nothing. This often creates a situation where people need to "justify" their jobs. Usually this happens due to an over reliance upon metrics (see Goodhart's Law) rather than understanding what the metrics are proxying and what the actual purpose of the job is. A bad QA team is…

When I work with new devs I can often trip them up with basic tests of double-clicking, leading spaces, using the back button on Android. They then learn these and from then on these issues dont appear ( well, OK, it might take a couple of times of a ticket being rejected because of these but they do quickly learn all my tricks ) I don't get measured on bugs found so there's no pressure on me to find stupid bugs just to boost my figures.

Re: Maybe getting rid of your QA team was bad

#39

Post by itqwertz »

I noticed this trend a couple of years ago, they called it Shift-Left.

Basically, it was a way to get a developer to do more testing during development before handing over a feature to QA. This sets up the imagined possibility of firing all QA staff and having developers write perfect code that never needs to be tested thoroughly. Looks great on paper...

At a previous company, they started firing all of the manual QA devs and replacing them with offshore QA people who could do automated testing development (Cypress tests). The only problem was that those fired QA team members had significant business domain knowledge that never was transferred or written down. The result was a dumpster fire every week, with the usual offshore nightmares and panicked responses.

Make no mistake about this, it's just a cost-cutting measure with an impact that is rarely felt immediately by management. I've worked with stellar manual QA people and our effort resulted in bulletproof software that handled edge cases, legacy customers, API versions, etc. Without these people, your software eventually becomes IT-IS-WHAT-IT-IS and your position is always threatened due to the lack of quality controls.

Post reply on HN