Maybe getting rid of your QA team was bad • Hacker News Board

Re: Maybe getting rid of your QA team was bad

#121

Post by steveBK123 » Thu, Dec 14, 2023, 9:43 PM UTC

pavel_lishin wrote: post #116

pjsg wrote: post #103

At the start of my career (late 70s), I worked at IBM (Hursley Park) in Product Assurance (their version of QA). We wrote code and built hardware to test the product that was our area of responsibility (it was a word processing system). We wrote test cases that our code would drive against the system under test. Any issues we would describe in general terms to the development team -- we didn't want them to figure out…

> When I've worked for organizations without QA teams, I introduce the concept of "sniff tests". This is a short (typically 1 hour) test session where anybody in the company / department is encouraged to come and bash on the new feature. We call those bug-bashes where we work, and they're also typically very productive in terms of defects discovered! It's especially useful since during development of small features,…

A good QA person is basically a personification of all the edge cases of your actual production users. Our good QA person knew how human users used our app better than the dev or product team. It was generally a competition between QA & L2 support as to who actually understood the app best.

The problem with devs testing their own & other devs code is that we test what we expect to work in the way we expect the user to use it. This completely misses all sorts of implementation error and edge cases.

Of course the dev tests the happy path they coded.. that's what they thought users would do, and what they thought users wanted! Doesn't mean devs were right, and frequently they are not..

Re: Maybe getting rid of your QA team was bad

#122

Post by tstrimple » Thu, Dec 14, 2023, 9:43 PM UTC

JohnMakin wrote: post #96

godelski wrote: post #32

The main problem with QA teams is the same problem with IT teams or even management. If they are doing their jobs well they appear to be doing nothing. This often creates a situation where people need to "justify" their jobs. Usually this happens due to an over reliance upon metrics (see Goodhart's Law) rather than understanding what the metrics are proxying and what the actual purpose of the job is. A bad QA team is…

> This often creates a situation where people need to "justify" their jobs. On DevOps teams I see this constantly. Usually the best-compensated or most senior "Ops" guy or whatever they're called at the company spends a lot of his time extinguishing fires that were either entirely of his own creation/incompetence, which makes it look like he's "doing something." You automate away the majority of the toil there and th…

Which is one reason DevOps teams don't make sense. DevOps is a skill developers need to have. It needs to be embedded within the development team, not some other team's responsibility who only focuses on "DevOps" work. You create the build and deployment pipelines and move on to other project work. If you give someone a role and say their job is to do "DevOps" they will HAVE to invent things to do because that's such a small part of a project and once implemented doesn't need a ton of maintenance.

Re: Maybe getting rid of your QA team was bad

#123

Post by wrs » Thu, Dec 14, 2023, 9:43 PM UTC

pjsg wrote: post #103

At the start of my career (late 70s), I worked at IBM (Hursley Park) in Product Assurance (their version of QA). We wrote code and built hardware to test the product that was our area of responsibility (it was a word processing system). We wrote test cases that our code would drive against the system under test. Any issues we would describe in general terms to the development team -- we didn't want them to figure out…

At Microsoft back in the day, we called those “bug bashes”, and my startup inherited the idea. We encouraged the whole company to take an afternoon off to participate, and gave out awards for highest impact bug, most interesting bug, etc.

Re: Maybe getting rid of your QA team was bad

#124

Post by perlgeek » Thu, Dec 14, 2023, 9:43 PM UTC

amlozano wrote: post #21

This point is brought up in the article but I think it is at the real heart of the issue. QA is almost always seen as a 'cost center' by the business and upper management. I have a hypothesis that you never ought to work in a department that is seen as a 'cost center'. The bonuses, the recognition, and the respect always goes to the money makers. The cost center is the first place to get more work with less hands, ge…

> I have a hypothesis that you never ought to work in a department that is seen as a 'cost center'

That's why I don't work in an IT department of a traditional business.

Re: Maybe getting rid of your QA team was bad

#125

Post by godelski » Thu, Dec 14, 2023, 9:44 PM UTC

philk10 wrote: post #38

godelski wrote: post #32

The main problem with QA teams is the same problem with IT teams or even management. If they are doing their jobs well they appear to be doing nothing. This often creates a situation where people need to "justify" their jobs. Usually this happens due to an over reliance upon metrics (see Goodhart's Law) rather than understanding what the metrics are proxying and what the actual purpose of the job is. A bad QA team is…

When I work with new devs I can often trip them up with basic tests of double-clicking, leading spaces, using the back button on Android. They then learn these and from then on these issues dont appear ( well, OK, it might take a couple of times of a ticket being rejected because of these but they do quickly learn all my tricks ) I don't get measured on bugs found so there's no pressure on me to find stupid bugs just…

> I don't get measured on bugs found so there's no pressure on me to find stupid bugs just to boost my figures.

Sounds like the right incentive structure. If you don't mind, how are you judged? Do you feel like the system you're in is creating the appropriate incentives and actually being effective? Certainly this example is but I'd like to know more details from an expert so I can update my understanding.

Re: Maybe getting rid of your QA team was bad

#126

Post by bumby » Thu, Dec 14, 2023, 9:50 PM UTC

IKantRead wrote: post #69

wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

A good QA person is to a software developer as a good editor is to a writer. Both take a look at your hard work and critique it ruthlessly. Annoying as hell when it's happening, but in my experience well worth it because the end result is much higher quality. I might just be too old, but I remember when QA people didn't typically write tests, they manually tested your code and did all those weird things you were real…

>Now we hoist QA on the user.

This drives me crazy. It's a cheap way of saying we're ok shipping crap. In the past, I've been part of some QA audits where the developers claimed their customer support log sufficed as their test plan. This wasn't safety-critical software, but it did involve what I would consider medium risk (e.g., regulatory compliance). The fact that they openly admit they are okay shipping bad products in that environment just doesn't make sense to me.

Re: Maybe getting rid of your QA team was bad

#127

Post by onlyrealcuzzo » Thu, Dec 14, 2023, 9:55 PM UTC

PH95VuimJjqBqy wrote: post #109

Zelphyr wrote: post #56

I worked at two companies 15-20 years ago that invested in top-tier QA teams. They were worth their weight in gold. The products were world class because the QA team were fantastic at finding bugs we developers didn't think of looking for because we were too close to the problem. We are too used to looking at the happy path. One key attribute to both companies is that it was dictated from on high that the QA team had…

The QA culture has to be there, not just dictates that the QA has final say. I've seen companies where that's true and it was still trash because the QA were mostly low-paid contract workers who only did exactly what they were told and no more.

Worked at a company where QA had the final say - and that was by far the most toxic / worst environment I have ever been in.

QA also REFUSED to let developers write automation tests, also REFUSED to let us run them ourselves.

What a nightmare.

YMMV, but just having the final say is not a silver bullet for sure.

Re: Maybe getting rid of your QA team was bad

#128

Post by hornban » Thu, Dec 14, 2023, 9:55 PM UTC

wrs wrote: post #123

pjsg wrote: post #103

At the start of my career (late 70s), I worked at IBM (Hursley Park) in Product Assurance (their version of QA). We wrote code and built hardware to test the product that was our area of responsibility (it was a word processing system). We wrote test cases that our code would drive against the system under test. Any issues we would describe in general terms to the development team -- we didn't want them to figure out…

At Microsoft back in the day, we called those “bug bashes”, and my startup inherited the idea. We encouraged the whole company to take an afternoon off to participate, and gave out awards for highest impact bug, most interesting bug, etc.

This is a bit of an aside, but I have a question that I'd like to ask the wider community here. How can you do a proper bug-bash when also dealing with Scrum metrics that result in a race for new features without any regard for quality? I've tried to do this with my teams several times, but ultimately we're always coming down to the end of the sprint with too much to do to implement features, and so anybody that "takes time off" to do bug bashing looks bad because ultimately they complete fewer story points than others that don't do it?

Is the secret that it only works if the entire company does it, like you suggest?

And yes, I completely realize that Scrum is terrible. I'm just trying to work within a system.

Re: Maybe getting rid of your QA team was bad

#129

Post by slv77 » Thu, Dec 14, 2023, 9:58 PM UTC

JohnMakin wrote: post #96

godelski wrote: post #32

The main problem with QA teams is the same problem with IT teams or even management. If they are doing their jobs well they appear to be doing nothing. This often creates a situation where people need to "justify" their jobs. Usually this happens due to an over reliance upon metrics (see Goodhart's Law) rather than understanding what the metrics are proxying and what the actual purpose of the job is. A bad QA team is…

> This often creates a situation where people need to "justify" their jobs. On DevOps teams I see this constantly. Usually the best-compensated or most senior "Ops" guy or whatever they're called at the company spends a lot of his time extinguishing fires that were either entirely of his own creation/incompetence, which makes it look like he's "doing something." You automate away the majority of the toil there and th…

Sometimes people want to be firefighters to protect the people they serve but others love how being in a crisis makes them feel alive. The later are why arson investigators first look at firefighters when doing an arson investigation. Some of them need the fires and will start them just to fight them.

There are a lot of these types that gravitate to crisis management roles like DevOps.

Re: Maybe getting rid of your QA team was bad

#130

Post by mikestew » Thu, Dec 14, 2023, 10:01 PM UTC

onlyrealcuzzo wrote: post #127

PH95VuimJjqBqy wrote: post #109

Earlier quoted context omitted.

The QA culture has to be there, not just dictates that the QA has final say. I've seen companies where that's true and it was still trash because the QA were mostly low-paid contract workers who only did exactly what they were told and no more.

Worked at a company where QA had the final say - and that was by far the most toxic / worst environment I have ever been in. QA also REFUSED to let developers write automation tests, also REFUSED to let us run them ourselves. What a nightmare. YMMV, but just having the final say is not a silver bullet for sure.

That’s not because “QA had the final say”, it’s because your QA team were ass clowns. Any QA team that discourages dev from writing or running tests needs to be burnt to ground and rebuilt.