Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Maybe getting rid of your QA team was bad

davidkcaudill.medium.com

Post reply on HN
Print view View original

161–170 of 269 posts

Re: Maybe getting rid of your QA team was bad

#161

Post by feoren »

ponector wrote: post #132
Zelphyr wrote: post #56

I worked at two companies 15-20 years ago that invested in top-tier QA teams. They were worth their weight in gold. The products were world class because the QA team were fantastic at finding bugs we developers didn't think of looking for because we were too close to the problem. We are too used to looking at the happy path. One key attribute to both companies is that it was dictated from on high that the QA team had…

I've seen developers writing tests with no assertions, or with assertTrue(true). Always green, with 100% coverage! The same people have been asking: why should I write tests if I can write new features? And then one senior QA comes and destroys everything. Once I found that if I press f5 50 times during a minute then backend will go in outOfMemory while spinning requests to the database.

> I've seen developers writing tests with no assertions

This can be OK if the code executing without throwing exceptions is itself testing something. If you have a lot of assertions written directly into the code, as pre- or post-conditions for instance. But I'm guessing that wasn't the case here.

Re: Maybe getting rid of your QA team was bad

#162

Post by ponector »

ahtihn wrote: post #157
ponector wrote: post #138

Earlier quoted context omitted.

Unless dev is switching to write test 100% of time and become a tester it is highly not recommended to let developers write tests. That QA was not a clown, they've seen some shit... Would you let QA to write features in your production code?

Test code is just code. If you can write test code you can write production code. If you can write production code you can write tests. If your concern is that devs don't have the right mindset for testing, you can have them collaborate with a QA specialist to define the test cases and review the test implementation.

In theory yes.

In practice, devs will not write good tests for their features, QA will be kept away from committing to production code.

Btw, if it is just code - why developers cannot implement features without bugs? No need in QA in such ideal world.

Re: Maybe getting rid of your QA team was bad

#163

Post by ansible »

jxramos wrote: post #106
pjsg wrote: post #103

At the start of my career (late 70s), I worked at IBM (Hursley Park) in Product Assurance (their version of QA). We wrote code and built hardware to test the product that was our area of responsibility (it was a word processing system). We wrote test cases that our code would drive against the system under test. Any issues we would describe in general terms to the development team -- we didn't want them to figure out…

that's very interesting to hide the source of the automated tests from the developers as a strategy. I can see that shifting the focus to not just disabling the test or catering to the test etc. I'll have to think about this one, there's some rich thoughts to meditate on with this one.

It is an interesting approach I hadn't heard of before. For complex systems though, often reproducing the bug reliably is a large part of the problem. So giving the developers the maximum information is necessary.

Any time a "fix" is implemented, someone needs to be asking the right questions. Can this type of problem occur in other features / programs? What truly is the root cause, and how has that been addressed?

Re: Maybe getting rid of your QA team was bad

#164

Post by JonChesterfield »

pjsg wrote: post #103

At the start of my career (late 70s), I worked at IBM (Hursley Park) in Product Assurance (their version of QA). We wrote code and built hardware to test the product that was our area of responsibility (it was a word processing system). We wrote test cases that our code would drive against the system under test. Any issues we would describe in general terms to the development team -- we didn't want them to figure out…

> This meant that you could use fancy statistics to estimate the actual number of bugs left.

That's very clever. Precise test case in QA plus vague description given to dev. Haven't seen it before, thank you for sharing that insight.

Re: Maybe getting rid of your QA team was bad

#165

Post by Shaanie »

hornban wrote: post #128
wrs wrote: post #123

Earlier quoted context omitted.

At Microsoft back in the day, we called those “bug bashes”, and my startup inherited the idea. We encouraged the whole company to take an afternoon off to participate, and gave out awards for highest impact bug, most interesting bug, etc.

This is a bit of an aside, but I have a question that I'd like to ask the wider community here. How can you do a proper bug-bash when also dealing with Scrum metrics that result in a race for new features without any regard for quality? I've tried to do this with my teams several times, but ultimately we're always coming down to the end of the sprint with too much to do to implement features, and so anybody that "tak…

That's not a problem with Scrum, it's a problem with your team. If you're doing a bug bash every sprint, then your velocity is already including the time spent on bug bashes. If it's not in every sprint, you can reduce the forecast for sprints where you do them to account for it (similar to what you do when someone is off etc).

If you're competing within the team to complete as many story points as possible that's pretty weird. Is someone using story points as a metric of anything other than forecasting?

Re: Maybe getting rid of your QA team was bad

#166

Post by JCharante »

hornban wrote: post #128
wrs wrote: post #123

Earlier quoted context omitted.

At Microsoft back in the day, we called those “bug bashes”, and my startup inherited the idea. We encouraged the whole company to take an afternoon off to participate, and gave out awards for highest impact bug, most interesting bug, etc.

This is a bit of an aside, but I have a question that I'd like to ask the wider community here. How can you do a proper bug-bash when also dealing with Scrum metrics that result in a race for new features without any regard for quality? I've tried to do this with my teams several times, but ultimately we're always coming down to the end of the sprint with too much to do to implement features, and so anybody that "tak…

> because ultimately they complete fewer story points than others that don't do it?

Solution: don't measure story points

Re: Maybe getting rid of your QA team was bad

#167

Post by JonChesterfield »

nlavezzo wrote: post #142

When we built FoundationDB, we had a maniacal focus on quality and testing. So much so that we built it in a language we invented, called Flow, that allowed us to deterministically simulate arbitrary sized FDB clusters and subject them to crazy conditions, then flag each system property violation and be able to perfectly reproduce the test run that triggered the violation. We got to a point where the default was that…

Is this the language in question? https://github.com/apple/foundationdb/blob/main/flow/README....

Re: Maybe getting rid of your QA team was bad

#168

Post by ponector »

kcb wrote: post #151
ponector wrote: post #132

Earlier quoted context omitted.

I've seen developers writing tests with no assertions, or with assertTrue(true). Always green, with 100% coverage! The same people have been asking: why should I write tests if I can write new features? And then one senior QA comes and destroys everything. Once I found that if I press f5 50 times during a minute then backend will go in outOfMemory while spinning requests to the database.

That's why the dev teams tests and the QA teams tests are not mutually exclusive.

Testing is continuous multilayered multistaged process.

Testing should start before first developer wrote first line of code for the project. Architecture blueprints, set of requirements should be tested as early as possible. But that is not happening in real life, only in books. In real life pm will bring cheap contractor from India one month before the target release date.

Re: Maybe getting rid of your QA team was bad

#169

Post by jabroni_salad »

amtamt wrote: post #54

> The most conscientious employees in your organization are the most bitter. They see the quality issues, they often address them, and they get no recognition for doing so. When they speak up about quality concerns, they get treated like mouthbreathers who want to slow down. They watch the “move fast and break things” crowd get rewarded time after time, while they run around angrily cleaning up their messes. To these…

My idea of the pendulum swinging back is 'you build it you run it', personally. Don't like the oncall pager? don't make it ring.

Re: Maybe getting rid of your QA team was bad

#170

Post by munificent »

supportengineer wrote: post #67
Zelphyr wrote: post #56

I worked at two companies 15-20 years ago that invested in top-tier QA teams. They were worth their weight in gold. The products were world class because the QA team were fantastic at finding bugs we developers didn't think of looking for because we were too close to the problem. We are too used to looking at the happy path. One key attribute to both companies is that it was dictated from on high that the QA team had…

I've seen QA/QE greatness and it was similar to how you describe. A different chain of command for deciding if releases are certified for production. Different incentive structures as well. Not to mention, at one recent employer, the QE team wrote an enormous amount of code to perform their tests - It was more LOC than the modules being tested/certified.

> It was more LOC than the modules being tested/certified.

So much code. I hope they had a QAQA team to test all that.

Post reply on HN