Asking developers to do QA is broken – why anyone should own QA • Hacker News Board

Re: Asking developers to do QA is broken – why anyone should own QA

#11

Post by bob1029 » Wed, Jun 30, 2021, 10:10 PM UTC

The complexity of our application necessitates that our business owners do a lot of the final integration testing.

It also requires that our product owners handle a large part of the implementation & QA efforts.

For our application, a single piece of code might be reused 40 different ways across 100 different customers based on configuration-time items.

To ask a developer to figure out why something is broken would be a fool's errand for us. Only with the aid of trace files & configuration references are we able to perform RCA and clean up an issue from live usage. For us, 99% of the time it is a configuration thing (either on our end or some 3rd party system). If the code works for 99 clients and 39 other configurations, it might be your client+config that is wrong, not the code.

Re: Asking developers to do QA is broken – why anyone should own QA

#12

Post by ukd1 » Wed, Jun 30, 2021, 10:11 PM UTC

g051051 wrote: post #8

The last place I worked would pull devs randomly to do QA tasks when QA got behind. Of course, since we weren't trained in the QA processes, and didn't do it all the time, it often wound up taking longer to do the testing, plus the devs would get behind in their tasks.

It's also common practice; usually caused by testing getting under-resourced, bad tooling, or longer shipping cycles (more stuff to test, more pressure to get it out).

Re: Asking developers to do QA is broken – why anyone should own QA

#13

Post by franciscassel » Wed, Jun 30, 2021, 10:11 PM UTC

ppeetteerr wrote: post #10

Love the idea, but have you met product owners?

Yeah, the good ones really care about the product experience and customer outcomes, so this makes a lot of sense to me.

But what do you mean?

Re: Asking developers to do QA is broken – why anyone should own QA

#14

Post by void_mint » Wed, Jun 30, 2021, 10:12 PM UTC

I think QA-specific people that are specifically not SDETs create bad habits on teams. I think asking developers and product owners/managers to "own" QA fixes the "just throw it over the wall and hope for the best" problem.

Re: Asking developers to do QA is broken – why anyone should own QA

#15

Post by ukd1 » Wed, Jun 30, 2021, 10:12 PM UTC

marielvargas wrote: post #6

With more folks building using low code and no-code tools many different types of users will need to know how to do QA

This; especially now more apps are being done with no-code tooling (bubble, anyone?) - not a lot of existing tooling work with it, and code-based testing isn't viable even if it did.

Re: Asking developers to do QA is broken – why anyone should own QA

#16

Post by ukd1 » Wed, Jun 30, 2021, 10:14 PM UTC

bob1029 wrote: post #11

The complexity of our application necessitates that our business owners do a lot of the final integration testing. It also requires that our product owners handle a large part of the implementation & QA efforts. For our application, a single piece of code might be reused 40 different ways across 100 different customers based on configuration-time items. To ask a developer to figure out why something is broken would b…

What do you use to plan it or otherwise manage the process? Is it all manual?

Re: Asking developers to do QA is broken – why anyone should own QA

#17

Post by travisjungroth » Wed, Jun 30, 2021, 10:16 PM UTC

nullspace wrote: post #5

tpmx wrote: post #2

I think product owners should do (manual) QA. On their own, and when necessary for scalability reasons also via a small team under their direct control.

Yeah, this is the only thing that practically worked, in a complex product I used to work on. - If you ask the devs to do QA, you'll get no bugs other than the ones they already caught during testing and deployment. - If you have a mostly independent QA team, they will find somewhat silly/trivial bugs like the login page not working in an extreme edge case scenario. - However, when you ask your Product team to own QA…

On independent QA teams finding trivial bugs, I think this is a social problem. Specifically an alignment one. If a bug goes out, whose fault is it? Eng for making it or QA for not finding it? The answers are different at different orgs, and they have more to do with power dynamics than anything else.

QA is pretty easy to fake (for a while). The last thing you want to do is quality check your QA team. So the further the incentives of the QA team are from the success of the product, the worse things get. It's a spectrum from the other cofounder doing QA to someone in another country charging by the hour.

There are opposing forces to this. It's inherently difficult for people to check themselves. Also, QA is its own skill. It's one more thing to ask devs to be great at. Maybe it's worth having some specific experts around. It's kind of a specific mindset that not all devs have.

This mindset issue is where I'm not entirely sold on what looks like a new strategy from Rainforest QA (where I used to work) of strongly targeting product teams. I haven't seen any QA tool so good that it removes the complete need for skill like indoor heating means you don't need to know anything about how to tend a fire. The best ones are more like how a modern gas range helps a chef. So I question how great the results will be if you have a PM or CSM doing it. Still, I wish them the best of luck.

Re: Asking developers to do QA is broken – why anyone should own QA

#18

Post by mobilene » Wed, Jun 30, 2021, 10:20 PM UTC

I've yet to see the automated test suite that replaces a skilled, sapient, human, functional tester. The automation takes away the drudgery of repeating tests, but it takes a skilled human to figure out what risks are in the code and figure out coverage to determine whether those risks are realized. If you have developers write good unit and integration tests, and build their work to their local environment to make sure it basically works, you avoid most of the "throw it over the wall" mentality. You also deliver way fewer needles in the haystack, if you will. Testers are free to look for more impactful bugs manually, and then to automate coverage as appropriate.

Re: Asking developers to do QA is broken – why anyone should own QA

#19

Post by politelemon » Wed, Jun 30, 2021, 10:21 PM UTC

We've suffered from the same problems as highlighted here. What helped us was a low code solution, https://smashtest.io, which is basically an English wrapper over Selenium. The developers spend less time on tests, and the QAs aren't an afterthought.

Re: Asking developers to do QA is broken – why anyone should own QA

#20

Post by tpmx » Wed, Jun 30, 2021, 10:23 PM UTC

travisjungroth wrote: post #17

nullspace wrote: post #5

Earlier quoted context omitted.

Yeah, this is the only thing that practically worked, in a complex product I used to work on. - If you ask the devs to do QA, you'll get no bugs other than the ones they already caught during testing and deployment. - If you have a mostly independent QA team, they will find somewhat silly/trivial bugs like the login page not working in an extreme edge case scenario. - However, when you ask your Product team to own QA…

On independent QA teams finding trivial bugs, I think this is a social problem. Specifically an alignment one. If a bug goes out, whose fault is it? Eng for making it or QA for not finding it? The answers are different at different orgs, and they have more to do with power dynamics than anything else. QA is pretty easy to fake (for a while). The last thing you want to do is quality check your QA team. So the further…

We had a very large QA silo in a previous company. ~150 people all working under a former QA (non-developer) person turned into a "QA VP". It was a disaster in so many ways. Empire-building tendencies, large numbers of bad hires, etc.

The solution was to break up this silo and getting these people into the product teams where more technical people could handle management and hiring.