Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Asking developers to do QA is broken – why anyone should own QA

rainforestqa.com

Post reply on HN
Print view View original

21–30 of 193 posts

Re: Asking developers to do QA is broken – why anyone should own QA

#21

Post by franciscassel »

mobilene wrote: post #18

I've yet to see the automated test suite that replaces a skilled, sapient, human, functional tester. The automation takes away the drudgery of repeating tests, but it takes a skilled human to figure out what risks are in the code and figure out coverage to determine whether those risks are realized. If you have developers write good unit and integration tests, and build their work to their local environment to make s…

Totally agree that automation will never fully replace the value of human-powered testing. (Though it is great for the rote regression-testing stuff. The "drudgery", as you put it.)

Isn't the problem with relying too much on unit and integration tests that they don't consider the end-to-end app experience? (Which, in my mind, is what ultimately matters when customers think of "quality".)

Re: Asking developers to do QA is broken – why anyone should own QA

#22

Post by ukd1 »

mobilene wrote: post #18

I've yet to see the automated test suite that replaces a skilled, sapient, human, functional tester. The automation takes away the drudgery of repeating tests, but it takes a skilled human to figure out what risks are in the code and figure out coverage to determine whether those risks are realized. If you have developers write good unit and integration tests, and build their work to their local environment to make s…

I think you're pretty on point here; today human nuance is needed to decide WHAT to test as well as HOW MUCH to test. It's also useful for executing some types of tests too (which is why we support both automation, and human testing). IMHO Unit and integration tests should be a given. Human testers should be used with the highest leverage where possible. Being a functional-tester though, today is partly limited by what tooling is accessible to you - which we want to fix.

Re: Asking developers to do QA is broken – why anyone should own QA

#23

Post by bilalq »

In a model where developers adhere to a devops philosophy and are product owners, there's no need for a split. Developers should not be measured only quantity of releases, but also on metrics related to availability, customer impact, latency, operational costs, etc.

I'm not opposed to a model where non-technical roles are empowered to define test cases and contribute to approval workflows of a release pipeline, but that doesn't absolve developers of the primary responsibility of being end-to-end owners.

I know "devops" is an overloaded term that means different things to different people. To me, it's when developers are involved in product research, requirements gathering, user studies, design, architecture, implementation, testing, release management, and ongoing operations. They're supported by a dedicated product manager who leads the research/requirements gathering/user studies initiatives.

Re: Asking developers to do QA is broken – why anyone should own QA

#24

Post by ukd1 »

franciscassel wrote: post #21
mobilene wrote: post #18

I've yet to see the automated test suite that replaces a skilled, sapient, human, functional tester. The automation takes away the drudgery of repeating tests, but it takes a skilled human to figure out what risks are in the code and figure out coverage to determine whether those risks are realized. If you have developers write good unit and integration tests, and build their work to their local environment to make s…

Totally agree that automation will never fully replace the value of human-powered testing. (Though it is great for the rote regression-testing stuff. The "drudgery", as you put it.) Isn't the problem with relying too much on unit and integration tests that they don't consider the end-to-end app experience? (Which, in my mind, is what ultimately matters when customers think of "quality".)

IMHO, yep - it's a balance, but the great thing is they can be quick to run, and run locally-easily; which is great for developers to get fast-feedback. Unit testing is unlikely to catch all end-user level issues though; traditional automation too, which is why human-testing is still valuable today.

Re: Asking developers to do QA is broken – why anyone should own QA

#25

Post by mschuster91 »

nullspace wrote: post #5
tpmx wrote: post #2

I think product owners should do (manual) QA. On their own, and when necessary for scalability reasons also via a small team under their direct control.

Yeah, this is the only thing that practically worked, in a complex product I used to work on. - If you ask the devs to do QA, you'll get no bugs other than the ones they already caught during testing and deployment. - If you have a mostly independent QA team, they will find somewhat silly/trivial bugs like the login page not working in an extreme edge case scenario. - However, when you ask your Product team to own QA…

> However, when you ask your Product team to own QA, you get the real good stuff - "Why does this feature not actually work well with this other feature when you combine the configuration" etc. It's great!

That would require a skilled product management that is on equal footing with sales and has a veto right before stuff gets sold to customers. All too often however, the only thing that matters is customer wishes for new features, with no one taking care that the tugs the customers are pulling on don't rip the product apart.

Re: Asking developers to do QA is broken – why anyone should own QA

#26

Post by wefarrell »

This feels a lot like a reincarnation of cucumber/gherkin, except they've replaced business-facing text DSL with a no-code visual UI. The intention is the same - to have the customer own the tests.

This looks like it has a shallower learning curve to get started, but I would imagine that after a certain point it winds up being less productive to use the UI than to write code and this is ultimately why visual coding hasn't taken off. At some point someone will need to become a power user and be their organization's resident expert on Rainforest, but at that point they're spending all of their time in Rainforest and they're no longer a product owner embedded in the business.

At the end of the day the business owner should be writing specifications, the engineers should be using them to create tests and they each should be collaborating closely on both.

Re: Asking developers to do QA is broken – why anyone should own QA

#27

Post by ukd1 »

travisjungroth wrote: post #17
nullspace wrote: post #5

Earlier quoted context omitted.

Yeah, this is the only thing that practically worked, in a complex product I used to work on. - If you ask the devs to do QA, you'll get no bugs other than the ones they already caught during testing and deployment. - If you have a mostly independent QA team, they will find somewhat silly/trivial bugs like the login page not working in an extreme edge case scenario. - However, when you ask your Product team to own QA…

On independent QA teams finding trivial bugs, I think this is a social problem. Specifically an alignment one. If a bug goes out, whose fault is it? Eng for making it or QA for not finding it? The answers are different at different orgs, and they have more to do with power dynamics than anything else. QA is pretty easy to fake (for a while). The last thing you want to do is quality check your QA team. So the further…

Thanks for the luck Travis! Def something we've been edging around for a while. From what we've been seeing, a lot of PMs and no/low code folks already do this kind of thing manually, and don't have tooling for it. RF automation is now way better / easier to use than when you were with us (imho, of course).

Re: Asking developers to do QA is broken – why anyone should own QA

#28

Post by kazinator »

The right person for doing QA for code written by developer A is developer B, who is motivated to show that there is breakage in the code written by A. This is just good old peer review, in the sphere of development.

The industry uses dedicated QA people based on the assumption that you can get two for the price of a developer.

In fact, if you have two developers, one of whom is more clever than the other, you want to give the code cranking to the less clever one, and use the clever one to verify the code cranking and make improvement suggestions.

If you have clever people designing the system, and clever people finding problems in the commits, you can just have average coders cranking out the bulk of it. Most code doesn't need cleverness, just persistence and consistency.

Re: Asking developers to do QA is broken – why anyone should own QA

#29

Post by regularfry »

Devs doing QA worked fine for us. You need to have a team that actually cares about the product, and you need figurehead devs - not necessarily seniors or team leads, but charismatic people others will fall into line with - who model the behaviours you want.

The problem with almost anything else is that it increases the number of hand-offs between groups of differently-aligned people on the route to production. If you're aiming at multiple deployments per day, with sub-hour commit-to-prod times, the latency in those hand-offs adds up way too fast.

Re: Asking developers to do QA is broken – why anyone should own QA

#30

Post by thrower123 »

wefarrell wrote: post #26

This feels a lot like a reincarnation of cucumber/gherkin, except they've replaced business-facing text DSL with a no-code visual UI. The intention is the same - to have the customer own the tests. This looks like it has a shallower learning curve to get started, but I would imagine that after a certain point it winds up being less productive to use the UI than to write code and this is ultimately why visual coding h…

I've had a couple goes at teams trying to roll out cucumber tests, and I still don't understand quite what the point is.

Nobody but developers could actually manage to write any tests, and it was harder than just using the normal tools, plus maintaining all the glue besides.

Post reply on HN