Asking developers to do QA is broken – why anyone should own QA • Hacker News Board

Re: Asking developers to do QA is broken – why anyone should own QA

#141

Post by Shacklz » Thu, Jul 01, 2021, 9:09 AM UTC

cwkoss wrote: post #112

notmyfuture wrote: post #82

Earlier quoted context omitted.

I've also arrived at this approach and don't think it's that uncommon - IMO the article is presenting a false dichotomy. There are still gotchas to look out for in the team-embedded QA approach. In typical team sizes, you often end up with only one QA per team - you need to make sure they have cover (everyone needs a break), and they need support in their discipline (do something to share QA knowledge across teams).

Absolutely. Disappointing to see this sort of shallow sales pitch blogspam making it to the frontpage. I'm surprised more HN readers don't see through this. The entire purpose of this article is to self-servingly attempt to convince the reader that their product is the only solution to QA problems. It correctly identifies some challenges with QA, but this solution is certainly not the only way to have effective QA. T…

Oh c'mon. Of course a blog on the website of a company is ultimately trying to pitch their product, one way or another. That does not invalidate the points they bring up per se.

It obviously needs a bit of critical thinking when reading it and taking everything with a grain of salt, but that's something I really hope we can expect people on HN to be capable of?

Re: Asking developers to do QA is broken – why anyone should own QA

#142

Post by intellix » Thu, Jul 01, 2021, 9:11 AM UTC

Am I reading the pricing correctly? 135$ for running 30 specs?

We have Cypress tests that run automatically for every single git push which are running throughout the day constantly... if you did a price comparison we'd be saving something like $140k per month by not having a pretty UI for building tests

Re: Asking developers to do QA is broken – why anyone should own QA

#143

Post by andix » Thu, Jul 01, 2021, 9:12 AM UTC

Cthulhu_ wrote: post #132

andix wrote: post #127

Earlier quoted context omitted.

But that's just bad management. It's most of the time better to drop/delay some features, than compromising stability. If the manager doesn't get that, he/she is the problem. And not the software developer.

OTOH, a manager doesn't see outage - they give you an assignment and expect you, the developer, do to a good job. Outages are not because you missed a deadline, but because you didn't do the job that is specified on your job description. I'm sure you can make analogies. If you get a new kitchen installed, you expect it to be done properly, if it's finished within the day or so they quoted for you but the doors fall o…

That's the bad management I'm referring to. I've seen a lot of bad management, but never actually that bad.

If software fails it's a team failure. QA is equally responsible for that, not just developers. Quite often there root of the failure is ambiguous specification (so whoever did that is responsible too).

And in the end the manager is also responsible, because he didn't do his job right (picking suitable people and coaching them, so they can do the job).

Re: Asking developers to do QA is broken – why anyone should own QA

#144

Post by andix » Thu, Jul 01, 2021, 9:18 AM UTC

indogooner wrote: post #135

andix wrote: post #126

Wherever I worked so far as a software developer we had the QA department very close. In the same room or next door. They were there from the beginning until the end, tested our software continuously. If we developed something, that was hard for them to test or to understand, they gave us a hard time. So we tried to make software testable. Do meaningful logging. Make things repeatable. Develop tools to export state/p…

This has been my experience as well. The places where testing was an afterthought meant the software was not testable easily. Executing multiple steps to test one feature meant QA folks kept running into one or other service issues causing delay in bug discovery. Rather than giving time to devs for fixing this, the management put more devs on test writing and were more interested in looking at dashboards.

I've seen this behaviour in outsourced software development. In the end they had 10 times the people they would actually need on projects. And they were mostly trapped inside deadlocks and doing nothing.

Just like if you try to parallelize a single threaded application without redesigning it. Just multiplying the core count of the CPU by 10 may bring you a few percent increase in speed, but nowhere close to the expected +1000%.

Re: Asking developers to do QA is broken – why anyone should own QA

#145

Post by regularfry » Thu, Jul 01, 2021, 9:24 AM UTC

protomyth wrote: post #98

regularfry wrote: post #74

Earlier quoted context omitted.

This can absolutely be learned, enabled, and encouraged by the right guidance and, if necessary, training at the dev-team level. It's just part of the job.

I really don't think any Dev team will be as good at QA as a dedicated QA team. The mindset is very different and the context switch is hard. Checking your own work is also not the greatest of ideas.

You are free to think that. I've seen it done.

Re: Asking developers to do QA is broken – why anyone should own QA

#146

Post by fnord77 » Thu, Jul 01, 2021, 9:33 AM UTC

asking developers to do OPs is broken, too.

Re: Asking developers to do QA is broken – why anyone should own QA

#147

Post by leghifla » Thu, Jul 01, 2021, 9:34 AM UTC

ende wrote: post #102

alexpotato wrote: post #83

Many years ago I asked a developer which was a bigger deal: - Having an outage - Missing a deadline They answered: "oh having an outage is WAY worse". I then asked: "if that's the case, why do you push so hard to hit your deadlines with code you know and I both know is probably not ready?" They didn't really answer at the time but it eventually dawned on me what's happening: - odds of being yelled at if you miss a de…

This is the classic case of development velocity pitted against operational stability. Entirely different incentives, and when vested into the same role that role is bound to prioritize one incentive over the other. For this reason I think they must be separated at least by body if not by team. They certainly need to separated by different managers. I tend to think QA is perhaps well situated alongside Ops("DevOps"),…

About prioritizing, I want to share a story:

We were a small and good team developping hardware/software combos. We ruined a demo to a client by promising some hard to do feature, and during the demo, the said feature had not been well tested for a particular environment.

The debriefing of that failure was memorable. The big boss was yelling at us, saying we should do better, work harder, longer, whatever was required to succeed.

When this calmed down, I only asked one question: when going back to my desk, should I work on this new feature promised to some other customer, or test this old one for any combination of inputs/environments? The response was: "You do both" I insisted that I will do both, but which one first? He responded with some blabla I do not remember, but no response to my question.

To any manager which cannot decide between feature and stability: If you cannot prioritize, the dev will do it, with whatever information/incentive they have. You may not be happy of the result.

Re: Asking developers to do QA is broken – why anyone should own QA

#148

Post by ozim » Thu, Jul 01, 2021, 9:45 AM UTC

MontagFTB wrote: post #48

Asking developers to own QA is broken because developers are naturally biased towards the happy path. If you want to build a bar, you need someone to order -1 beers[1]. Handing off QA to an external team is broken because those people don't have the necessary experience with the product, nor can they quickly and easily engage with development to get to the heart of a problem (and a fix.) Having QA rely exclusively on…

Problem comes from BigCo approach to people. Such corporations would like to have a QA team that should be utilized to a 100%.

So that if they don't have much work on project X you just drop them into project Y.

In reality we all know that is just not possible, because people have to have domain knowledge and keep up with changes. It is not like if you come back to project after 2 months you will be proficient with it as the world changed.

This way if project X has slow times, dedicated QA will have less to do and BigCo has to eat that "loss".

In small startup it is easier to have QA incorporated into dev/product team because as software is constantly changed there is work to be done all the time.

Re: Asking developers to do QA is broken – why anyone should own QA

#149

Post by ozim » Thu, Jul 01, 2021, 9:54 AM UTC

jokethrowaway wrote: post #57

mobilene wrote: post #18

I've yet to see the automated test suite that replaces a skilled, sapient, human, functional tester. The automation takes away the drudgery of repeating tests, but it takes a skilled human to figure out what risks are in the code and figure out coverage to determine whether those risks are realized. If you have developers write good unit and integration tests, and build their work to their local environment to make s…

They're not claiming people are not needed. They say that instead of being developers writing codee, they should be non technical product managers using no-code.

Company from the article is not claiming that but all the business people are talking about it.

They would like that "test-automation" would mean instead of 2 QA engineers now you need 1 doing 2x more.

Sad reality is that with "test-automation" you actually still need those 2 QA engineers and maybe part time dev to help them out. The upside would be that those QA people would spend less time repeating clicks and improve quality of checks.

Re: Asking developers to do QA is broken – why anyone should own QA

#150

Post by ChrisMarshallNY » Thu, Jul 01, 2021, 9:59 AM UTC

In my experience (and I have a great deal of that), true product Quality is dependent upon an endemic cultural philosophy of an organization; not a single tool or technique. It's a shared discipline, and a cultural imperative.

It's like those ads for exercise machines, where professional athletic models, who train for five hours a day, and drink broccoli smoothies for lunch, are shown using a machine for a couple of minutes, with the inference that the machine is the reason for their washboard abs.

The Quality formula is hundreds; if not thousands, of years old. Software development is just another context.

There are significant costs to doing real Quality, which is the main reason that more people don't do it.

One of the biggest costs, is that there isn't really that much money to be made, doing superb Quality. You get to sell to snooty customers, and feel smug, but there just ain't that many people that are willing to pay the premium for top Quality.

Dreck sells. People who get truly rich, seldom do so, selling Quality. The ones that do, have that endemic culture. It's really difficult (and expensive) to maintain that; especially at scale. Lots of humbling and hiring expensive, cranky old craftsmen that can be difficult to work with.

Lot of broccoli smoothies.