Asking developers to do QA is broken – why anyone should own QA • Hacker News Board

Re: Asking developers to do QA is broken – why anyone should own QA

#191

Post by lowbloodsugar » Wed, Jul 07, 2021, 6:07 PM UTC

postalrat wrote: post #190

lowbloodsugar wrote: post #176

Earlier quoted context omitted.

I'd say google W. Edwards Deming to start.

Can you help me find where he examines "when people test what they are responsible for"?

I can't find the bit where he talks explicitly about this as I've lost access to the book I read it in (and don't even remember the name). I'll see if I can dig it out.

In the passage I am unable to find, he gets into the issue of having a worker making widgets, and another worker checking the widgets. In our world this would be a developer and a test author. He states that this reduces quality: the maker knows that he can make poor parts when he is in a hurry, because the tester will catch them; the tester knows the worker personally, and trusts that he will make good parts, so doesn't have to be very thorough. Instead, we need the worker making the widget to be responsible for its quality, and further to be empowered to do so.

Re: Asking developers to do QA is broken – why anyone should own QA

#192

Post by skyzyx » Sat, Jul 10, 2021, 5:03 AM UTC

I agree with the premises, but not necessarily the conclusion. Any team with an attitude of “that’s not my job” is fundamentally broken.

The team is responsible for product, development, quality, ops, CI, etc. But you have to adjust your definition of “team”.

I very much agree with Google’s SRE model, and implementing it has worked well for us. The dev team is not an island unto itself, but rather a piece of a larger whole. SREs, Quality, Ops, and others are also true software engineers, and we tear down the silos between teams by working across our imaginary lines. Because we all rise and fall together.

QA isn’t responsible for writing all of the tests, but they work to build a platform that makes it easier for anyone to write and run tests. Ops isn’t responsible for managing all of the infrastructure, but they work to build a platform that makes it easier for anyone to deploy services quickly and with great success. Security doesn’t stand in the corner telling what to do, but they work together to build tools and systems that can help an app team discover and remediate security issues.

The solution is not no-code anything. The solution is software engineering and building platforms to reduce the “toil”, build repeatable automated processes, and work together and listen to each other to make sure we ship high-quality services for our customers.

Full-cycle development. Similar to the concept of “cross-training” in your first job in fast food. It’s like “full-stack development”, but encompasses everything a service needs to succeed and not be a pile of shit.

Re: Asking developers to do QA is broken – why anyone should own QA

#193

Post by desc » Wed, Jul 14, 2021, 10:54 PM UTC

ozim wrote: post #154

desc wrote: post #49

Silver bullets. They don't exist. Code review. Read the results of someone thinking through a process. Spot more than they will, simply by throwing more eyes at it. Actually fairly effective: getting a senior dev to cast even a lazy eye over everything gives more opportunities to discuss Why It's Done This Way and Why We Don't Do That and Why This Framework Sucks And How To Deal With It with specific concrete example…

Code reviews should be about project structure and abstractions and keeping approach in order or to use team common approach instead of each team member doing whatever, well syntax/code should be linted and formatted automatically nothing for reviewer. Second thing is checking by second pair of eyes if they understand code in question in the same way. Unit and integration tests should not be generated. Those should b…

It depends how you count the unit/integration tests, really.

If you've got a general rule which must apply across an entire system, generate the necessary tests so that they fail granularly and don't require messing around to find the exact case which breaks. IMO that's one test, just applied to a range of cases.

An example might be mappings for Entity Framework (or similar ORMs, etc). Auto-generated migrations simply do not work if you need to limit migration downtime and maintain certain data invariants (which can't be specified in the schema, and yes, those always exist). So you need to write database migrations manually. This introduces risk of desync of entity mappings and schema.

So don't just spot-test roundtripping entities (a nontrivial system will have hundreds and something always gets missed). Instead, write a tool to introspect the DB schema and the Framework's mappings, and check that they match sufficiently closely. Every time someone adds an entity or property or something, it's already covered.

Similar cases exist when dealing with any interface between separate systems, especially when you don't control one of them. If you're regularly mapping between two models, use something like Automapper which can be asked to verify its mappings to check that every property is handled in some way.

(Granted, Automapper doesn't catch everything, but it builds a model that could probably be introspected over to spot encountered bugs and check that other possible examples of those bugs don't exist. Doing so generatively catches future additions of possible cases for free. If you're really paranoid, define some means of marking manually-written tests which cover each case, and test that a test exists for each case.)

Computers are really good at force-multiplication. They should trivially be capable of spotting other instances of known categories of bug. This is not hard to do and doesn't require wooly nebulous machine-learning shit: we've had introspectable ASTs since the dawn of compilers.