Live data from Hacker News

Should QA exist?

rubick.com

151–160 of 160 posts

Re: Should QA exist?

#151

100% and I’m a software developer and have been for ~30 years. Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering” and that it should exist. One of the QA people I work with currently is one of my favorite people. They don’t always make me happy (in the moment) with their bugs or with how they decide to break the software,…

Couldn't agree more strongly. The best QA people have the old-school “test to break” mentality. They do weird things, like pulling the network cable out of their machine mid-transaction, or kicking off a series of performance scripts and then powering off servers in a distributed system just to see what happens. When I got started in software, QA was already in a heavy decline. A mentor who had been a QA manager at A…

When I got into software that employer was pretty small (50 people overall I think).

Their approach to QA was to have this be a optional thing the service people could do.

It worked surprisingly well, with the caveat that they never created regression tests.

The employer eventually switched someone from that team to establish these regression tests full time, but they had no programming experience and by the time I left no real progress was done in around 6 month. No idea what came of that, and a few years later they fired a lot of the team

Re: Should QA exist?

#152

100% and I’m a software developer and have been for ~30 years. Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering” and that it should exist. One of the QA people I work with currently is one of my favorite people. They don’t always make me happy (in the moment) with their bugs or with how they decide to break the software,…

> with their bugs

This is funny.

Re: Should QA exist?

#153
> Note this isn’t because QA is finding issues — it’s because of the back and forth.

But it isn't? Finding and fixing are the major time sinks?

> You’re creating bad behavior for your engineers by not having them responsible for their quality.

What prevents you from maintaining responsibility if QA finds the bug? The thing you're throwing over is the investigation/finding, it doesn't follow that you should absolve the throwers

Re: Should QA exist?

#154

100% and I’m a software developer and have been for ~30 years. Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering” and that it should exist. One of the QA people I work with currently is one of my favorite people. They don’t always make me happy (in the moment) with their bugs or with how they decide to break the software,…

> Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering” and that it should exist.

I think the core of the issue is the weasel word "Good QA" instead of just "QA". You're underlining the importance of having someone in a team who has good product understanding and has good ownership skills. How many QAs fit that description? Not many, unfortunately. Some FANGs outright eliminated the role and replaced it with a mix of product owners and team ownership, and some QAs are just doing their 9-to-5 job going through their manual test scripts to verify it something meets a definition of done. What happens when you can have a LLM agent doing the same instantly as a step in a random CICD pipeline?

Re: Should QA exist?

#155

Earlier quoted context omitted.

Unit tests very roughly double the amount of effort required to make any meaningful change to your codebase. They are also require maintenance same as ordinary code - but the customer does not care in the slightest whether or not they pass. On the other side, they can only really tell you about low level bugs that you already expected, they cannot surface system level bugs - the actual hard bugs that cause problems f…

Thank you, that makes sense. What I meant was that today all unit tests are basically written by an AI so the "cost" is almost zero. Am I wrong?

Sorry yes. If an LLM written unit test fails, then it has to be determined whether the test was wrong or the code was wrong. This is an expense in human oversight, unless of course we believe that LLMs will get it right at a high enough rate that they can be left to code everything themselves completely automatically.

Re: Should QA exist?

#156

Earlier quoted context omitted.

Ive almost never worked on a project where there was the right number of QAs who were doing the right thing. Usually there either arent any in which case bugs get missed or there are 5 very cheap ones running mindless scripts who are standing in for the devs' inability or unwillingness to write decent automated tests but dont catch the really deep level thorny stuff.

QA has a reputation problem. Because it is considered as unimportant role, good people don't get attracted to it. The average people who do don't do a good job.

> QA has a reputation problem. Because it is considered as unimportant role, good people don't get attracted to it.

Hard disagree. QAs' main tasks are acceptance testing and V&V. This means their work is to act as independent parties whose role is to verify that the results are indeed being delivered. Their importance is up there next to Product Owners.

The problem is that some QA roles are relegated to running the same old test scripts and do some bullshit manual exploratory tests that catch and assert nothing. It's the ultimate bullshit job as it's performative. This is clear in the way that LLM agents are seen as threatening the very existence of the role.

Re: Should QA exist?

#157
In my opinion, the first mistake this article makes is assuming QA should be a completely separate team. My experience has shown that QA works best when it is an integral part of the engineering team. When you silo QA, you create a "throw it over the wall" dynamic where testing becomes a bottleneck rather than a collaborative process. Instead, they should be managed alongside developers and attend the same meetings.

What I find works best is when the Product Manager, Developer Lead, and QA Lead collaborate on requirements. The Product Manager designs the feature, the Developer focuses on technical design and architecture, and the QA Lead maps out user scenarios, integration points, and edge cases. Together, they define the requirements that constitute a successful feature.

When QA is involved early in the process, they understand and drive exactly how the feature should behave. This ensures that critical cases are considered and potential risks are handled before a single line of code is written.

Re: Should QA exist?

#159
post #4

"Before I weigh in further, I’d like to make sure you’re familiar with the testing pyramid." The testing pyramid is a par excellance SWE kool-aid. Someone wrote a logically-sounding blogpost about it many years ago and then people started regurgitating it without any empirical evidence behind it. Many of us have realised that you need a "testing hourglass", not a "testing pyramid". Unit tests are universally consider…

> Isolated QA should not exist because anything a QA engineer can do manually can be automated.

I almost entirely agree. You'd certainly want a majority of QA responsibilities to fall to devs since that keeps the feedback loop tight. Also with the amount of compute we have you can trivially outpace a typical QA team with a single dev working on fuzz tests and property based tests. If you really still feel the need to outsource then there are cool tools like Antithesis [1] which let you trade money for compute in their complicated fuzz testing suite.

Where that falls apart imo is when your software interacts with hardware in any non-trivial way. Fuzz testing doesn't really work when your system is spinning big and heavy things for example. Real hardware that interacts with the world will always create this gap where the devs don't know how to fully test it. Your systems engineers are best equipped to handle this and ensure certain controls follow the right curves, but they typically don't have the bandwidth or are very inexperienced with software. I think a QA organization _can_ fill this gap and deliver value. But I'd almost always prefer hiring another 1 or 2 system engineers who can work towards this problem full time instead. It's much easier to train someone in software than it is to train them in motor control or sealed systems.

[1] https://antithesis.com/

Re: Should QA exist?

#160

I was in testing for 17 years before moving back into Engineering. I have spent my time in Engineering leading teams to push quality left. But I think it's better to say "quality is a system" than to say "engineers own quality." What are you building into your SDLC that makes sure quality happens? Testing is just a part of that, and not even the biggest one.

Quality mindset should always exist, so should quality checkpoints. To mature your system and practices, your code generator, code reviewer and quality checker, shouldn't be the same (people or agents)
Post reply on HN