Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Maybe getting rid of your QA team was bad

davidkcaudill.medium.com

Post reply on HN
Print view View original

211–220 of 269 posts

Re: Maybe getting rid of your QA team was bad

#211

Post by semireg »

ansible wrote: post #163
jxramos wrote: post #106

Earlier quoted context omitted.

that's very interesting to hide the source of the automated tests from the developers as a strategy. I can see that shifting the focus to not just disabling the test or catering to the test etc. I'll have to think about this one, there's some rich thoughts to meditate on with this one.

It is an interesting approach I hadn't heard of before. For complex systems though, often reproducing the bug reliably is a large part of the problem. So giving the developers the maximum information is necessary. Any time a "fix" is implemented, someone needs to be asking the right questions. Can this type of problem occur in other features / programs? What truly is the root cause, and how has that been addressed?

Wow, less IS more. Hear me out.

How do we measure “more” information? Is it specificity or density?

Because here, assuming they can reproduce, keeping the information fuzzy can make the problem space feel larger. This forces a larger mental-map.

Re: Maybe getting rid of your QA team was bad

#212

Post by jdlshore »

rubidium wrote: post #120
robofanatic wrote: post #102

Earlier quoted context omitted.

> QA is almost always seen as a 'cost center' by the business and upper management Well everything involved in making a product is seen as a cost, that includes the entire development team - QA, Developers, Devops, PM ....

No. That’s not actually how most orgs break it down. R&D, marketing, sales is “bringing new business” so are profit centers. This means their budget grows with revenue. Manufacturing, QA, IT and service are cost centers so get squeezed year-over-year even if revenue is flat.

It depends on the org. In my company, which is a SaaS-like company, all of product and engineering is a cost center, despite creating the product the company sells. It’s just the way they do their accounting.

Re: Maybe getting rid of your QA team was bad

#214

Post by itsdrewmiller »

rubidium wrote: post #120
robofanatic wrote: post #102

Earlier quoted context omitted.

> QA is almost always seen as a 'cost center' by the business and upper management Well everything involved in making a product is seen as a cost, that includes the entire development team - QA, Developers, Devops, PM ....

No. That’s not actually how most orgs break it down. R&D, marketing, sales is “bringing new business” so are profit centers. This means their budget grows with revenue. Manufacturing, QA, IT and service are cost centers so get squeezed year-over-year even if revenue is flat.

What software companies are not classifying QA as R&D?

Re: Maybe getting rid of your QA team was bad

#215

Post by shiroiuma »

EvanAnderson wrote: post #12
robotnikman wrote: post #4

Microsoft is probably the one case where this sticks out the most, at least for me anyways. Noticeably more bugs in updates since they dropped their QA team, in Windows as well as cloud products.

The revenue keeps rolling in so, clearly, they made the right business decision... >sigh I use a lot of MSFT software and services in the "day job". I wish there was some kind of consequence to them for their declining quality.

>The revenue keeps rolling in so, clearly, they made the right business decision... >sighThis is exactly the point I was going to make. Their stock price is doing great! So obviously they've done the right thing for their position in the market: the market has rewarded them for not wasting money on QA and just letting users suffer with the bugs.

>I wish there was some kind of consequence to them for their declining quality.

If people keep insisting on throwing money at them no matter how bad their software is, then there's no reason for them to improve their quality.

Re: Maybe getting rid of your QA team was bad

#216

Post by jdlshore »

temuze wrote: post #62

I strongly disagree. I worked at a company with a world-class QA team. They were amazing and I can't say enough nice things about them. They were comprehensive and professional and amazing human beings. They had great attention to detail and they catalogued a huge spreadsheet of manual things to test. Engineers loved working with them. However -- the end result was that engineers got lazy. They were throwing code ove…

This phenomenon is very well described by Elisabeth Hendrickson’s “Better Testing, Worse Quality” article.

Re: Maybe getting rid of your QA team was bad

#217

Post by notpachet »

krisoft wrote: post #172
pjsg wrote: post #103

At the start of my career (late 70s), I worked at IBM (Hursley Park) in Product Assurance (their version of QA). We wrote code and built hardware to test the product that was our area of responsibility (it was a word processing system). We wrote test cases that our code would drive against the system under test. Any issues we would describe in general terms to the development team -- we didn't want them to figure out…

> I've been in these test session where we have generated 100 bug tickets in an hour. Is that like… usefull to anyone? Especially if they are duplicates. It feels to me that 10 different bugs is enough to demonstrate that the feature is really bad, after that you are just kinda bouncing the rubble?

As noted in the post, one characteristic of a healthy QA environment at your work is how effectively you triage bugs. That includes detecting duplicates. One big QA smell for me is opening a team's bug backlog and realizing that there are shitload of dupes in there, because it means that no one really looked at them in detail.

Re: Maybe getting rid of your QA team was bad

#218

Post by godelski »

ponector wrote: post #158
tstrimple wrote: post #122

Earlier quoted context omitted.

Which is one reason DevOps teams don't make sense. DevOps is a skill developers need to have. It needs to be embedded within the development team, not some other team's responsibility who only focuses on "DevOps" work. You create the build and deployment pipelines and move on to other project work. If you give someone a role and say their job is to do "DevOps" they will HAVE to invent things to do because that's such…

It is not about DevOps. In every organization if you do your work good and have no fuckups - you will not be recognized and promoted. But if you are hero-firefighter - managers will love you and help with promotion. Because visibility is a key! Key to everything in the corporate life. If your work is not visible for managers - you are doing nothing.

> It is not [just] DevOps

Definitely agree. In a sister comment[0] I mention a concept I've been calling "Goodhart's Hell" for a more general term. But I think what you are specifically mentioning is sometimes laughably called "Loud Laboring"[1]. I think this all falls under the broader umbrella of metric hacking and thus Goodhart's Law.

Idk why, but it really does seem like metric hacking is extremely pervasive in our modern society, and can be found nearly everywhere. What upsets me the most is that it too is found in the sciences. There also appears to be a strong correlation between the popularity (or hype) of a field and metric hacking.

[0] https://news.ycombinator.com/item?id=38647582

[1] https://news.ycombinator.com/item?id=37147707 || https://www.cnbc.com/2023/08/09/forget-quiet-quitting-loud-l...

Re: Maybe getting rid of your QA team was bad

#219

Post by godelski »

JohnMakin wrote: post #195
ponector wrote: post #158

Earlier quoted context omitted.

It is not about DevOps. In every organization if you do your work good and have no fuckups - you will not be recognized and promoted. But if you are hero-firefighter - managers will love you and help with promotion. Because visibility is a key! Key to everything in the corporate life. If your work is not visible for managers - you are doing nothing.

The fortunate thing is there are measurable metrics you can hit and improve upon as an X-Ops team, but implementing those takes buyin from the org or enough freedom to go rogue and do it on your own.

> The fortunate thing is there are measurable metrics you can hit and improve upon as an X-Ops team

Question(s):

- What are the metrics?

- How aligned are the metrics with the actual goal?

- What isn't covered by the metrics?

- How is what's not covered by metrics evaluated?

- Can what's not currently covered by metrics theoretically be covered by some potentially unknown metric? (best guess)

Re: Maybe getting rid of your QA team was bad

#220

Post by itsdrewmiller »

mixmastamyk wrote: post #101
temuze wrote: post #62

I strongly disagree. I worked at a company with a world-class QA team. They were amazing and I can't say enough nice things about them. They were comprehensive and professional and amazing human beings. They had great attention to detail and they catalogued a huge spreadsheet of manual things to test. Engineers loved working with them. However -- the end result was that engineers got lazy. They were throwing code ove…

False dichotomy. Poor dev practice is not fixed by elimination of QA, but rather fixed by improving dev practice. The “five why’s” can help.

If you're working without a net, you're going to be more careful. And 5 whys is not a particularly great overall practice. https://qualitysafety.bmj.com/content/26/8/671
Post reply on HN