Live data from Hacker News

Maybe getting rid of your QA team was bad

davidkcaudill.medium.com

151–160 of 269 posts

Re: Maybe getting rid of your QA team was bad

#151
post #56

I worked at two companies 15-20 years ago that invested in top-tier QA teams. They were worth their weight in gold. The products were world class because the QA team were fantastic at finding bugs we developers didn't think of looking for because we were too close to the problem. We are too used to looking at the happy path. One key attribute to both companies is that it was dictated from on high that the QA team had…

I've seen developers writing tests with no assertions, or with assertTrue(true). Always green, with 100% coverage! The same people have been asking: why should I write tests if I can write new features? And then one senior QA comes and destroys everything. Once I found that if I press f5 50 times during a minute then backend will go in outOfMemory while spinning requests to the database.

That's why the dev teams tests and the QA teams tests are not mutually exclusive.

Re: Maybe getting rid of your QA team was bad

#152
So for me, the QA team is the best source of product information in the entire engineering team. If they also do customer triage, then probably in the entire company.

They should know the product inside out, moreover, they know all the annoying bits that are unsexy and not actively developed.

yes, they find bugs and junk, but, they know how your product should be used, and the quickest/easiest way to use it. Which are often two different paths.

Bring your QA in the product cycle, ask them what the stuff that pisses them off the most.

They also should be the masters of writing clear and precise instructions, something devs and product owners could learn from.

Re: Maybe getting rid of your QA team was bad

#153
post #103

At the start of my career (late 70s), I worked at IBM (Hursley Park) in Product Assurance (their version of QA). We wrote code and built hardware to test the product that was our area of responsibility (it was a word processing system). We wrote test cases that our code would drive against the system under test. Any issues we would describe in general terms to the development team -- we didn't want them to figure out…

> When I've worked for organizations without QA teams, I introduce the concept of "sniff tests". This is a short (typically 1 hour) test session where anybody in the company / department is encouraged to come and bash on the new feature. We call those bug-bashes where we work, and they're also typically very productive in terms of defects discovered! It's especially useful since during development of small features,…

Maybe we work in the same company. I'd like to add that usually the engineer responsible for the feature being bug-bashed is also responsible of refining the document where everyone writes the bugs they find since a lot are duplicates, existing bugs, or not bugs at all. The output is then translated into Jira to be tackled before (or after) a release, depending on the severity of the bugs found.

Re: Maybe getting rid of your QA team was bad

#154
post #103

At the start of my career (late 70s), I worked at IBM (Hursley Park) in Product Assurance (their version of QA). We wrote code and built hardware to test the product that was our area of responsibility (it was a word processing system). We wrote test cases that our code would drive against the system under test. Any issues we would describe in general terms to the development team -- we didn't want them to figure out…

I've always been impressed by hardware QA test teams I've worked with. On Google Fiber, they had an elaborate lab with every possible piece of consumer electronics equipment in there, and would evaluate every release against a (controlled) unfriendly RF environment. ("In version 1.2.3.4, the download from this MacBook Pro while the microwave was running was 123.4Mbps, but in version 1.2.4.5, it's 96.8Mbps." We actually had a lot of complexity beyond this that they tested, like bandsteering, roaming, etc.) I was always extremely impressed because they came up with test cases I wouldn't have thought of, and the feedback to the development team was always valuable to act on. If they're finding this issue, we get pages of charts and graphs and an invite to the lab. If a customer finds this issue, it just eats away at our customer satisfaction while we guess what could possibly have changed. Best to find the issue in QA or development.

As for software engineers handling QA, I'm very much in favor of development teams doing as much as possible. I often see tests bolted on to the very end of projects, which isn't going to lead to good tests. I think that software engineers are missing good training on what to be suspicious of, and what best practices are. There are tons of books written on things like "how to write baby's first test", but honestly, as an industry, we're past that. We need resources on what you should look out for while reviewing designs, what you should look out for while reviewing code, what should trigger alarm bells in your head while you're writing code.

I'm always surprised how I'll write some code that's weird, say to myself "this is weird", and then immediately write a test to watch it change from failing to passing. Like times when you're iterating over something where normally the exit condition is "i > I like putting "" into every field and pressing submit.

Going deeper into the training aspect, something I find very useful are fuzz tests. I have written a bunch of them and they have always found a few easy-to-fix but very-annoying-to-users bugs. I would never make a policy like "every PR must include a fuzz test", but I think it would be valuable to tell new hires how to write them, and why they might help find bugs. No need to have a human come up with weird inputs when your idle CI supercomputer can do it every night! (Of course, building that infrastructure is a pain. I run them on my workstation when I remember and it interests me. Great system.)

At the end of the day, I'm somewhat disappointed in the standards that people set for software. To me, if I make something for you and it blows up in your hands... I feel really shitty. So I try to avoid that in the software world by trying to break things as I make them, and ensure that if you're going to spend time using something, you don't have a bad experience. I think it's rare, and it shouldn't be, it should be something the organization values from the top to the bottom. I suppose the market doesn't incentive quality as much as it should, and as a result, organizations don't value it as much as they should. But wouldn't it be nice to be the one software company that just makes good stuff that always works and doesn't require you to have 2 week calls with the support team? I'd buy it. And I like making it. But I'm just a weirdo, I guess.

Re: Maybe getting rid of your QA team was bad

#155

I had a boss at Yahoo who gave our QA to another team because "Facebook doesn't use QA, we shouldn't either". I can't remember if it was Facebook or MS, but he was willing to buy all of us a book talking about how amazing it was. Long story short, it wasn't. It was like taking away a crutch. Of course we could have been more diligent about testing before having QA validate it, but it slowed development down so much t…

An article about Facebook's reason for no QA with some of the mitigations:

https://blog.southparkcommons.com/move-fast-or-die/

A bit recent to have affected Yahoo - but it sells a good story.

  We would celebrate the first time someone broke something.

  Let anyone touch any part of the codebase and get in there to fix a bug or build a feature. Yes, this can cause bugs. Yes, that is an acceptable tradeoff. We had a shared mythology about the times that the site went down and we all rallied together to quickly bring it back up.

Sounds like hell: running as close to the edge of the cliff as you can. Presumably totally ignoring thousands of papercuts of slightly broken functionality. Optimising to produce an infinite number of shallow bugs.

Re: Maybe getting rid of your QA team was bad

#156
“ To these folks, it feels like giving a damn is a huge career liability in your organization. Because it is.”

And it’s easy to see why.

Software Quality, Cose Maintainability, Good Design. These things only matter if you are planning to work on that company for a long time. If you’re planning to stay a couple years then hop to the next company, the most optimal path is to rise fast by doing high visibility work, then find use your meteoric rise as a resume material to get a higher paying job. Rinse and repeat. If that project is going to break or become unmaintainable in a couple years, who cares? You’re not going to be there.

Recognize the pattern? Startups work the same. It’s the “growth mindset” imprinted everywhere. If this product becomes unmaintainable in 5 years, who cares? I will have exited and cashed in.

I don’t judge people who do that exactly because it’s the practice the companies themselves use. I don’t like it, I actually hate it, but I understand people are just playing by the rules.

The fun part is watching managers and executives complaining about employee turnover, lack of company engagement, quiet quitting, like this isn’t them tasting their own poison.

Re: Maybe getting rid of your QA team was bad

#157

Earlier quoted context omitted.

That’s not because “QA had the final say”, it’s because your QA team were ass clowns. Any QA team that discourages dev from writing or running tests needs to be burnt to ground and rebuilt.

Unless dev is switching to write test 100% of time and become a tester it is highly not recommended to let developers write tests. That QA was not a clown, they've seen some shit... Would you let QA to write features in your production code?

Test code is just code. If you can write test code you can write production code. If you can write production code you can write tests.

If your concern is that devs don't have the right mindset for testing, you can have them collaborate with a QA specialist to define the test cases and review the test implementation.

Re: Maybe getting rid of your QA team was bad

#158

Earlier quoted context omitted.

> This often creates a situation where people need to "justify" their jobs. On DevOps teams I see this constantly. Usually the best-compensated or most senior "Ops" guy or whatever they're called at the company spends a lot of his time extinguishing fires that were either entirely of his own creation/incompetence, which makes it look like he's "doing something." You automate away the majority of the toil there and th…

Which is one reason DevOps teams don't make sense. DevOps is a skill developers need to have. It needs to be embedded within the development team, not some other team's responsibility who only focuses on "DevOps" work. You create the build and deployment pipelines and move on to other project work. If you give someone a role and say their job is to do "DevOps" they will HAVE to invent things to do because that's such…

It is not about DevOps. In every organization if you do your work good and have no fuckups - you will not be recognized and promoted. But if you are hero-firefighter - managers will love you and help with promotion.

Because visibility is a key! Key to everything in the corporate life. If your work is not visible for managers - you are doing nothing.

Re: Maybe getting rid of your QA team was bad

#159

I hired a great QA Developer a few months ago. They are building out integration, performance and end to end tests; finding bugs, speeding up releases and generally making things better. I get asked, every week, if they are contributing and what are they contributing. It's exhausting, so I can't imagine what it feels like to actually BE in QA.

# of bugs found per week should be sufficient metric of productivity.

Re: Maybe getting rid of your QA team was bad

#160

“ To these folks, it feels like giving a damn is a huge career liability in your organization. Because it is.” And it’s easy to see why. Software Quality, Cose Maintainability, Good Design. These things only matter if you are planning to work on that company for a long time. If you’re planning to stay a couple years then hop to the next company, the most optimal path is to rise fast by doing high visibility work, the…

> Startups work the same. If this product becomes unmaintainable in 5 years, who cares?

This is a reasonable stance for a startup to take. The majority of startups likely won't last five years as they tend to fail.

Being alive in five years with technical debt is a good problem for most startups to have, because that means they managed to make it five years.

Post reply on HN