Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Maybe getting rid of your QA team was bad

davidkcaudill.medium.com

Post reply on HN
Print view View original

231–240 of 269 posts

Re: Maybe getting rid of your QA team was bad

#231

Post by scruple »

wmichelin wrote: post #7

This might be my personal experience, but I've never encountered a QA team that actually writes the tests for engineering. I have only had QA teams that wrote "test plans" and executed them manually, and in rarer cases, via automated browser / device tests. I consider these types of tests to be valuable, but less so than "unit tests" or "integration tests". With this model, I have found that the engineering team ends…

We have SDETs for that. And they do a great job. But QA is where polish happens. When you get good QA people, who know the app better than the developers, better than the users, who anticipate how the users will use the product? These people should be paid their weight in gold.

Re: Maybe getting rid of your QA team was bad

#232

Post by truculent »

bradleyjg wrote: post #200
amtamt wrote: post #54

> The most conscientious employees in your organization are the most bitter. They see the quality issues, they often address them, and they get no recognition for doing so. When they speak up about quality concerns, they get treated like mouthbreathers who want to slow down. They watch the “move fast and break things” crowd get rewarded time after time, while they run around angrily cleaning up their messes. To these…

DBAs and Infra, are in the same boat as QAs. Pendulam will swing back in not so long time frame i hope. Ultimately it’s up to the customers. Will they walk because of bugs and outages or stay because of shiny new features?

I’m not so sure. I think software trends in the last 10-15 years or so have been driven heavily by a small number of influential players (FAANG, mega-VCs).

An influx of dumb money like that can shape and distort the market and overwhelm the feedback loops that would otherwise give consumers influence.

Perhaps now rates are up, the equilibrium changes, but I think it’s still easy to overestimate the number of “first movers” in an industry and the power of tacit or unconscious collusion.

Re: Maybe getting rid of your QA team was bad

#233

Post by thaumasiotes »

alternatex wrote: post #229
thaumasiotes wrote: post #223

Earlier quoted context omitted.

It's more because code coverage isn't measured correctly. What you want is branch coverage. If you have something like this: if condition1: do_something1() if condition2: do_something2() if condition3: do_something3() There are 8 possible paths for the code to follow here, but you can cover 100% of lines of code in one test. If your code coverage measure tells you that testing the case where conditions 1, 2, and 3 ar…

I think most code coverage analysis tools are smart enough to consider flow control statements. At least when it comes to C# ones.

Why do you think that? It isn't true.

Re: Maybe getting rid of your QA team was bad

#234

Post by zaphirplane »

JohnFen wrote: post #175
Shaanie wrote: post #165

Earlier quoted context omitted.

That's not a problem with Scrum, it's a problem with your team. If you're doing a bug bash every sprint, then your velocity is already including the time spent on bug bashes. If it's not in every sprint, you can reduce the forecast for sprints where you do them to account for it (similar to what you do when someone is off etc). If you're competing within the team to complete as many story points as possible that's pr…

> Is someone using story points as a metric of anything other than forecasting? Very nearly every company I've worked at that uses Scrum uses story points, velocity, etc., as a means of measuring how good you or your team are. Forecasting is a secondary purpose.

Sounds easy to game. Click a button to foo the bar is one million points

Re: Maybe getting rid of your QA team was bad

#235

Post by deniscepko2 »

amtamt wrote: post #54

> The most conscientious employees in your organization are the most bitter. They see the quality issues, they often address them, and they get no recognition for doing so. When they speak up about quality concerns, they get treated like mouthbreathers who want to slow down. They watch the “move fast and break things” crowd get rewarded time after time, while they run around angrily cleaning up their messes. To these…

Yeah this hit home so hard. Had to leave the startup i loved working at, because started doing all this release fast crap. And somehow people think releasing fintech stuff fast and untested is fine.

Re: Maybe getting rid of your QA team was bad

#236

Post by Kinrany »

Osmose wrote: post #50
chopin wrote: post #36

Earlier quoted context omitted.

In my experience such tests are brittle as hell.

You're not wrong, but a good, well resourced QA org can both help write or develop more flexible tests, and also help fix brittle tests when they do break. The idea of brittle tests that break often being a blocker is predicated on practices like running every type of test on every commit that exist to deal with a lack of QA effort in the first place. Maybe recorded integration tests are run on every release instead…

> Maybe recorded integration tests are run on every release instead of every commit?

That would limit the frequency of releases.

Re: Maybe getting rid of your QA team was bad

#237

Post by physicsguy »

I personally think that having worked in both environments, the biggest issue is companies missing that triage function for bugs, etc. from people who can turn a customer reported bug into a step by step guide for reproducing things.

When I worked in the simulation space we used to get models sent in by customers where a convergence problem or crash would occur in 12 hours of running on a 128 core machine. Those were impossible as a developer to work with in debug mode which made the runtime even longer, so they needed someone to identify the cause of the problem and distill it down to a much smaller model where the bug could be replicated. The QA team in that were really application engineers and were subject matter experts, and they were absolutely invaluable.

Re: Maybe getting rid of your QA team was bad

#238

Post by lulznews »

pjsg wrote: post #103

At the start of my career (late 70s), I worked at IBM (Hursley Park) in Product Assurance (their version of QA). We wrote code and built hardware to test the product that was our area of responsibility (it was a word processing system). We wrote test cases that our code would drive against the system under test. Any issues we would describe in general terms to the development team -- we didn't want them to figure out…

>Any issues we would describe in general terms to the development team -- we didn't want them to figure out our testcases

I’m sorry but this is just lol. Did the devs play back by creating bugs and seeing if your team could find them?

Re: Maybe getting rid of your QA team was bad

#239

Post by Arainach »

hedora wrote: post #208
Zelphyr wrote: post #56

I worked at two companies 15-20 years ago that invested in top-tier QA teams. They were worth their weight in gold. The products were world class because the QA team were fantastic at finding bugs we developers didn't think of looking for because we were too close to the problem. We are too used to looking at the happy path. One key attribute to both companies is that it was dictated from on high that the QA team had…

The best approach I’ve seen is to do all of the above. - Have QA run pessimal versions of real use cases. Trying to sell a word processor to lawyers? Format the entire US legal code and a bajillion contracts in it, then duplicate it 10x and start filing usability/performance bugs. - Have the engineers test everything with randomly generated workloads before committing. Run those tests nightly, and fix all the crashes…

The devil is in the details of "essential development tasks".

An uncountable number of products have died or devolved because "we don't have time to do it that way, put in the quick fix"

Re: Maybe getting rid of your QA team was bad

#240

Post by ryan-duve »

jrockway wrote: post #154
pjsg wrote: post #103

At the start of my career (late 70s), I worked at IBM (Hursley Park) in Product Assurance (their version of QA). We wrote code and built hardware to test the product that was our area of responsibility (it was a word processing system). We wrote test cases that our code would drive against the system under test. Any issues we would describe in general terms to the development team -- we didn't want them to figure out…

I've always been impressed by hardware QA test teams I've worked with. On Google Fiber, they had an elaborate lab with every possible piece of consumer electronics equipment in there, and would evaluate every release against a (controlled) unfriendly RF environment. ("In version 1.2.3.4, the download from this MacBook Pro while the microwave was running was 123.4Mbps, but in version 1.2.4.5, it's 96.8Mbps." We actual…

> Going deeper into the training aspect, something I find very useful are fuzz tests.

Could you share some details of fuzz tests that you've found useful? I tend to work with backend systems and am trying to figure out whether they will still be useful in addition to unit and integration tests.

Post reply on HN