Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yahoo’s Engineers Move to Coding Without a QA Team

spectrum.ieee.org

Post reply on HN
Print view View original

141–150 of 188 posts

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#141

Post by jrumbut »

pc86 wrote: post #45
arielweisberg wrote: post #37

Earlier quoted context omitted.

Moving from an environment with a 10:1 dev:qa to 2:1 showed me what happens when dev is not responsible for shipping working software. No thanks. It's a bunch of deflection and diffusion of responsibility coupled with high latency flakey interactions between different teams. Everything that can slip through the cracks does slip through the cracks. I'm sure QA can be done well, but I am convinced that giving your devs…

The best way I've seen it done is when another developer, on the same team, who is responsible for the same product , reviews your code. Not someone who writes tests for a living, someone who does the exact same job as you on the same product. Any time I've experienced something different it's been exactly as you described.

I agree with you that that is a necessary part of the process and a great first step for small teams moving away from "just release it and see if it works". The unfortunate reality though is that people in the same role on the same team often have the same misconceptions and blind spots.

Now getting devs to see the product through the users' eyes goes a long way toward solving that, but if you have a process and team of devs that are doing that you're way ahead of the game in a lot of ways.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#142

Post by jrumbut »

jboy55 wrote: post #122
vkou wrote: post #120

Earlier quoted context omitted.

What will happen is that devs will ask their QA counterparts to report issues through an undocumented side-channel. If you give an engineer a career incentive to optimize something, you'd be surprised how seriously some will take it.

Incentives often cause really nasty political wars within orgs. When a site gives a 500 because a database went down and the web app couldn't connect to it... is that a bug, a missed test case or should ops take a hit on down time? Furthermore, if you argue that the dev team should have reasonable failsafes in code to connect to a db, in a 10 year old organization, should the current Dev team pay for something that c…

You're absolutely right, assuming we're talking about business/consumer apps and not pacemakers or rocket ships.

An incentive/penalty system for bugs without an incentive/penalty system for features completed leads to paralysis. And a complicated incentive system leads to game playing over productivity.

It's an old saying: be careful what you measure because you'll get a lot more of it.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#143

Post by Pizzalover »

ef4 wrote: post #29

Surprised to see the negativity here. I have worked in environments with traditional manual QA, and environments where all development is test-driven and nobody is allowed to merge a feature that lacks automated test coverage. Both the productivity and the quality were higher in the places with fully automated testing. Which is not shocking at all: does anybody really think a human can run through 800 test cases bett…

Surprised? Man, this is HN!!

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#145

Post by plinkplonk »

When I was working at ThoughtWorks, we had devs writing automated unit tests with close to 100 percent coverage, and also QA (who also automated as much of their testing as possible, using a variant of the FIT framework in those days) finding significant bugs and show stoppers,.

In my experience, one is not a substitute for the other.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#146

Post by mixmastamyk »

hofmann wrote: post #103
vox_mollis wrote: post #28

Earlier quoted context omitted.

I worked for a company that didn't invest in QA Where are these magical places that do invest in QA? In nearly 20 years of professional development, I've never seen an organization in which the criteria for shipping was anything other than "works for me". I have never seen an organization in which there was either budget or managerial patience for proper QA, let alone anything other than VERY basic acceptance testing…

I once worked in an organization that has 1.5-2x QA then dev. I now work in a place which has 1 QA for every 8 devs -- and there are far less bugs here then the other. I think the reason is proper tooling, a culture of thorough automated testing, and ownership of code.

Ownership and the resulting pride in workmanship makes a big difference.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#147

Post by kyllo »

steve371 wrote: post #144

I'd vote for having a QA team. Not for quality control purpose. But to have someone think outside the box. Sometimes, you will be surprise when you talked to the QA team. And you could not get those ideas from dev peer review.

QA Team is great if it is a team of developers who are interested in QA and test automation/tooling. Not so cool if it's a department full of people who make low hourly wages to execute manual tests and don't write code.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#148

Post by diivio »

This isn't surprising.

Microsoft switched to this model a few months after Satya took over.

For the majority of Microsoft teams it worked really well and showed the kinds of results mentioned in this yahoo article. Look at many of our iOS apps as an example.

But for some parts of the Windows OS team apparently it didn't work well (according to anonymous reports leaked online to major news outlets by some Windows team folks) and they say it caused bugs.

First of all I think that argument is semi-BS and a cover up for those complainer's lack of competence in testing their code thus making them bad engineers because a good engineer knows how to design, implement, and test their product imo. But I digress.

I in no way want to sound like a dk but as an engineer it is your responsibility to practice test driven development but that's not enough.

Like reading an essay you usually can't catch all of your own bugs and thus peer editing or in this case cross testing is very useful.

You should write the Unit tests and integration tests for your feature

BUT

There should always be an additional level of end to end tests for your feature written by someone else who is not you.

Everyone should have a feature and design and implement it well including its Unit tests and integration tests BUT they should also be responsible for E2E tests for someone else's feature.

That way everyone has feature tasks and test tasks and no one feels like they are only doing one thing or stuck in a dead end career.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#149

Post by diivio »

diivio wrote: post #148

This isn't surprising. Microsoft switched to this model a few months after Satya took over. For the majority of Microsoft teams it worked really well and showed the kinds of results mentioned in this yahoo article. Look at many of our iOS apps as an example. But for some parts of the Windows OS team apparently it didn't work well (according to anonymous reports leaked online to major news outlets by some Windows team…

Also making sure you have a good automated build,test,deploy structure is important too.

It's all the little details that will determine if this system will succeed or not.

Not the overall "big idea".

Implementation of this system and the competence and willingness to adapt of the team members is key imo. At least for this issue.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#150

Post by benihana »

DougWebb wrote: post #88
jboy55 wrote: post #70

Earlier quoted context omitted.

I'm in full agreement. In practise, a manual QA team encourages Devs to throw shit over the wall and expect someone else to do some basic sanity checks they should have already done. By the time those are done, what the QA team theoretically could find gets shipped. Then, when its discovered in Prod, the QA team will get in the way of a speedy fix.

That's a management problem, not a problem with having a QA team. The top-level QA manager and the top-level Dev manager should both report to the CTO, and the Dev manager should be judged by how many issues the QA team finds. To keep things fair, the QA manager should not be rewarded nor penalized based on the number of issues found pre-release, and everyone should be rewarded or penalized based on the number of iss…

>and everyone should be rewarded or penalized based on the number of issues found post-release.

this sounds absolutely horrible. What a great way to remove all agency and ownership from an organization and incentivize people to pass the buck off to literally anyone but themselves and to work on easy, cupcake features with no risk.

>This keeps the devs incentivized to make sure everything works before the code goes to QA, and it keeps everyone incentivized to eliminate as many bugs as possible before release.

What kind of organization are you working in where you think devs have to be motivated by punishment or reward? Do you not hire the kind of people who are intrinsically motivated and want to produce software that doesn't have bugs? Do you think that if you don't explicitly incentivize developers not to write bugs that they would just fuck around and make shitty, buggy software? I just can't imagine working in an organization so dysfunctional that leadership thinks the default state is being happy with shitty buggy software.

Post reply on HN