Yahoo’s Engineers Move to Coding Without a QA Team • Hacker News Board

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#171

Post by room271 » Sat, Dec 12, 2015, 1:02 PM UTC

DougWebb wrote: post #88

jboy55 wrote: post #70

Earlier quoted context omitted.

I'm in full agreement. In practise, a manual QA team encourages Devs to throw shit over the wall and expect someone else to do some basic sanity checks they should have already done. By the time those are done, what the QA team theoretically could find gets shipped. Then, when its discovered in Prod, the QA team will get in the way of a speedy fix.

That's a management problem, not a problem with having a QA team. The top-level QA manager and the top-level Dev manager should both report to the CTO, and the Dev manager should be judged by how many issues the QA team finds. To keep things fair, the QA manager should not be rewarded nor penalized based on the number of issues found pre-release, and everyone should be rewarded or penalized based on the number of iss…

That sounds like a terrible strategy for a lot of us, where time to fix is much more important than bug count.

This is because there is likely to be a tradeoff between speedy development and bug count.

If you are shipping physical CDs of software, or releases which you can't update easily, or if you are working in a critical environment where mistakes are disastrous (finance, health, space, etc.), then it is fair enough to be so concerned about bugs.

But many if not most developers work on the web in areas where the product is uncertain and evolving. Prioritising rate of change is more important here. Rather than extensive QA, monitoring and fast rollback are better choices.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#172

Post by philk10 » Sat, Dec 12, 2015, 2:16 PM UTC

grandalf wrote: post #5

Yahoo's main issue is UX and usability, not software quality, so this sounds like another way of saying Yahoo laid off its QA team.

Depends on your definition of 'quality' - if s/w isn't usable and has bad UC then how can it be high quality?

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#173

Post by philk10 » Sat, Dec 12, 2015, 2:20 PM UTC

Kabacaru wrote: post #169

Manual testing is basically a 0 skill job. Can you click around this website and tell me when you see a bug. This is the most common form of QA but adds very little value that can't be added with more reliability using automation. Given this QA can still bring value. The two roles that they really add value in are a Test developer specialist writing non-flaky automated tests, and a BA type role where they have conver…

I might be biased ( ex-dev with 20 years experience from Assembler to C to .Net etc ) but GOOD manual testing requires a lot of skill. If it didn't then I wouldn't have switched from dev to test. I work in a shop where they had devs doing all the testing, tons of automation but they still found that a good exploratory tester added value. But from your comment it may well be that you've never worked with someone like me - maybe when you do you'll think different

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#174

Post by DougWebb » Sat, Dec 12, 2015, 3:27 PM UTC

room271 wrote: post #171

DougWebb wrote: post #88

Earlier quoted context omitted.

That's a management problem, not a problem with having a QA team. The top-level QA manager and the top-level Dev manager should both report to the CTO, and the Dev manager should be judged by how many issues the QA team finds. To keep things fair, the QA manager should not be rewarded nor penalized based on the number of issues found pre-release, and everyone should be rewarded or penalized based on the number of iss…

That sounds like a terrible strategy for a lot of us, where time to fix is much more important than bug count. This is because there is likely to be a tradeoff between speedy development and bug count. If you are shipping physical CDs of software, or releases which you can't update easily, or if you are working in a critical environment where mistakes are disastrous (finance, health, space, etc.), then it is fair eno…

A formal QA process, which goes along with a formal release process and release schedule, certainly does slow things down. I think for large-scale projects with a large feature/requirement count that's ok, because there's a lot of little details to keep track of and the formal process helps with that. But you also need a fast-track for fixing small things quickly, and for that Dev, QA, and Ops need to work together to quickly diagnose, fix, test, and release changes.

My experience with the formal process I've been talking about was in a company whose product is a large-scale web application that interacted with a complex custom back end search engine and a proprietary content database. The QA and Dev teams had a friendly adversarial relationship, but ultimately we all felt we were on the same team with the same goals: to produce the best software we could. It was a fantastic place to work.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#175

Post by DougWebb » Sat, Dec 12, 2015, 3:37 PM UTC

zhte415 wrote: post #168

DougWebb wrote: post #109

Earlier quoted context omitted.

So, not the CTO, but some shared higher-level manager. The point is that the QA lead cannot report into Development, and Development cannot report into QA. Both of those arrangements can lead to a bad actor stopping the other group from doing the right thing. (Eg: if QA reports to Dev, then Dev manager can force bad code to ship, and if Dev reports to QA than QA manager can prevent shipping and force Dev to waste tim…

I've met people that call something similar 'fake agile'. Surely everyone should report into the business (end-user on the business side) lead responsible for making this functionality happen. This includes Dev, QA, business. QA should be business? This is no longer 1985, there is no reason business (product owner) shouldn't be a capable PM and BA for their own product.

In the company I used to work for, the CTO and VP of Product Management were equals, both reporting to the CEO. The CTO's reports included the QA director, Dev director, and Project Management director.

On any given project, we had a Product Manager who set the business requirements, a Project Manager who ran the project, a Dev lead in charge of the development team, and a QA lead in charge of QA. None of these people could override the others because of the reporting arrangement, so we had to collaborate instead to make sure we all met our goals.

Early on this arrangement worked really well. But it started falling apart when ownership of the company changed and politics became a factor as people who cared more about moving up through the organization than about the success of the company and quality of the products were hired and promoted. My take is that bad management will destroy any approach you might take to producing good software. That's why I eventually left.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#176

Post by otterley » Sat, Dec 12, 2015, 4:31 PM UTC

minimaxir wrote: post #22

The stereotype that QA-is-pointless in Silicon Valley is persistent and actually annoying. There will always be issues that even the most comprehensive test suite will miss. Startups still glorify Facebook's "Move Fast and Break Things" without noting that Facebook has backpedaled from that. After all, people expect startup software to have issues, so what's the harm? Technical debt? Pfft. Engineers are not the best…

You can be sure that any system at Yahoo! or anywhere else that is the bread and butter of the operation (ad systems, accounting, fraud tracking) still involves a software QA team.

Facebook could afford to "move fast and break things" when they weren't taking money from anyone. Now that they're a real company, that kind of attitude can get them adverse attention from the SEC if it causes an accounting discrepancy or from trial lawyers if it upsets their advertisers.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#177

Post by kumarski » Sat, Dec 12, 2015, 4:36 PM UTC

otterley wrote: post #176

minimaxir wrote: post #22

The stereotype that QA-is-pointless in Silicon Valley is persistent and actually annoying. There will always be issues that even the most comprehensive test suite will miss. Startups still glorify Facebook's "Move Fast and Break Things" without noting that Facebook has backpedaled from that. After all, people expect startup software to have issues, so what's the harm? Technical debt? Pfft. Engineers are not the best…

You can be sure that any system at Yahoo! or anywhere else that is the bread and butter of the operation (ad systems, accounting, fraud tracking) still involves a software QA team. Facebook could afford to "move fast and break things" when they weren't taking money from anyone. Now that they're a real company, that kind of attitude can get them adverse attention from the SEC if it causes an accounting discrepancy or…

Facebook has an amazing 'dogfooding their own product' scenario that's hard to replicate for any other company.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#178

Post by gsnedders » Sat, Dec 12, 2015, 6:13 PM UTC

philk10 wrote: post #173

Kabacaru wrote: post #169

Manual testing is basically a 0 skill job. Can you click around this website and tell me when you see a bug. This is the most common form of QA but adds very little value that can't be added with more reliability using automation. Given this QA can still bring value. The two roles that they really add value in are a Test developer specialist writing non-flaky automated tests, and a BA type role where they have conver…

I might be biased ( ex-dev with 20 years experience from Assembler to C to .Net etc ) but GOOD manual testing requires a lot of skill. If it didn't then I wouldn't have switched from dev to test. I work in a shop where they had devs doing all the testing, tons of automation but they still found that a good exploratory tester added value. But from your comment it may well be that you've never worked with someone like…

I think a lot depends on specific definitions: a manual test suite, where you have a list of tests with clear steps and a clear expectation, is definitely near-zero skill to execute. Actual exploratory testing, on the other hand, is skilled: especially if they're expected to write up tests (automated or not) that test code paths that haven't previously been tested.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#179

Post by knughit » Sat, Dec 12, 2015, 6:31 PM UTC

DougWebb wrote: post #110

dragonwriter wrote: post #107

Earlier quoted context omitted.

> The top-level QA manager and the top-level Dev manager should both report to the CTO For an organization with a single product (and, particularly, one small enough that the "top-level" QA and Dev manager are also first level managers), this makes some sense. Otherwise, this means that each product has no common level of technical management below the CTO, meaning it has no meaningful technical ownership.

It doesn't have to be the CTO; as I mentioned in another reply my intent was that QA and Dev need to be equals in the reporting chain, rather than one reporting to the other. That can work at any scale; even on an agile team with one QA and one Dev, they both report to the same boss who's responsible for both sides.

Of you are fine tuning the reporting chain, your org is already hopelessly broken. That is the central planning Communist model.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#180

Post by knughit » Sat, Dec 12, 2015, 7:06 PM UTC

Is it odd that this article describes, but highlight , that yahoo is a decade behind the industry here? Continuous integration and skipping QA has been the web standard for years now.