Yahoo’s Engineers Move to Coding Without a QA Team • Hacker News Board

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#161

Post by TheAceOfHearts » Sat, Dec 12, 2015, 9:27 AM UTC

greydius wrote: post #139

eecks wrote: post #3

I can see why this would work in a place that uses QA as a crutch. All developers should aim for no bugs and test their stuff themselves but of course when deadlines are looming its easier to just code and let the QA team pick it up.

This was exactly the situation at a previous employer. There were twice as many testers as devs. I tried to advocate for comprehensive unit and integration tests and was met with a "if it doesn't work, the testers will let us know" attitude. Baffling. At my current job we have no QA testers and write much better software.

Comprehensive unit and integration tests strike me as a waste of time. From what I've seen, getting around 70%~80% code coverage is usually going to be around the tipping point where you start to get diminishing returns.

Don't get me wrong, I think testing is important. But there's tons of code where you get no value by writing tests for it. (At least in frontend development.)

We don't have a QA person, but I think it'd be great to have one. You can't write automated tests to check that things all look like they should. You can't write automated tests to check that all of the interactions are behaving as expected.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#162

Post by darkr » Sat, Dec 12, 2015, 9:55 AM UTC

jeswin wrote: post #151

diivio wrote: post #148

This isn't surprising. Microsoft switched to this model a few months after Satya took over. For the majority of Microsoft teams it worked really well and showed the kinds of results mentioned in this yahoo article. Look at many of our iOS apps as an example. But for some parts of the Windows OS team apparently it didn't work well (according to anonymous reports leaked online to major news outlets by some Windows team…

> I in no way want to sound like a dk but as an engineer it is your responsibility to practice test driven development but that's not enough. One of the things that put me off when it comes to TDD is that it has always been a bit like religion. What matters is whether the tests exist, not when they were written. I'd even argue that writing a test first and then being constrained by that box is a bad idea. Write the m…

> What matters is whether the tests exist, not when they were written

Technically true, but with myself at least; when I do TDD, I tend to write more, and better tests. When I write tests after code, especially when working on tight deadlines, there are substantially less tests written, just lots of TODOs that never get done.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#163

Post by stevoski » Sat, Dec 12, 2015, 10:01 AM UTC

chojeen wrote: post #62

I worked at Yahoo before and during this period, first as a QA contractor and then as a full-time developer. Before the switch, our team (advertising pipeline on Hadoop) used the waterfall method with these gigantic, monolithic releases; we probably released a handful of times a year. Almost without exception, QA was done manually and was painfully slow. I started to automate a lot of the testing after I arrived, but…

> the whole endeavor made the job a lot more fun

That is an important part of producing quality output in any job, I believe. The more employees actually enjoy what they are doing (or at least, don't actively hate it), the better their output is likely to be.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#164

Post by ljk » Sat, Dec 12, 2015, 10:05 AM UTC

diivio wrote: post #148

This isn't surprising. Microsoft switched to this model a few months after Satya took over. For the majority of Microsoft teams it worked really well and showed the kinds of results mentioned in this yahoo article. Look at many of our iOS apps as an example. But for some parts of the Windows OS team apparently it didn't work well (according to anonymous reports leaked online to major news outlets by some Windows team…

maybe a big company like microsoft can afford to not do as much of "qa tests" since they have people lining up wanting to be the beta tester for them(for free too!)

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#165

Post by datashovel » Sat, Dec 12, 2015, 10:45 AM UTC

In the past I've even loosely spec'd out a system that would build integration tests simply from crawling a website. In my head. I'm surprised this hasn't become a bigger priority from some of the biggest tech companies.

I know it would be a tough problem and a big project, but I think with only a small amount of human interaction you can build all the integration testing you would ever need simply by allowing the crawler to build them for you.

In fact the way I imagine it would work, the system would automatically build a framework and a user could (in a very structured way via structured UI) coerce the integration tests in small ways to ensure it understands what's going on. For example: "This form is used for registration". "This form is for logging in".

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#166

Post by rikkus » Sat, Dec 12, 2015, 11:24 AM UTC

fein wrote: post #105

DougWebb wrote: post #88

Earlier quoted context omitted.

That's a management problem, not a problem with having a QA team. The top-level QA manager and the top-level Dev manager should both report to the CTO, and the Dev manager should be judged by how many issues the QA team finds. To keep things fair, the QA manager should not be rewarded nor penalized based on the number of issues found pre-release, and everyone should be rewarded or penalized based on the number of iss…

Have you ever worked for a fortune 100 company? The dev manager has probably never met the CTO. Im in this right now with my currrent job. I still keep my builds and test them longer than i should. Less broken shit gets through, but i regularly delay my qa ba's because of it.

I work for a FTSE 100 company and the dev manager and CTO in the division where I work are very well connected. I don't think the size of company is necessarily the only factor here.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#167

Post by greydius » Sat, Dec 12, 2015, 11:48 AM UTC

TheAceOfHearts wrote: post #161

greydius wrote: post #139

Earlier quoted context omitted.

This was exactly the situation at a previous employer. There were twice as many testers as devs. I tried to advocate for comprehensive unit and integration tests and was met with a "if it doesn't work, the testers will let us know" attitude. Baffling. At my current job we have no QA testers and write much better software.

Comprehensive unit and integration tests strike me as a waste of time. From what I've seen, getting around 70%~80% code coverage is usually going to be around the tipping point where you start to get diminishing returns. Don't get me wrong, I think testing is important. But there's tons of code where you get no value by writing tests for it. (At least in frontend development.) We don't have a QA person, but I think i…

Point taken. I've never done frontend work, but I can understand the value of a human tester when a complex UI is involved. The kind of work I do is very data-oriented and quite easy to get high coverage when the code is well written.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#168

Post by zhte415 » Sat, Dec 12, 2015, 12:22 PM UTC

DougWebb wrote: post #109

fein wrote: post #105

Earlier quoted context omitted.

Have you ever worked for a fortune 100 company? The dev manager has probably never met the CTO. Im in this right now with my currrent job. I still keep my builds and test them longer than i should. Less broken shit gets through, but i regularly delay my qa ba's because of it.

So, not the CTO, but some shared higher-level manager. The point is that the QA lead cannot report into Development, and Development cannot report into QA. Both of those arrangements can lead to a bad actor stopping the other group from doing the right thing. (Eg: if QA reports to Dev, then Dev manager can force bad code to ship, and if Dev reports to QA than QA manager can prevent shipping and force Dev to waste tim…

I've met people that call something similar 'fake agile'.

Surely everyone should report into the business (end-user on the business side) lead responsible for making this functionality happen. This includes Dev, QA, business. QA should be business? This is no longer 1985, there is no reason business (product owner) shouldn't be a capable PM and BA for their own product.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#169

Post by Kabacaru » Sat, Dec 12, 2015, 12:26 PM UTC

Manual testing is basically a 0 skill job. Can you click around this website and tell me when you see a bug. This is the most common form of QA but adds very little value that can't be added with more reliability using automation.

Given this QA can still bring value. The two roles that they really add value in are a Test developer specialist writing non-flaky automated tests, and a BA type role where they have conversations that expand a product owner's idea into an implementable feature.

Given that neither of these roles require manual testing, if a QA team has over specialised on manual testing, there's little value in keeping it.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#170

Post by jaggederest » Sat, Dec 12, 2015, 12:27 PM UTC

mstade wrote: post #39

I don't think it's that cut and dry. I've worked in some places where the QA team was useless, meaningless red tape to get your stuff deployed. They wouldn't do much but sign off on deployment at some point , yet bore no responsibility if shit hit the fan. In these cases, they really were just an unnecessary cost and you learned pretty quickly to make sure your tests were in place, that you were testing for the right…

I think the key difference is an engineering mindset.

When you have software developers skilled in quality assurance who have the job of finding the edge cases and producing comprehensive additional acceptance and functional testing, they're an asset. It's a particular perverse mindset that I personally really enjoy interacting with as a software developer 'customer' - those evil bastards find the best bugs, regression test them, and expand to find all of that class of error in the application.