Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yahoo’s Engineers Move to Coding Without a QA Team

spectrum.ieee.org

Post reply on HN
Print view View original

61–70 of 188 posts

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#61

Post by tajen »

pc86 wrote: post #44
ef4 wrote: post #29

Surprised to see the negativity here. I have worked in environments with traditional manual QA, and environments where all development is test-driven and nobody is allowed to merge a feature that lacks automated test coverage. Both the productivity and the quality were higher in the places with fully automated testing. Which is not shocking at all: does anybody really think a human can run through 800 test cases bett…

The arguments here such as "oh they just make the devs do it and don't pay them anymore" are ridiculous. Nobody's working 80 hour weeks doing two full-time jobs at once.

Please let me tip a name for that: "Volkswagening" - assigning the quality check to the production team.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#62

Post by chojeen »

I worked at Yahoo before and during this period, first as a QA contractor and then as a full-time developer.

Before the switch, our team (advertising pipeline on Hadoop) used the waterfall method with these gigantic, monolithic releases; we probably released a handful of times a year. Almost without exception, QA was done manually and was painfully slow. I started to automate a lot of the testing after I arrived, but believe you me when I say that it was a tall order.

Soon after I moved into development, QA engineers without coding chops were let go, while the others were integrated into the development teams. The team switched over to agile, and a lot of effort was made to automate testing wherever possible. Despite some initial setbacks, we got down to a bi-weekly release cycle with better quality control than before.

Around the time I left, the company was mandating continuous delivery for all teams, as well as moving from internal tools to industry-standard ones like Chef. I left before it was completed, but at least as far as the data pipeline teams were concerned, the whole endeavor made the job a lot more fun, increased release quality, and weeded out a lot of the "that's not my job" types that made life hell for everyone else.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#63

Post by DanielBMarkham »

Welcome Yahoo engineers to the year 2010. Or 2005. It's nice here.

The suckiest part of this story is the number of folks who are stuck with gated handoff processes that can't see how this would ever work. Some of those folks might be waiting 10, 20 years catching up to the other folks.

Just to be clear, QA the function isn't going anywhere. It's all being automated by the folks writing the code. QA the people/team? Turns out that this setup never worked well.

I work with tech organizations all the time. I find that poor tech organizations, when faced with a complex problem, give it to a person or team. Good organizations bulldoze their way through it the first time with all hands on board, then figure out how to offload as much of that manual BS as possible to computers. If they can't automate it, they live with it until they can. Same goes for complex cross-team integration processes.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#64

Post by devonkim »

This reminds me a bit about how SREs and developers conflict on contradictory goals. With a self regulating system that gets established between development and operations, if your code is bad in prod, you'll spend more time in operations to try to take care of the mistakes made in development. If SREs really don't get along with how developers throw over crap and quit, the developers will get the pagers instead. This move to consolidate test and development seems to be consistent with recent trends to pile upon more and more work for developers in the efforts to reduce siloization.

So I'd have to ask how getting rid of QA has affected the pace of feature development.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#65

Post by vvanders »

tajen wrote: post #61
pc86 wrote: post #44

Earlier quoted context omitted.

The arguments here such as "oh they just make the devs do it and don't pay them anymore" are ridiculous. Nobody's working 80 hour weeks doing two full-time jobs at once.

Please let me tip a name for that: "Volkswagening" - assigning the quality check to the production team.

Haha, I'm going to start using that term. That's a great description.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#66

Post by yborg »

dkopi wrote: post #20

This is actually a story about the triumph of continuous integration and staged rollout. By shipping code constantly, but slowly rolling it out to users - bugs can be detected very quickly by the users themselves, instead of employing large QA teams. Keeping a central code repository, automating builds, frequent commits and automatic tests for code are taking away a lot of load for QA teams.

This is actually a story about the triumph of ecosystem lock-in and training consumers to exchange high quality for low price or new shiny. It's outsourcing your testing to your customers. When you are either shipping free/freemium apps, or are a company that rigidly controls all APIs to their system (Twitter/Facebook), you can force the users to swallow any level of quality you choose to give them.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#67

Post by [deleted] »

arielweisberg wrote: post #37
ef4 wrote: post #29

Surprised to see the negativity here. I have worked in environments with traditional manual QA, and environments where all development is test-driven and nobody is allowed to merge a feature that lacks automated test coverage. Both the productivity and the quality were higher in the places with fully automated testing. Which is not shocking at all: does anybody really think a human can run through 800 test cases bett…

Moving from an environment with a 10:1 dev:qa to 2:1 showed me what happens when dev is not responsible for shipping working software. No thanks. It's a bunch of deflection and diffusion of responsibility coupled with high latency flakey interactions between different teams. Everything that can slip through the cracks does slip through the cracks. I'm sure QA can be done well, but I am convinced that giving your devs…

[deleted]

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#68

Post by alexlajeunesse »

dragonwriter wrote: post #24
minimaxir wrote: post #22

The stereotype that QA-is-pointless in Silicon Valley is persistent and actually annoying. There will always be issues that even the most comprehensive test suite will miss. Startups still glorify Facebook's "Move Fast and Break Things" without noting that Facebook has backpedaled from that. After all, people expect startup software to have issues, so what's the harm? Technical debt? Pfft. Engineers are not the best…

The QA function is not pointless. Whether a distinct QA team is the best means of performing the QA function is, however, a separate question.

Where I work teams are made up of a product designer, UX designer or two, devs, and QAs. The QAs are responsible for high level automated tests as well as manual testing for the parts that are hard to automate. It makes communication between devs/qa/design much faster as we all sit together.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#69

Post by IanCal »

philk10 wrote: post #60
duaneb wrote: post #48

Earlier quoted context omitted.

> Which is not shocking at all: does anybody really think a human can run through 800 test cases better than a computer can? I think this rather misses the point; it's the bug that doesn't have a test case where QA helps.

Exactly. And having the computer run through 800 checks means a skilled tester can do what they are good at - finding bugs.

This to me was always the point of automating tests. One of the best QA people I knew refused to look at the user stories. He was brilliant at finding things that fit the spec but didn't make sense and things that users might do that weren't specified.

He found bugs in the specs, gaps in the specs and just plain untested behaviour.

QA is a job that requires skill, benefits greatly from technical understanding and requires a lot of domain knowledge.

Re: Yahoo’s Engineers Move to Coding Without a QA Team

#70

Post by jboy55 »

arielweisberg wrote: post #37
ef4 wrote: post #29

Surprised to see the negativity here. I have worked in environments with traditional manual QA, and environments where all development is test-driven and nobody is allowed to merge a feature that lacks automated test coverage. Both the productivity and the quality were higher in the places with fully automated testing. Which is not shocking at all: does anybody really think a human can run through 800 test cases bett…

Moving from an environment with a 10:1 dev:qa to 2:1 showed me what happens when dev is not responsible for shipping working software. No thanks. It's a bunch of deflection and diffusion of responsibility coupled with high latency flakey interactions between different teams. Everything that can slip through the cracks does slip through the cracks. I'm sure QA can be done well, but I am convinced that giving your devs…

I'm in full agreement.

In practise, a manual QA team encourages Devs to throw shit over the wall and expect someone else to do some basic sanity checks they should have already done. By the time those are done, what the QA team theoretically could find gets shipped.

Then, when its discovered in Prod, the QA team will get in the way of a speedy fix.

Post reply on HN