Live data from Hacker News

Automated Testing for League of Legends

engineering.riotgames.com

51–60 of 78 posts

Re: Automated Testing for League of Legends

#51
post #50
post #48

Earlier quoted context omitted.

> Find me a business owner who thinks major bugs in production are not "a huge problem". "Move fast and break things" - Mark Zuckerberg.

They moved away from that, didn't they?

Indeed. Quote from Mark:

>> We used to have this famous mantra... and the idea here is that as developers, moving quickly is so important that we were even willing to tolerate a few bugs in order to do it. What we realized over time is that it wasn't helping us to move faster because we had to slow down to fix these bugs and it wasn't improving our speed.

Re: Automated Testing for League of Legends

#52
post #49

Earlier quoted context omitted.

It definitely has. League of Legends is considered a serious e-sport game. Individual tournaments for League of Legends have $2,000,000+ prize pools. If a new bug decided a round of the tournament, it would undermine LOL as a serious contender for this type of attention. There is a terrific business case for investing the time to make this game as stable as possible.

This has actually happened before at official League of Legends tournaments (that is, a fairly serious bug in a serious match). I believe their policy in the case of a serious bug is to re-do the match, but I am not 100% sure.

It depends. If the bug was caught before game of record, you just restart the game. Afterwards, it must qualify as a game deciding bug. If so, the game gets scrapped and the match is redone.

Re: Automated Testing for League of Legends

#53
post #27

Earlier quoted context omitted.

They'd soon be drowning in individual tests. And it wouldn't necessarily fit to separate them, considering all those 3 have to be tested on each skill usage. One shot has to obey those 3 rules. Making each an individual test would triple the test time, pollute the test codebase and not bring any advantage.

The advantage in splitting up the assertions is that you get more visibility. If one test fails and not the other two that's extra information you can use to debug you wouldn't have if you have all 3 in one test.

If that test fails, the dev will just run it locally with a debugger attached. The cost of extra time in the test for the normal passing case isn't worth it.

Re: Automated Testing for League of Legends

#55
post #50

Earlier quoted context omitted.

They moved away from that, didn't they?

Indeed. Quote from Mark: >> We used to have this famous mantra... and the idea here is that as developers, moving quickly is so important that we were even willing to tolerate a few bugs in order to do it. What we realized over time is that it wasn't helping us to move faster because we had to slow down to fix these bugs and it wasn't improving our speed.

They grew as a company to a place where they could slow down.

Re: Automated Testing for League of Legends

#57
post #26

Earlier quoted context omitted.

I am well aware of that and I agree on being pragmatic, but more often than not, the reason for multiple assertions per test is developer laziness and/or unawareness of the benefits of isolating failures by having just one assertion per test.

Seems to me you actually want all three assertions checked in the same run. Otherwise you could get three runs where only the first assertion was true in the first run; only the second assertion succeeding in the second run with the others failing; etc. I bet the tests aren't pixel-by-pixel frame-by-frame reproducible (they are talking about a one week staging period for tests to prove themselves, so surely there's s…

They should be tested independently, each with a completely reproducible setup that does only what is required for the assertion that will be made. I'm not sure what's hard to understand here... Are you trying to say that they need to be run together? Because it doesn't seem like that to me.

Re: Automated Testing for League of Legends

#58
post #27

We see the test checks three things: Verifies: - KogMaw deals less damage to non-lane minions - KogMaw deals percentile magic damage - KogMaw deals normal damage to lane minions and the verify method has three assertions. In my opinion, this should be three tests, each with only one assertion.

They'd soon be drowning in individual tests. And it wouldn't necessarily fit to separate them, considering all those 3 have to be tested on each skill usage. One shot has to obey those 3 rules. Making each an individual test would triple the test time, pollute the test codebase and not bring any advantage.

> Making each an individual test would triple the test time, pollute the test codebase and not bring any advantage.

Would it really? None of what you said here is true.

Re: Automated Testing for League of Legends

#59
post #43
post #23

too bad dota 2 is a better game

Funnily enough, the latest patch was rushed out by Valve, and it contained a lot of bugs and strange behavior. Lots of bugs in Dota 2 seems to be happening because of manual testing.

Not to say that DotA wouldn't benefit from more rigorous testing, but in Valve's defense I'd say that the complexity of interactions in DotA is far beyond LoL. Some of the bugs you're seeing might only arise in extremely rare situations.

DotA is balanced with creativity in mind. When an exploitative strategy is found the game is not changed to remove it but instead through some combination of tweaks and modifications, often to seemingly unrelated aspects it's made into something that's only situationally viable.

The reason the game is so deep/interesting is because this philosophy has been in place for many (10+) years now resulting in a game that's much more complex than pretty much anything else out there. DotA 2 retains edge case behavior that was present in the original Warcraft 3 mod. Things that were once engine limitations are now part essential parts of gameplay. Preserving these weird interactions instead of attempting to make the game more regular/comprehensible is one of the things that allows such a high level of creativity in competitive play. There's always a way you can outplay your opponent that's clever. In go parlance: there's always new tesuji to be found.

All that being said I think Riot's efforts here are quite laudable and DotA could certainly find benefit from more rigorous testing.

Re: Automated Testing for League of Legends

#60
post #56

Earlier quoted context omitted.

Lol I love the haters.

i dunno i keep reading on the dota 2 subreddit how riot is ruining LoL and new players asking for advice.

Ah, a well-informed opinion then.

None of your comments so far have contributed anything to this discussion. Please try to keep to the spirit of HN when commenting.

Post reply on HN