Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Automated Testing for League of Legends

engineering.riotgames.com

Post reply on HN
Print view View original

21–30 of 78 posts

Re: Automated Testing for League of Legends

#21

Post by huac »

derFunk wrote: post #12

Wow, this is pretty sophisticated and commendable. I'd love to have the resources to do something similar. In contrast to 'standard' application development, automated testing is really rare in the non-AAA games industry. At least in terms of logic-/active gameplay testing. It's pure luxury, you can only do it if you can afford it. In a project-based work for hire game shop this is an almost unthinkable thing to do,…

On the other hand, just because you have a great testing framework, it doesn't mean you should launch new game features/changes left and right.

(LoL is structured around roughly yearly 'seasons' for competitive play and for ranked play rewards. This season they have released a far greater amount of game mechanic changes, which make it difficult to plan competitive strategy and difficult for regular players to keep up.)

Re: Automated Testing for League of Legends

#22

Post by JamieAli »

Honestly I don’t know why some people look at these pictures, especially when they could be getting better at the game. No wonder 90% of people stay at BRONZE level while the big boys-including me, rank up and get DIAMOND level.

It wasn't hard once I started using this: http://gamesinfun.com/league-of-legends

Re: Automated Testing for League of Legends

#25

Post by errantspark »

dylz wrote: post #24
mangix wrote: post #23

too bad dota 2 is a better game

really, this stuff goes on even on HN?

Maybe we extend the benefit of the doubt and perceive the comment as a reminder that having code that passes all the tests doesn't necessarily equate to having a good product?

It's obviously a stretch as that guy's username is the name of a DotA hero, but hey, rose colored glasses are fun sometimes.

Re: Automated Testing for League of Legends

#26

Post by 0x0 »

finishingmove wrote: post #20
swanson wrote: post #15

Earlier quoted context omitted.

I tend to agree, but sometimes in these kind of instrumented test environments, the setup cost of each test case might be very high so maybe the team found it reasonable to group all the assertions into a single test (especially given that the suite is already taking 1-2 hours to give feedback to developers).

I am well aware of that and I agree on being pragmatic, but more often than not, the reason for multiple assertions per test is developer laziness and/or unawareness of the benefits of isolating failures by having just one assertion per test.

Seems to me you actually want all three assertions checked in the same run. Otherwise you could get three runs where only the first assertion was true in the first run; only the second assertion succeeding in the second run with the others failing; etc.

I bet the tests aren't pixel-by-pixel frame-by-frame reproducible (they are talking about a one week staging period for tests to prove themselves, so surely there's some non-deterministic wiggle room in remote controlling the full graphical client), so you really want to cover all assertions together for each run.

Re: Automated Testing for League of Legends

#27

Post by d0100 »

finishingmove wrote: post #10

We see the test checks three things: Verifies: - KogMaw deals less damage to non-lane minions - KogMaw deals percentile magic damage - KogMaw deals normal damage to lane minions and the verify method has three assertions. In my opinion, this should be three tests, each with only one assertion.

They'd soon be drowning in individual tests. And it wouldn't necessarily fit to separate them, considering all those 3 have to be tested on each skill usage. One shot has to obey those 3 rules.

Making each an individual test would triple the test time, pollute the test codebase and not bring any advantage.

Re: Automated Testing for League of Legends

#28

Post by dimino »

swanson wrote: post #4

A great anecdote in the 'we can't test this code' discussion -- for an aggressive two-week release cycle of this massively popular game, this level of testing has probably paid for itself many times over.

In this case? Probably. But the idea that catching a bug in production is a huge problem is a myth, pushed by research that was later completely withdrawn.

Re: Automated Testing for League of Legends

#29

Post by dimino »

finishingmove wrote: post #10

We see the test checks three things: Verifies: - KogMaw deals less damage to non-lane minions - KogMaw deals percentile magic damage - KogMaw deals normal damage to lane minions and the verify method has three assertions. In my opinion, this should be three tests, each with only one assertion.

The code itself isn't super duper pythonic, which is not ideal, but definitely the right direction for a project like this to move.

It's just hard sometimes to set and follow standards when you're pushing to demonstrate value. Not impossible, but hard.

Re: Automated Testing for League of Legends

#30

Post by debaserab2 »

dimino wrote: post #28
swanson wrote: post #4

A great anecdote in the 'we can't test this code' discussion -- for an aggressive two-week release cycle of this massively popular game, this level of testing has probably paid for itself many times over.

In this case? Probably. But the idea that catching a bug in production is a huge problem is a myth, pushed by research that was later completely withdrawn.

When customers discover bugs before you do, it reflects poorly on your reputation and harms the trust you have established with the consumer of your application. That's not a myth.
Post reply on HN