11–20 of 96 posts
I do buy the argument that the trade off between effort and value is different, but that's because it's harder to unit test user interactions than it is to unit test a physics engine.
It's more or less the reason in the early life of the web few did end to end testing involving browsers, or unit tested iOS apps in the first releases of the iPhone.
Earlier quoted context omitted.
> Also, remember that games are not very long-lived pieces of software. You build it, release it, maybe patch it, and move on. This was true a couple decades ago. Nowadays many games are cash cows for decades. Path of Exile was released in 2013, Minecraft in 2011, and World of Warcraft in 2004, and all of those continue to receive regular updates (and have over the course of their lives) and still make plenty of mone…
I maintain that the majority of games are still disposable, despite the occasional subscription model or long-lived hit that pops up. Remember that most games aren't made by AAA studios. Wasn't Minecraft completely rewritten from scratch in Java after a few years? And the EA one, like you said, it's just model updates. Very few gameplay mechanics get more than a simple tweak. Just recompile with the new models. You d…
All product development involves poorly defined boundaries where the product meets the user, where requirements shift frequently, and where the burdens of test maintenance have to be weighed against the benefits.
You don’t throw out all of unit testing because it doesn’t work well for a subset of your code. You throw out all of unit testing because writing tests is annoying, none of your coworkers have set it up, and the rest of your industry doesn’t do it, so you feel justified in not doing it either.
I don't buy this argument. Most game developers I know have said that unit tests are a waste of time so they never use them, but they're struggling with making changes to utility code and making sure that it doesn't do the wrong thing. Y'know, what unit tests are for. I think the key here is that the perceived cost / benefit ratio is too high. It's the perception that drives their behavior though. I'm in a company no…
I am pretty sure there is some sort of automated testing happening that is catching these bugs before release.
Earlier quoted context omitted.
We produce a library that gets included in software made by our clients, and we have several thousand clients. The uptake on new releases is low (most of the clients believe in "if it ain't broke, don't fix it"). So every release has the potential to live in the wild and need support for a long time. We're also in an industry with a ton of competitors. On top of that, the company was founded by some very junior engin…
Thank you for the write up. That seems like a bad scenario with bad technical management. I am wondering if you have considered not trying to implement unit tests and think about end to end tests. This might be easier for antitesting people to buy into because it’s directly ensuring your end users get the desired outcomes. It doesn’t matter what bad terrible practices you have inside your library if the output is cor…
Earlier quoted context omitted.
I am curious as to why your current company does not have unit tests. Do you mind sharing?
We produce a library that gets included in software made by our clients, and we have several thousand clients. The uptake on new releases is low (most of the clients believe in "if it ain't broke, don't fix it"). So every release has the potential to live in the wild and need support for a long time. We're also in an industry with a ton of competitors. On top of that, the company was founded by some very junior engin…
Earlier quoted context omitted.
Thank you for the write up. That seems like a bad scenario with bad technical management. I am wondering if you have considered not trying to implement unit tests and think about end to end tests. This might be easier for antitesting people to buy into because it’s directly ensuring your end users get the desired outcomes. It doesn’t matter what bad terrible practices you have inside your library if the output is cor…
As a corollary to 2, management tends to love graphs… whatever your using to build should have a plugin that could show unit test success counts and generate even a simple line graph… that alone might be enough incentive to add more testing
Edit; in fact, don’t say test at all. Talk about verification of the output
Having said that, sometimes you need unit tests. Overwatch had this bug where there is an ultimate ability called "amplification matrix" that is a window that you shoot through and the bullets do twice as much damage. One patch, that stopped working. This kind of issue is pretty easy to miss in play testing; if you're hitting headshots, then the bullets are doing the 2x damage they would if they were body shots that got properly amplified. If is very hard to tell damage numbers while play testing (as evidenced by how many patches are "we made character X do 1 more damage per bullet", and it smooths things out over the scale of millions of matches, but isn't really that noticeable to players unless breakpoints change). So for this reason, write an integration test where you set up this window thingie, put an enemy behind it, fire a bullet at a known point, and require.Equals(damage, 200). Ya just do it, so you don't ship the bug, make real people lose real MMR, and then have to "git stash" that cool thing you're working on today, check out the release branch, and uncomment the code that makes the amp matrix actually work. Games are just software engineering. Fun software engineering. But it's the same shit that your business logic brothers and sisters are working on.
(Overwatch also had a really neat bug, that the community believes was due to a x == 0 check instead of an x Follow up follow up: all of these silly bugs are in ultimates, which come up the least often of all abilities in the games. That's what happens with playtesting. You don't get test coverage where you need it. A test you write covers the stuff you're most scared about. A careful engineer that likes testing would have never shipped these.)
Earlier quoted context omitted.
We produce a library that gets included in software made by our clients, and we have several thousand clients. The uptake on new releases is low (most of the clients believe in "if it ain't broke, don't fix it"). So every release has the potential to live in the wild and need support for a long time. We're also in an industry with a ton of competitors. On top of that, the company was founded by some very junior engin…
How do you stay sane working with clowns?
In any games company I've worked for the designer is responsible for mapping and balancing the rules and mechanics of the game, they would provide a specification of what "red vs blue numbers" would look like and a balanced idea of how to remove the number 5 from the game (balancing and changing the rules like this being entirely within the domain of game design). incidentally any game company I've worked at has had an extensive set of test suites.