Live data from Hacker News

Addition with flamethrowers – why game devs don't unit test

pixelatedplaygrounds.com

41–50 of 96 posts

Re: Addition with flamethrowers – why game devs don't unit test

#41
post #34
post #2

I don't buy this argument. Most game developers I know have said that unit tests are a waste of time so they never use them, but they're struggling with making changes to utility code and making sure that it doesn't do the wrong thing. Y'know, what unit tests are for. I think the key here is that the perceived cost / benefit ratio is too high. It's the perception that drives their behavior though. I'm in a company no…

I’ve seen people slog through untested code where they fear to make a change but I’ve also seen people slog through code with too much test coverage where the tests go through constant churn. I don’t understand why people don’t just add one test even if the codebase otherwise has zero tests if they’re so scared of one area and I don’t get why people keep adding excessive coverage if it’s wasting their time. It’s like…

This is the way. My work codebase has probably 5% unit test coverage -- it's frontend and a lot of it isn't sensible to unit test -- but I'm quite happy to have the tests we do. If it's nontrivial logic, just test it. If it isn't (it's trivial, it's aesthetic, whatever your reason)... just don't.

Re: Addition with flamethrowers – why game devs don't unit test

#42
I wrote a game using a 'bottom up' design (i.e. IO and business logic first), and I wrote unit tests for the business logic as I went. With no UI, I effectively tested and stepped through my code with unit tests. I had the luxury of working by myself at my own pace.

I have a reasonably clean separation between the UI and the rest of the code, but I don't have any unit tests for the UI (I think - correct me if I'm wrong here - that would require integration tests rather than unit tests?) What I'm trying to say is that, if you don't do it this way around, and/or you have multiple programmers writing the game at once, and/or you _really_ optimise for performance, I can imagine that would make it much harder to write unit tests.

Re: Addition with flamethrowers – why game devs don't unit test

#43
post #6
post #4

Earlier quoted context omitted.

> Also, remember that games are not very long-lived pieces of software. You build it, release it, maybe patch it, and move on. This was true a couple decades ago. Nowadays many games are cash cows for decades. Path of Exile was released in 2013, Minecraft in 2011, and World of Warcraft in 2004, and all of those continue to receive regular updates (and have over the course of their lives) and still make plenty of mone…

I maintain that the majority of games are still disposable, despite the occasional subscription model or long-lived hit that pops up. Remember that most games aren't made by AAA studios. Wasn't Minecraft completely rewritten from scratch in Java after a few years? And the EA one, like you said, it's just model updates. Very few gameplay mechanics get more than a simple tweak. Just recompile with the new models. You d…

the original minecraft is in java, it's probably gone through a lot of code transformation. The version you're thinking of is the microsoft version, rewritten in c++

Re: Addition with flamethrowers – why game devs don't unit test

#44
"It's never good to be dogmatic."

In some situations unit tests can be very effective and useful, such as in testing complex algorithms, or in code bases where some serious refactoring is required, and where one don't want to break existing behavior. In backend development, where user facing output is limited, there is typically no other practical way to check that things are working properly.

However, in games, and typical front-end development, especially in its early stages, it can be beneficial to be as flexible as possible. And however way you put it, unit tests simply make your code more rigid.

In the latter situation, some people prefer guard rails and find that they are more flexible with unit tests in place. Others prefer not to care about unit tests and attain higher productivity without them.

Only when an application grows to a certain size where a developer does not naturally inspect typical behavior all day, and if quality is important, it starts to make sense to put in automated testing, because it is simply more cost effective.

Similar reasoning goes for dynamic vs static typing.

It seems that some people think that everyone should always use the same approach for any kind of software development, because it worked for them at some point in time. Over time I have grown a preference to avoid working with such people.

Re: Addition with flamethrowers – why game devs don't unit test

#45
For an alternative perspective on testing & game development, here's a video I've seen from few years ago:

* "Automated Testing of Gameplay Features in 'Sea of Thieves'": https://www.youtube.com/embed/X673tOi8pU8?si=uj_lcMEC9nvMpa6...

~via https://www.gdcvault.com/play/1026366/Automated-Testing-of-G... :

"Automated testing of gameplay features has traditionally not been embraced by the industry, due to the perceived time required and difficulty in creating reliable tests. Sea of Thieves however was created with automated testing for gameplay from the start. This session explains why automated testing was the right choice for Sea of Thieves and how it could benefit your game. It shows the framework that was built by Rare to let team members create automated tests quickly and easily, the different test types we built, and what level of test coverage was found to be appropriate. The session also contains best practices for making tests work reliably and efficiently, using clear worked through examples."

Looks like there's also related talks in later years (which may or may not be currently available as free-to-view--I've not watched these ones):

* "Lessons Learned in Adapting the 'Sea of Thieves' Automated Testing Methodology to 'Minecraft'": https://www.gdcvault.com/play/1027345/Lessons-Learned-in-Ada...

* "Automated Testing of Shader Code" (GDC 2024): https://schedule.gdconf.com/session/automated-testing-of-sha...

Re: Addition with flamethrowers – why game devs don't unit test

#46
post #26

Most video game bugs are subtle and not things that are easy to catch with unit testing because they are dynamic systems with many interacting parts. The interaction is where the bugs come from. QA processes do a good job catching the rest.

I would bet that a lot of those bugs come from utility code that is testable

Perhaps in development, but the stuff that tends to make it into the release of games seems to be gameplay related. Npc behaviors not lining up, the developer literally not implementing certain stats in the game (looking at you, Diablo 4), graphical bugs caused by something not loading or loading too slowly, performance issues from something loading 1000 copies of itself etc.

Re: Addition with flamethrowers – why game devs don't unit test

#47
This makes me think of the claim you sometimes see that memory-safety is not that relevant for game development because in many cases games aren't security-sensitive software. But even putting security vulnerabilities aside completely, plain old memory corruption can be a major drag when it rears its head (and can even kill projects if the game can't be wrangled into being crash-free by the deadline). This particularly applies to games with huge codebases and numbers of programmers.

Re: Addition with flamethrowers – why game devs don't unit test

#48
post #2

I don't buy this argument. Most game developers I know have said that unit tests are a waste of time so they never use them, but they're struggling with making changes to utility code and making sure that it doesn't do the wrong thing. Y'know, what unit tests are for. I think the key here is that the perceived cost / benefit ratio is too high. It's the perception that drives their behavior though. I'm in a company no…

Testing is a continuum. I don't write a test for every change. Sometimes I spend a week writing tests for a simple change.

I will say that I've never said "I wish I didn't write a test for that". I have also never said, "your PR is fine, but please delete that test, it's useless".

I throw away a lot of code. I still test stuff I expect to throw away. That's because it probably needs to run once before I throw it away, and I can't start throwing it away until it works :/

What it comes down to is what else you have to spend your time on. Sometimes you need to experiment with a feature; get it out to customers, and if it's buggy and rough around the edges, it's OK, because you were just trying out the idea. But sometimes that's not what you want; whatever time you spend on support back and forth finding a bug would have been better spent not doing that. The customer needed something rock solid, not an experiment. Test that so they don't have to.

There are no rules. "Write a test for every change" is just as invalid and unworkable as "Never write any tests". It's a spectrum, and each change is going to land somewhere different. If you're unsure, ask a coworker. I have been testing stuff for 20+ years, and I usually guess OK (that is when I take a shortcut and don't test as much as I should, it's rarely the thing that caused the production outage), but a guess is just that, a guess. Solicit opinions.

Re: Addition with flamethrowers – why game devs don't unit test

#49

Bugs are kind of the fun part of games. If every subroutine worked perfectly, you wouldn't have the chaos of real life. Some of players favorite mechanics are just bugs. (Overwatch example: Mercy's super jump, now a legitimate predictable mechanic that everyone can do, not just people that read the forums and watch YouTube videos about bugs. It started out as a bug, and it was so cool and skill-ceiling increasing tha…

> Games are just software engineering. Fun software engineering.

I do question the "fun" part. Midnight crunches, unpaid overtime and - as far as I have read - some of the worst working conditions in all of software engineering. I pass.

Re: Addition with flamethrowers – why game devs don't unit test

#50
post #2

I don't buy this argument. Most game developers I know have said that unit tests are a waste of time so they never use them, but they're struggling with making changes to utility code and making sure that it doesn't do the wrong thing. Y'know, what unit tests are for. I think the key here is that the perceived cost / benefit ratio is too high. It's the perception that drives their behavior though. I'm in a company no…

Also, non-testable code is often faster (as in cpu time).
Post reply on HN