Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Addition with flamethrowers – why game devs don't unit test

pixelatedplaygrounds.com

Post reply on HN
Print view View original

21–30 of 96 posts

Re: Addition with flamethrowers – why game devs don't unit test

#21

Post by stouset »

mistercow wrote: post #13

I’ve found that the one thing you can always count on engineers to do is to dismiss sensible tools from adjacent domains using flimsy, post hoc justifications. All product development involves poorly defined boundaries where the product meets the user, where requirements shift frequently, and where the burdens of test maintenance have to be weighed against the benefits. You don’t throw out all of unit testing because…

Right. And because the rest of the industry isn’t doing it, there’s no institutional knowledge of how to do it well. So someone tries it, they do a crap job of it out of understandable ignorance, and rather than taking forward any lessons learned the effort is discarded as a waste of time.

Re: Addition with flamethrowers – why game devs don't unit test

#23

Post by jayd16 »

mistercow wrote: post #13

I’ve found that the one thing you can always count on engineers to do is to dismiss sensible tools from adjacent domains using flimsy, post hoc justifications. All product development involves poorly defined boundaries where the product meets the user, where requirements shift frequently, and where the burdens of test maintenance have to be weighed against the benefits. You don’t throw out all of unit testing because…

So wait, who's being dismissive of who's practices here?

Re: Addition with flamethrowers – why game devs don't unit test

#24

Post by grumpyprole »

mark38848 wrote: post #12
shaftway wrote: post #6

Earlier quoted context omitted.

I maintain that the majority of games are still disposable, despite the occasional subscription model or long-lived hit that pops up. Remember that most games aren't made by AAA studios. Wasn't Minecraft completely rewritten from scratch in Java after a few years? And the EA one, like you said, it's just model updates. Very few gameplay mechanics get more than a simple tweak. Just recompile with the new models. You d…

I think Minecraft was originally written in Java and rewritten in a good programming language (i.e. not Java).

Whether or not one thinks C++ is a "good" language, I always thought that (original) Minecraft busted the myth that blockbuster games had to be written in C++.

Re: Addition with flamethrowers – why game devs don't unit test

#25

Post by SillyUsername »

Ok I know which AAA game studio this might be because I interviewed with them and had to sign an NDA.

In their case their flagship game is full of bugs, and they had to ship their product asap pre-aquistion when they were a startup.

Because of the mentality of the managers, and weak minded devs, they don't write unit tests, and instead spend the vast majority of their days fighting bugs, so much so they have to hire dedicated staff for their (single game) backlog as they were struggling to keep up "with its success".

This is BS of course, I saw their backlog and it was a shit show, with Devs expected to work overtime free of charge to get actual features out (funny how this works isn't it, never affects the business execs' time/life who make the demands of no tests).

I was asked what I would bring to the company to help them support their now AAA game, and I stated up front "more unit tests" and indirectly criticised their lack of them. I got a call later that day that (the manager thought) "I would not be a good fit".

I got a lead job elsewhere that has the company's highest performing team, literally because of testing practices being well balanced between time and effectiveness (i.e. don't bother with low value tests, add tests if you find a bug etc, if an integration test takes too long leave it and use unit tests).

I think back to that interview every time I interview at games studios now, and wonder if I shouldn't push unit tests if they're missing. I'd still do it. The managers at that job were assholes to their developers, and I now recognise the trait in a company.

Re: Addition with flamethrowers – why game devs don't unit test

#26

Post by lionkor »

kelseydh wrote: post #10

Most video game bugs are subtle and not things that are easy to catch with unit testing because they are dynamic systems with many interacting parts. The interaction is where the bugs come from. QA processes do a good job catching the rest.

I would bet that a lot of those bugs come from utility code that is testable

Re: Addition with flamethrowers – why game devs don't unit test

#27

Post by Joel_Mckay »

1. Most game engines have the horrible compatibility layers abstracted away, and already fully tested under previous mass deployments

2. Anything primarily visual, audio, and control input based is extremely hard to reliably automate testing. Thus, if the clipping glitches are improbable and hardly noticeable... no one cares.

Some people did get around the finer game-play issues by simply allowing their AI character to cheat. Mortal Kombat II was famous for the impossible moves and combos the AI would inflict on players... yet the release was still super popular, as people just assumed they needed more practice with the game.

Have fun out there, =)

Re: Addition with flamethrowers – why game devs don't unit test

#28

Post by magoghm »

fwlr wrote: post #9

Arguments against testing tend to fall prey to the von Neumann Objection: they insist there is something tests can’t catch, and then they tell you precisely what it is that tests can’t catch… so you can always imagine writing tests for that specific thing. E.g. this article uses an example of removing the number 5, causing the developer to have to implement a base-9 numbering system. Unit tests that confirm this cust…

Tests can't catch race conditions in multithreaded code. Now that I told you what the tests can't catch, can you imagine writing tests for that specific thing?

Re: Addition with flamethrowers – why game devs don't unit test

#29

Post by throwaway115 »

Having written a 3d game engine from scratch, I had automated tests, but they were more comparable to "golden" tests, which are popular in the UI test world. Basically, my renderer needed to produce a pixel-perfect frame. If a pixel didn't match, an image diff was produced. This saved my butt numerous times when I broke subtle parts of the renderer.

Re: Addition with flamethrowers – why game devs don't unit test

#30

Post by jayd16 »

fwlr wrote: post #9

Arguments against testing tend to fall prey to the von Neumann Objection: they insist there is something tests can’t catch, and then they tell you precisely what it is that tests can’t catch… so you can always imagine writing tests for that specific thing. E.g. this article uses an example of removing the number 5, causing the developer to have to implement a base-9 numbering system. Unit tests that confirm this cust…

The lesson is more about the degree of churn and how game rules are not hard rules. A valid base 9 number system is NOT a design goal and doing that work can be a waste.

It's like testing that the website landing page is blue. Sure you can but breaking that rule is certainly valid and you'll end up ripping out a lot of tests that way.

Now, instead of calcifying the designer's whims, testing should be focused around things that actually need to make sense, ie abstract systems, data structures etc etc.

Post reply on HN