Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Addition with flamethrowers – why game devs don't unit test

pixelatedplaygrounds.com

Post reply on HN
Print view View original

81–90 of 96 posts

Re: Addition with flamethrowers – why game devs don't unit test

#81

Post by cableshaft »

tmtvl wrote: post #69
2muchcoffeeman wrote: post #38

Earlier quoted context omitted.

I still play games that came out a couple decades ago…

Let me guess... Super Metroid? Chrono Trigger? Final Fantasy VI? Ultima Underworld? Symphony of the Night? There were a few decent games released in the '80s and '90s.

The fact that you put Ultima Underworld in the company of those masterpieces makes me think I should probably give that game a shot.

I only tried playing one Ultima game a long, long time ago, and I couldn't get into it. I'm guessing that one is a particularly good one, though.

Re: Addition with flamethrowers – why game devs don't unit test

#82

Post by cableshaft »

mistercow wrote: post #55
jayd16 wrote: post #23

Earlier quoted context omitted.

So wait, who's being dismissive of who's practices here?

I didn’t say engineers are dismissive of other engineers’ practices. The general pattern is “that makes sense for your field, but we can’t use it because…” followed by silly reasons. I was guilty of this myself back when I was an indie dev. It took me an embarrassingly long time, for example, to admit that git wasn’t just something teams needed to coordinate, and that I should be using it as the sole developer of a p…

Version control I think is worth the effort much more than unit testing in games. If you lose your code it's catastrophic. I also used to not use version control in games, just saved extra folders. It took working in webdev for a while for it to click that I should use it for my games, also (and I didn't even start using Github issues for my personal projects until about a year ago, and I've been kicking myself for not doing that earlier, I just kept track in my head what I wanted to work on next for the game and maybe a simple list of planned major features in a text file somewhere).

If you don't unit test something in a game, pretty much the worst you can do in a game is give players a good time for a little while with an infinite money glitch or maybe temporarily block quest progress and annoy them for a bit.

Although large studios get away with the latter all the time, I played Cyberpunk 2077 a year after it was released and it still had a bug that nearly killed my ability to finish a required story mission. A suitcase that was supposed to be on the room table (we referred to it being there in the scene) teleported into the hotel room's entrance and completely blocked my ability to walk through the doorway to continue the mission.

It even stayed there when I quit and reloaded, and I verified that other quite a few other players online were complaining about it at least six months prior so it was a known bug for that long. I eventually accidentally found a work around by waiting for another character in the scene to leave the hotel room and got in front of him and his collision detection apparently trumped the suitcase's collision detection and he was able to push me past the obstacle.

Re: Addition with flamethrowers – why game devs don't unit test

#83

Post by cableshaft »

AndyPa32 wrote: post #49
jrockway wrote: post #18

Bugs are kind of the fun part of games. If every subroutine worked perfectly, you wouldn't have the chaos of real life. Some of players favorite mechanics are just bugs. (Overwatch example: Mercy's super jump, now a legitimate predictable mechanic that everyone can do, not just people that read the forums and watch YouTube videos about bugs. It started out as a bug, and it was so cool and skill-ceiling increasing tha…

> Games are just software engineering. Fun software engineering. I do question the "fun" part. Midnight crunches, unpaid overtime and - as far as I have read - some of the worst working conditions in all of software engineering. I pass.

I've done both gamedev and webdev professionally. I don't miss the long hours or the low pay of gamedev, but I do miss what I worked on. Enough that I'm still working on my own games in my spare time.

I still play the games I used to work on professionally from time to time, and like to show them off to people, even though none were financially successful (a few I worked on solo were at least popular, but we made some 'bad decisions in hindsight' or got really unlucky with release timing for all the games I worked on professionally).

I don't care hardly at all about all the webdev projects I worked on professionally. They each had some intresting problem solving or coding challenges, and some of them were used WAY more than the games I made were played, but I'm still more into the games I made, and those are most likely to still be around after I'm dead, in a Flash game or ROM archive or torrent somewhere.

Also, the worst long hours I ever had was actually in webdev. I once worked in software for a call center, and whenever the phone systems went down, I was expected to be on a call to help fix it (as there was 100+ call center employees that couldn't make calls and we were losing lots of money every minute they were down), sometimes for 16 hours or more sitting on a Zoom call, mostly waiting for people on the server teams to figure things out. And there was a data center migration that had some unexpected problems and required three of us to work for almost three days straight (I literally worked a 24 hour shift, then had 4 hours of sleep, then worked an 18 hour shift right after it).

But that's not the norm in webdev at least. My current webdev job I only worked >40 hours a couple weeks to rework some issues with some junior dev's code for a feature before a big demo.

Re: Addition with flamethrowers – why game devs don't unit test

#84

Post by bluefirebrand »

wesselbindt wrote: post #74
bluefirebrand wrote: post #64

Earlier quoted context omitted.

> This doesn't sound so much as too much coverage but rather like having your automated tests be coupled to implementation details Depending on how high coverage you are aiming for, I find it hard to imagine a way to achieve it without inevitably tying the tests to implementation details

Decoupling tests from what they test does require a concerted effort, and is a skill that requires practice, but in general not as much as you'd think. Most devs are quite comfortable getting rid of coupling between two non-test components (functions, classes, services, whatever) of their system. The main mental hurdle seems to be treating your automated test code as any other code, capable and worthy of being decoup…

> So here, by increasing the scope of your test suite from `prep` and `finish` to `sort`, you've achieved looser coupling between the test suite and what is being tested

But you won't have very good coverage of "prep" or "finish"

You can of course test those functions by carefully constructing test cases for "sort", but that essentially just re-introduces the coupling to implementation details at a higher level

Re: Addition with flamethrowers – why game devs don't unit test

#85

Post by hahn-kev »

Aerroon wrote: post #46
lionkor wrote: post #26

Earlier quoted context omitted.

I would bet that a lot of those bugs come from utility code that is testable

Perhaps in development, but the stuff that tends to make it into the release of games seems to be gameplay related. Npc behaviors not lining up, the developer literally not implementing certain stats in the game (looking at you, Diablo 4), graphical bugs caused by something not loading or loading too slowly, performance issues from something loading 1000 copies of itself etc.

And where do you think many of those bugs come from? Usually utility code that isn't working in some edge case, those types of things can be easy to pull out and test in their own independent of the game

Re: Addition with flamethrowers – why game devs don't unit test

#86

Post by wesselbindt »

bluefirebrand wrote: post #84
wesselbindt wrote: post #74

Earlier quoted context omitted.

Decoupling tests from what they test does require a concerted effort, and is a skill that requires practice, but in general not as much as you'd think. Most devs are quite comfortable getting rid of coupling between two non-test components (functions, classes, services, whatever) of their system. The main mental hurdle seems to be treating your automated test code as any other code, capable and worthy of being decoup…

> So here, by increasing the scope of your test suite from `prep` and `finish` to `sort`, you've achieved looser coupling between the test suite and what is being tested But you won't have very good coverage of "prep" or "finish" You can of course test those functions by carefully constructing test cases for "sort", but that essentially just re-introduces the coupling to implementation details at a higher level

That is indeed a drawback of this approach, and is problematic if and only if `prep` and `finish` are part of the public interface. If they're not, it's a worthwhile exercise to ask yourself what you want from your test suite, and whether or not individual tests for prep and finish support that goal. For me personally, my ultimate goal for a test suite is to get as close to the equivalence "the tests do not pass pushing this to production would break the product" as humanly possible, as I really don't like spending time "fixing" tests. This leads me to test at the system boundaries, i.e., to leave the lower level tests out when I can. What value does testing stuff that will not affect production in any way bring me?

> but that essentially just re-introduces the coupling to implementation details at a higher level

The definition of coupling that I find useful is the following. A thing f is coupled to a thing g w.r.t. a change d in g if changing g by d requires you to change f. I find that it captures exactly the phenomenon which makes maintenance and extension of brownfield projects so costly.

So, for example, both test suites I gave as an example are coupled to sort with respect to the change "sort sorts stuff in descending order instead of ascending". Making this change requires you change pretty much every assert but the trivial ones, in both test suites.

With respect to the change "move some code from prep to finish" or "get rid of prep and finish entirely and move everything to the body of sort", only the second suite is coupled to sort.

This may not be the definition of coupling that you like to use. If it is, I don't see how the test suite of the first kind including edge cases for prep and finish at the level of sort is still coupled to implementation details.

Re: Addition with flamethrowers – why game devs don't unit test

#87

Post by bluefirebrand »

wesselbindt wrote: post #86
bluefirebrand wrote: post #84

Earlier quoted context omitted.

> So here, by increasing the scope of your test suite from `prep` and `finish` to `sort`, you've achieved looser coupling between the test suite and what is being tested But you won't have very good coverage of "prep" or "finish" You can of course test those functions by carefully constructing test cases for "sort", but that essentially just re-introduces the coupling to implementation details at a higher level

That is indeed a drawback of this approach, and is problematic if and only if `prep` and `finish` are part of the public interface. If they're not, it's a worthwhile exercise to ask yourself what you want from your test suite, and whether or not individual tests for prep and finish support that goal. For me personally, my ultimate goal for a test suite is to get as close to the equivalence "the tests do not pass push…

I strongly do agree with your approach, for the record

I suppose it really comes down to what kind of coverage is important

In your case you seem to take the approach of having coverage on the high level surface area

Mostly I see coverage referring to the actual lines of code and branches

Having high coverage of lines and branches does require having tests that even dig into your private interfaces

Re: Addition with flamethrowers – why game devs don't unit test

#88

Post by renox »

HideousKojima wrote: post #39
fwlr wrote: post #9

Arguments against testing tend to fall prey to the von Neumann Objection: they insist there is something tests can’t catch, and then they tell you precisely what it is that tests can’t catch… so you can always imagine writing tests for that specific thing. E.g. this article uses an example of removing the number 5, causing the developer to have to implement a base-9 numbering system. Unit tests that confirm this cust…

Nah, my objection to unit testing is that too often it devolves into what I call "Testing that the code does what the code does." If you find yourself often writing code that also requires updating or rewriting unit tests, your tests are mostly worthless. Unit tests are best for when you have a predefined spec, or you have encountered a specific bug previously and make a test to ensure it doesn't reoccur, or you want…

Unfortunately this can also be the case of integration test : I spent the last week trying to understand if the regressio n in an integration test was a bug or if it was the previous behaviour which was buggy..

Re: Addition with flamethrowers – why game devs don't unit test

#89

Post by mabster »

treflop wrote: post #34
shaftway wrote: post #2

I don't buy this argument. Most game developers I know have said that unit tests are a waste of time so they never use them, but they're struggling with making changes to utility code and making sure that it doesn't do the wrong thing. Y'know, what unit tests are for. I think the key here is that the perceived cost / benefit ratio is too high. It's the perception that drives their behavior though. I'm in a company no…

I’ve seen people slog through untested code where they fear to make a change but I’ve also seen people slog through code with too much test coverage where the tests go through constant churn. I don’t understand why people don’t just add one test even if the codebase otherwise has zero tests if they’re so scared of one area and I don’t get why people keep adding excessive coverage if it’s wasting their time. It’s like…

All the places I've worked for had some balance here, but it would definitely be on the very few tests end.

We would write tests to catch a bug in a low level system, and keep the test after. We had lots of Design by Contract, including Invariants that were enabled in debug mode.

But the reality was that we couldn't test gameplay code very well. That changed so dramatically over the course of a project that if we did test we would just end up commenting tests by the end of a project.

And as an optimisation guy, I would often have to change the "feel" of gameplay code to get performance out of code, which is checked by a Quality Assurance team, because it's subjective. That kind of stuff would make gameplay tests very brittle.

The pace of game Dev was incredibly fast. We were struggling to get all our stuff in, never mind adding any scaffolding that would slow us down.

Re: Addition with flamethrowers – why game devs don't unit test

#90

Post by sfn42 »

cobalt wrote: post #43
shaftway wrote: post #6

Earlier quoted context omitted.

I maintain that the majority of games are still disposable, despite the occasional subscription model or long-lived hit that pops up. Remember that most games aren't made by AAA studios. Wasn't Minecraft completely rewritten from scratch in Java after a few years? And the EA one, like you said, it's just model updates. Very few gameplay mechanics get more than a simple tweak. Just recompile with the new models. You d…

the original minecraft is in java, it's probably gone through a lot of code transformation. The version you're thinking of is the microsoft version, rewritten in c++

I thought the MS version was C#?
Post reply on HN