Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is TDD or BDD used in AAA gaming industry?

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 64 posts

Re: Ask HN: Is TDD or BDD used in AAA gaming industry?

#31

Post by shubhamjain »

dfabulich wrote: post #8

In my experience, no, they don't write tests first, and in fact AAA game companies write almost no automated tests at all. Instead, AAA teams have large staffs of cheap manual testers--high school and college kids who think they're getting paid to play video games. The managers in charge of these QA teams work their manual testers to the bone, encouraging them to work late and do unpaid overtime. These testers inevit…

I find it hard to wrap my head around how the whole AAA VG ecosystem works. How is it able to function? AAA Video Games are highly complex engineering projects but, as I understand, under the hood you have naive and inexperienced kids, horrible engineering practices, and a chaotic work environment. How does this industry manage to hire so many quality engineers and treat them as dispensable that easily?

Re: Ask HN: Is TDD or BDD used in AAA gaming industry?

#32

Post by user5994461 »

shubhamjain wrote: post #31
dfabulich wrote: post #8

In my experience, no, they don't write tests first, and in fact AAA game companies write almost no automated tests at all. Instead, AAA teams have large staffs of cheap manual testers--high school and college kids who think they're getting paid to play video games. The managers in charge of these QA teams work their manual testers to the bone, encouraging them to work late and do unpaid overtime. These testers inevit…

I find it hard to wrap my head around how the whole AAA VG ecosystem works. How is it able to function? AAA Video Games are highly complex engineering projects but, as I understand, under the hood you have naive and inexperienced kids, horrible engineering practices, and a chaotic work environment. How does this industry manage to hire so many quality engineers and treat them as dispensable that easily?

Because games are cool and there is no shortage of good developers.

All games are throwaway projects. Write one, sell for a year, gone. There is no maintenance period.

Re: Ask HN: Is TDD or BDD used in AAA gaming industry?

#33

Post by StavrosK »

discreteevent wrote: post #29
zamalek wrote: post #14

Earlier quoted context omitted.

This is my impression too. There's also good reasoning: TDD is poorly suited to the industry. For example, how would you test your renderer? Do you run the pipeline and check it against a final image? No, strict TDD would require that the asserted image be hand-crafted. Do you check that each part of the pipeline works as a unit? You can't without running shaders on the GPU (requiring the previous image test). What g…

In my opinion it's the integration tests that provide 99.99% of the value in 99.99% of the cases. Maybe not full integration tests (which can be expensive to run) but at least subsystem tests. These are the ones that find the real bugs. A very high percentage of unit tests once passing tend to continue to pass and only fail when you intentionally change the implementation. i.e. they impede your progress. Low level un…

I agree, this matches my experience. I find that many bugs can crop up in the interface, which is something that TDD inherently doesn't catch, whereas integration testing will both catch bugs in the interface and the units themselves.

Re: Ask HN: Is TDD or BDD used in AAA gaming industry?

#34

Post by flohofwoe »

user5994461 wrote: post #32
shubhamjain wrote: post #31

Earlier quoted context omitted.

I find it hard to wrap my head around how the whole AAA VG ecosystem works. How is it able to function? AAA Video Games are highly complex engineering projects but, as I understand, under the hood you have naive and inexperienced kids, horrible engineering practices, and a chaotic work environment. How does this industry manage to hire so many quality engineers and treat them as dispensable that easily?

Because games are cool and there is no shortage of good developers. All games are throwaway projects. Write one, sell for a year, gone. There is no maintenance period.

Not true for MMOs and F2P games, these often run for many years some with biweekly or monthly content updates.

Re: Ask HN: Is TDD or BDD used in AAA gaming industry?

#35

Post by andybak »

discreteevent wrote: post #29
zamalek wrote: post #14

Earlier quoted context omitted.

This is my impression too. There's also good reasoning: TDD is poorly suited to the industry. For example, how would you test your renderer? Do you run the pipeline and check it against a final image? No, strict TDD would require that the asserted image be hand-crafted. Do you check that each part of the pipeline works as a unit? You can't without running shaders on the GPU (requiring the previous image test). What g…

In my opinion it's the integration tests that provide 99.99% of the value in 99.99% of the cases. Maybe not full integration tests (which can be expensive to run) but at least subsystem tests. These are the ones that find the real bugs. A very high percentage of unit tests once passing tend to continue to pass and only fail when you intentionally change the implementation. i.e. they impede your progress. Low level un…

That matches my intuition. The espousal of unit testing vs other types of tests as "the one true way" actually impeded my adoption of testing in general as I couldn't wrap my head around how to approach it without suffering a huge productivity hit. I thought it was just me "not getting it" but I sense the pendulum is beginning to reverse direction.

Re: Ask HN: Is TDD or BDD used in AAA gaming industry?

#36

Post by quadcore »

discreteevent wrote: post #29
zamalek wrote: post #14

Earlier quoted context omitted.

This is my impression too. There's also good reasoning: TDD is poorly suited to the industry. For example, how would you test your renderer? Do you run the pipeline and check it against a final image? No, strict TDD would require that the asserted image be hand-crafted. Do you check that each part of the pipeline works as a unit? You can't without running shaders on the GPU (requiring the previous image test). What g…

In my opinion it's the integration tests that provide 99.99% of the value in 99.99% of the cases. Maybe not full integration tests (which can be expensive to run) but at least subsystem tests. These are the ones that find the real bugs. A very high percentage of unit tests once passing tend to continue to pass and only fail when you intentionally change the implementation. i.e. they impede your progress. Low level un…

I agree with that.

Re: Ask HN: Is TDD or BDD used in AAA gaming industry?

#37

Post by [deleted] »

dfabulich wrote: post #8

In my experience, no, they don't write tests first, and in fact AAA game companies write almost no automated tests at all. Instead, AAA teams have large staffs of cheap manual testers--high school and college kids who think they're getting paid to play video games. The managers in charge of these QA teams work their manual testers to the bone, encouraging them to work late and do unpaid overtime. These testers inevit…

[deleted]

Re: Ask HN: Is TDD or BDD used in AAA gaming industry?

#38

Post by [deleted] »

user5994461 wrote: post #32
shubhamjain wrote: post #31

Earlier quoted context omitted.

I find it hard to wrap my head around how the whole AAA VG ecosystem works. How is it able to function? AAA Video Games are highly complex engineering projects but, as I understand, under the hood you have naive and inexperienced kids, horrible engineering practices, and a chaotic work environment. How does this industry manage to hire so many quality engineers and treat them as dispensable that easily?

Because games are cool and there is no shortage of good developers. All games are throwaway projects. Write one, sell for a year, gone. There is no maintenance period.

[deleted]

Re: Ask HN: Is TDD or BDD used in AAA gaming industry?

#39

Post by Negitivefrags »

I work on a game called Path of Exile.

At our studio we don't do TDD, but we do continuous integration with a lot of asset testing.

In the early days, we tried doing unit tests, but honestly, it's very hard in a game. We had little success testing gameplay code.

What we have a huge amount of is asset tests.

We have 4 times more non-programmers than programmers working on the game, so most of what is created isn't code anyway.

Every time anyone commits anything, it goes through our continuous integration pipeline, and any asset that changed, or is dependent on an asset that changed, is loaded and tested. This includes test loading entire levels.

Test loading every asset catches most types of failures.

We try, where possible, to add specific tests for common mistakes in order to catch things as early as possible.

For example, if an animation has "walk" or "run" in the name, then we test it also contains footstep events in the animation somewhere. If a model is used as a monster we test that it has an attachment point called "frame_upper" onto which various effects can be attached. We look for things like bounding box of an object being far away from the co-ordinate origin (a common mistake when making a big maya scene with lots of exported objects in it).

We also do a test start of the servers, which includes loading all of their data, and a test start of the client.

All of this means that a build that goes through green is unlikely to cause crashes for other people trying to work on the game.

We do, of course, also have an excellent QA team.

Post reply on HN