Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“What Went Right and What Went Wrong”: Analysis of 155 Postmortems from Gamedev

research.microsoft.com

Post reply on HN
Print view View original

11–20 of 52 posts

Re: “What Went Right and What Went Wrong”: Analysis of 155 Postmortems from Gamedev

#11

Post by powertower »

padobson wrote: post #8
teddyh wrote: post #3

> 9. CONCLUSIONS > > We find that we were able to identify both best practices and pitfalls in game development using the information present in the postmortems. Such information on the development of all kinds of software would be highly useful too. Therefore we urge the research community to provide a forum where postmortems on general software development can be presented, and practitioners to report their retrosp…

There's some world-shaking insight. The best piece of advice I can give is to never, ever give an estimate to stakeholders until you've worked on the estimate first. And don't finish an estimate until you've actually built some small proofs of concept. Your estimates will still be wrong, but they'll be much more accurate than some off-the-cuff number that your stakeholders will be building their various plans around.…

Skimming over it, there seems to be much more specific details and insights in the body of the analysis.

The conclusion does not accurately reflect that... Which seems to be more of a "recommendation" than a takeaway or a real summery.

Re: “What Went Right and What Went Wrong”: Analysis of 155 Postmortems from Gamedev

#13

Post by quakenul »

yread wrote: post #9
ekianjo wrote: post #4

The recommendations at the end of the pdf seem to be written by captain Obvious. They are so generic they can apply to about ANY project even non gamedev related.

Well they weren't that obvious to 155 projects it seems...

I would assume it was (to most of them) but knowing and doing are two very different things (i.e. everyone knows smoking is bad for you and yet people still do it / start doing it).

So to me the far more interesting and helpful question to answer is: Are there most efficient ways to keep people from making the same mistake over and over, other than repeating what everyone already knows, over and over?

Identify the forces that drive people away from best practices most often and give us ideas and tools to tackle them.

Re: “What Went Right and What Went Wrong”: Analysis of 155 Postmortems from Gamedev

#14

Post by Bahamut »

ekianjo wrote: post #4

The recommendations at the end of the pdf seem to be written by captain Obvious. They are so generic they can apply to about ANY project even non gamedev related.

One of my friends who just left being full time in the game industry pointed out to me that the game industry is 10 years behind in software development best practices. Churn and burn (like many startups) is a daily reality.

Re: “What Went Right and What Went Wrong”: Analysis of 155 Postmortems from Gamedev

#15

Post by jacques_chester »

ekianjo wrote: post #4

The recommendations at the end of the pdf seem to be written by captain Obvious. They are so generic they can apply to about ANY project even non gamedev related.

Empirical research often turns up "no duh" results.

It also, often, doesn't.

In both cases, there is value in publishing.

Re: “What Went Right and What Went Wrong”: Analysis of 155 Postmortems from Gamedev

#16

Post by tslug »

quakenul wrote: post #13
yread wrote: post #9

Earlier quoted context omitted.

Well they weren't that obvious to 155 projects it seems...

I would assume it was (to most of them) but knowing and doing are two very different things (i.e. everyone knows smoking is bad for you and yet people still do it / start doing it). So to me the far more interesting and helpful question to answer is: Are there most efficient ways to keep people from making the same mistake over and over, other than repeating what everyone already knows, over and over? Identify the fo…

The best approach is to apprentice with someone who knows not to make them. If you can't do that, check in with someone regularly who:

  1. Won't sugar coat feedback,
  2. Has shipped games,
  3. Has a healthy sense of humility, and
  3. Is genuinely concerned about your success.
I really like sitting on advisory boards in this role, but my secret to being good at it is that I regularly ask for feedback from others who are smarter than me at the things I don't get (which is most things).

In the long run, it's like most things. It takes practice.

To the criticism that the game industry is behind the rest of the software industry in software engineering practices, I do believe the rest of the software industry can eat something phallic. :)

I'd like to see them pull off what we have to with about 10X the competition, juggling massive game assets, a fraction of the budget, finance, and exit options, needing to meet a tough frame rate budget (even more exacting now with VR), all issues GPU-ish (when was the last time you wrote a shader, Mr. Website Developer?), and having people with a huge range of technical skills requiring direct access to the source/asset repo.

Re: “What Went Right and What Went Wrong”: Analysis of 155 Postmortems from Gamedev

#17

Post by Negative1 »

The old post-mortems in Game Developer magazine were written almost in template form.

Went right: 1) Great team/culture 2) Everyone put forth the effort (scheduled crunch) 3) Some amazing techwork or artwork or designwork etc...

Went wrong: 1) Didn't anticipate x/y/z 2) Burnout (remember that crunch)? 3) Bad/lazy scheduling and/or capacity planning

Kudos to the people who did this at MS but you're looking at tainted data. Most game developers know what really went wrong and do not publicly broadcast it in fear of losing future contracts or publisher trust. In the end the postmortem becomes a soundboard for 'shoutouts' and cheer-leading instead of that raw, unbiased feedback.

I've been to all hands post-mortems that were like that and it can get really really ugly.

Re: “What Went Right and What Went Wrong”: Analysis of 155 Postmortems from Gamedev

#18

Post by jblow »

Bahamut wrote: post #14
ekianjo wrote: post #4

The recommendations at the end of the pdf seem to be written by captain Obvious. They are so generic they can apply to about ANY project even non gamedev related.

One of my friends who just left being full time in the game industry pointed out to me that the game industry is 10 years behind in software development best practices. Churn and burn (like many startups) is a daily reality.

If your friend was not very senior (10+ years experience), I would not believe what he says, since he probably did not have enough experience to judge the situation.

I think the idea that the game industry is "behind" other fields is kind of comical, given that games are some of the most complex software in the world, and big game teams have only a few hundred people on them, and meanwhile something relatively trivial like Twitter has 4000 people. It's true that game teams don't do a lot of Agile or TDD or whatever the next buzzword is, but that is because those things are mostly superstition and obviously don't work when you start attacking hard problems.

So if you are someone a few years out of school who learned TDD it is easy to say "games are behind, they don't do all the new stuff!!" while being unaware that almost all the new stuff is bogus cargo-cultism anyway.

I do agree that the game industry engages in unhealthy levels of crunch that are to its long-term detriment, but this is mostly an orthogonal issue to software engineering practices.

Re: “What Went Right and What Went Wrong”: Analysis of 155 Postmortems from Gamedev

#19

Post by hythloday »

jblow wrote: post #18
Bahamut wrote: post #14

Earlier quoted context omitted.

One of my friends who just left being full time in the game industry pointed out to me that the game industry is 10 years behind in software development best practices. Churn and burn (like many startups) is a daily reality.

If your friend was not very senior (10+ years experience), I would not believe what he says, since he probably did not have enough experience to judge the situation. I think the idea that the game industry is "behind" other fields is kind of comical, given that games are some of the most complex software in the world, and big game teams have only a few hundred people on them, and meanwhile something relatively trivia…

I was in the games industry from 1999 to 2012 (and then left to go to Twitter!) and I think his friend is absolutely right, the games industry shoots itself in the foot repeatedly by believing that it's a special snowflake full of "hard problems" that no-one else tackles and continually (and badly) tries to reinvent the wheel. The universal culture of deathmarches and the denigration of mainstream software engineering and project management techniques are not in any way coincidental.

(I loved the Witness, btw.)

Re: “What Went Right and What Went Wrong”: Analysis of 155 Postmortems from Gamedev

#20

Post by pandaman »

Negative1 wrote: post #17

The old post-mortems in Game Developer magazine were written almost in template form. Went right: 1) Great team/culture 2) Everyone put forth the effort (scheduled crunch) 3) Some amazing techwork or artwork or designwork etc... Went wrong: 1) Didn't anticipate x/y/z 2) Burnout (remember that crunch)? 3) Bad/lazy scheduling and/or capacity planning Kudos to the people who did this at MS but you're looking at tainted…

Exactly this. Only one game I've shipped had a Gamasutra postmortem but that one was so generic that it could have been applied to any other game, ignoring the specific issues that project had.
Post reply on HN