Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

EA open sources their internal version of STL

github.com

Post reply on HN
Print view View original

21–30 of 37 posts

Re: EA open sources their internal version of STL

#21

Post by dfox »

jrockway wrote: post #9
ig1 wrote: post #8

Earlier quoted context omitted.

A surprisingly large number of companies have their own STL implementations from Investment Banks to Telecoms companies. Often because they're doing funky things (often related to memory allocation) which would be outright impossible to do in languages like Java or C#.

I have never seen anyone mix their custom memory manager with STL. Also, who said anything about Java or C#?

STL is more or less designed to support custom memory allocation, I would say that half of it's complexity comes from this.

Re: EA open sources their internal version of STL

#22

Post by corysama »

terryjsmith wrote: post #5
jrockway wrote: post #4

This is why game companies use C++. They've already reimplemented it internally and don't ever want to do that again.

I imagine most STL classes still don't fit game programming paradigms. My very first experience with this was using vectors to store vertices as an intermediary before sticking them into a vertex buffer. Granted, my math was awful, but the memory overhead was absurd. I constantly find myself over using STL in my programs because of how easy it is, then redoing it to use standard C/C++ arrays, linked lists, etc. for e…

A bit part of the optimization is the flattening of deeply nested calls to tiny functions that are prevalent in the STL. In the general case of STL usage, these calls serve an important purpose in the design and flexibility of the library. They also generally inline away in optimized builds. However, in the specific case of game development, they greatly slow down the performance of non-optimized builds. That performance is a big deal when you are debugging a game that by design stresses the limits of the fixed hardware platform it is running on (you can't upgrade the CPU in a PS3 devkit). The nested inline functions also make debugging much harder because stepping into observe simple actions can result in walking a long chain of calls that each effectively do nothing.

Re: EA open sources their internal version of STL

#23

Post by repsilat »

albertzeyer wrote: post #12
NickPollard wrote: post #10

Earlier quoted context omitted.

You're being more tolerant. When you're building a high performance game engine for fixed target hardware, the last thing you want is to be doubling up the size of your arrays.

If you know before the allocation how much entries it must hold at maximum, you can give an std::vector that information. There is simply no case where you are forced to require more memory with an std::vector but not with a manual implementation.

If you have a lot of small vectors those three pointers can be expensive. On a 64 bit machine they're 8 bytes apiece.

If you know the vector isn't going to be enormous you can swap two of those pointers for unsigned integers, cutting overhead down to 16 bytes total. If you know more about your data and usage patterns you can probably get it further. For very small vectors of largely static data you might trade one of the integers for linear-time insertion and deletion. If capacity is known at compile-time you can make it part of the type. If you know that size() will always equal capacity() you can get rid of the size int.

It's also worth noting that you don't have to double your storage on every reallocation to avoid linear-time insertions - you just have to multiply it by some constant that's greater than one. If you don't know how many reallocs you're going to do, but you know that it isn't many, you might want to just multiply your capacity by 1.1 every time to avoid a lot of wasted space. Assuming, of course, that you're happy with bringing floating point computation in, and that you've thought about how you're going to increase the capacity of a vector of size 1...

Re: EA open sources their internal version of STL

#24

Post by jrockway »

aaronblohowiak wrote: post #20
jrockway wrote: post #9

Earlier quoted context omitted.

I have never seen anyone mix their custom memory manager with STL. Also, who said anything about Java or C#?

Right, they have a custom STL because they want to have their custom memory management.

Here's my experience. Whenever a C++ programmer uses the word performance, it's never to describe using template-based generic data structures. I do it and the performance is fine, but the hard-core C++ programmers seem to hate STL.

Incidentally, I asked someone about memory management in an interview for a C++ position. He said he wrote his own memory manager (object pool, basically), and never even bothered to use "new" or "delete".

Personally, I'm sticking to Haskell...

Re: EA open sources their internal version of STL

#25

Post by sazzal »

aaronblohowiak wrote: post #20
jrockway wrote: post #9

Earlier quoted context omitted.

I have never seen anyone mix their custom memory manager with STL. Also, who said anything about Java or C#?

Right, they have a custom STL because they want to have their custom memory management.

The reason you need to really care about memory management in game development is because most of the consoles don't have virtual memory. Once you run out (and you don't have that much), your game stops working. It's very similar to embedded system / real-time programming in this way. So you need to plan out how much space everything will use in advance, basically. Lots of fixed-sized buffers and pool allocators.

And the reason you don't use garbage collection is because real-time games can't afford a GC pause.

Re: EA open sources their internal version of STL

#28

Post by s3graham »

dfox wrote: post #21
jrockway wrote: post #9

Earlier quoted context omitted.

I have never seen anyone mix their custom memory manager with STL. Also, who said anything about Java or C#?

STL is more or less designed to support custom memory allocation, I would say that half of it's complexity comes from this.

Badly. Whoever thought putting it in the type of the container was a good idea obviously never actually tried to use it.

Re: EA open sources their internal version of STL

#29

Post by lokijuhyhui »

sazzal wrote: post #25
aaronblohowiak wrote: post #20

Earlier quoted context omitted.

Right, they have a custom STL because they want to have their custom memory management.

The reason you need to really care about memory management in game development is because most of the consoles don't have virtual memory. Once you run out (and you don't have that much), your game stops working. It's very similar to embedded system / real-time programming in this way. So you need to plan out how much space everything will use in advance, basically. Lots of fixed-sized buffers and pool allocators. And…

same thing we used to do on supercomputers, allocating memory is very expensive and surprisingly limited.

You also don't want your code to fail after weeks of runtime with an alloc error, if it's going to run out of memory you want it to fail at the start.

Re: EA open sources their internal version of STL

#30

Post by lokijuhyhui »

archangel_one wrote: post #6
terryjsmith wrote: post #5

Earlier quoted context omitted.

I imagine most STL classes still don't fit game programming paradigms. My very first experience with this was using vectors to store vertices as an intermediary before sticking them into a vertex buffer. Granted, my math was awful, but the memory overhead was absurd. I constantly find myself over using STL in my programs because of how easy it is, then redoing it to use standard C/C++ arrays, linked lists, etc. for e…

Any sane implementation of std::vector is probably going to have three pointers of overhead, and might allocate twice as much memory as required in the worst case when you push a bunch of objects into it. That doesn't seem totally absurd to me - was yours much worse, or am I just being more tolerant of it?

It's great until you need to allocate 3.9Gb of vector on a 32bit machine.

Somebody said computer science is solving yesterdays problems on tomorrows hardware, the rest of science is solving tomorrows problems on yesterdays hardware

Post reply on HN