Live data from Hacker News

The Windows malloc() implementation from MSVCRT is slow

erikmcclure.com

11–20 of 187 posts

Re: The Windows malloc() implementation from MSVCRT is slow

#11
My knowledge is like 10 years old - For a long time, Microsoft's stl implementation was based on their licensning of dinkumware's STL (https://www.dinkumware.com/). Not something maintained in house. It seemed to work OK'ish - giving lowest common denominator functionality. However, it was pretty easy to create higher performing specialized data structures for your use case then what seemed like simple uses of dinkumware STL.

Re: The Windows malloc() implementation from MSVCRT is slow

#12

> I was taught that to allocate memory was to summon death itself to ruin your performance. A single call to malloc() during any frame is likely to render your game unplayable. Any sort of allocations that needed to happen with any regularity required writing a custom, purpose-built allocator, usually either a fixed-size block allocator using a freelist, or a greedy allocator freed after the level ended. Where do peo…

Strong agree. I recently wrote a semi-popular blog post about this. https://www.forrestthewoods.com/blog/benchmarking-malloc-wit...

It's interesting that LLVM is suffering so horrifically using default malloc. I really wish the author did a deeper investigation into why exactly.

Re: The Windows malloc() implementation from MSVCRT is slow

#13

> I was taught that to allocate memory was to summon death itself to ruin your performance. A single call to malloc() during any frame is likely to render your game unplayable. Any sort of allocations that needed to happen with any regularity required writing a custom, purpose-built allocator, usually either a fixed-size block allocator using a freelist, or a greedy allocator freed after the level ended. Where do peo…

If this person was taught game dev any time before about 2005, that would have still been relevant knowledge. Doing a large malloc or causing paging could have slaughtered game execution, especially during streaming.

>If you want a good malloc impl just use tcmalloc or jemalloc and be done with it

This wasn't applicable until relatively recently.

Re: The Windows malloc() implementation from MSVCRT is slow

#14
If you're depending on the performance of malloc, you're either using the language incorrectly or using the wrong language. There is no such thing as a general purpose anything when you care about performance, there's only good enough. If you are 1) determined to stick with malloc and 2) want something predictable and better, then you are necessarily on the market for one of the alternatives to the system malloc anyway.

Re: The Windows malloc() implementation from MSVCRT is slow

#15
post #14

If you're depending on the performance of malloc, you're either using the language incorrectly or using the wrong language. There is no such thing as a general purpose anything when you care about performance, there's only good enough. If you are 1) determined to stick with malloc and 2) want something predictable and better, then you are necessarily on the market for one of the alternatives to the system malloc anyw…

The whole point of the article, though, was that the system malloc was good enough on Linux and Darwin.

Re: The Windows malloc() implementation from MSVCRT is slow

#16
post #5

Has everyone forgotten that Unix is the common ancestor of Linux and every other Unixlike? I’m seeing an uptick of people writing nonsensical comments like “this was written for Linux (or Mac OS X, which implements POSIX and is therefore really Linux in drag)”.

No... That's why they had the parenthetical. The problem is, your computer probably doesn't boot the common ancestor. If you're writing UNIX-like stuff, most likely it boots macOS or Linux. If you're cool maybe it's one of the other modern BSD variants aside macOS. In practice there's a pretty low probability that your code also runs on all POSIX-compliant operating systems, and more honest/experienced people often don't kid themselves into thinking that they're seriously targeting that. Even if you believe it, you probably have some dependency somewhere that doesn't care, like Qt for example. Saying something like "Linux (or macOS, which is similar)" is a realization that you're significantly more likely to be targeting both Linux and macOS than you are to even test on BSD. And to solidify that point, note that lots of modern CI platforms don't even have great BSD support to begin with.

Of course, there is a semantic point here. macOS nominally really is UNIX, except for when someone finds out it's not actually POSIX compliant due to a bug somewhere every year or so. Still, it IS UNIX. But what people mostly run with that capability, is stuff that mostly targets Linux. So... yeah.

Of course it is true that some people really think macOS is actually Linux, but that misunderstanding is quite old by this point.

addendum: I feel like I haven't really done a good job putting my point across. What I'm really saying is, I believe most developers targeting macOS or Linux today only care about POSIX or UNIX insofar as they result in similarities between macOS and Linux. That macOS is truly UNIX makes little difference; if it happened to differ in some way, developers would happily adjust to handle it, just like they do for Linux which definitely isn't UNIX.

Re: The Windows malloc() implementation from MSVCRT is slow

#17
post #5

Has everyone forgotten that Unix is the common ancestor of Linux and every other Unixlike? I’m seeing an uptick of people writing nonsensical comments like “this was written for Linux (or Mac OS X, which implements POSIX and is therefore really Linux in drag)”.

Well, a pretty big part of the point of Linux is that it's not a Unix-descendant, just a Unix-clone.

Re: The Windows malloc() implementation from MSVCRT is slow

#20
Well... Who told you to link to MSVCRT (the one in System32)? Not Microsoft that's for sure. New software is supposed to link to the Visual Studio C runtime it was compiled with and then ship that library alongside the application itself. Even if you don't compile with VS you can distribute the runtime library (freely downloadable from some page on microsoft.com). Ostensibly, that library contains an efficient malloc. If you willingly link to the MSVCRT Microsoft for over a decade has stated is deprecated and should be avoided you are shooting yourself in the foot.

"Windows is not a Microsoft Visual C/C++ Run-Time delivery channel" https://devblogs.microsoft.com/oldnewthing/20140411-00/

Post reply on HN