Live data from Hacker News

The Windows malloc() implementation from MSVCRT is slow

erikmcclure.com

41–50 of 187 posts

Re: The Windows malloc() implementation from MSVCRT is slow

#42
post #21

Seeing someone refer to any piece of software technology as a "trash fire" makes it harder for me to view them as credible. It's unnecessarily divisive and insulting, and it means it's unlikely they will have any appreciation of the tradeoffs present during initial design and implementation.

[deleted]

Re: The Windows malloc() implementation from MSVCRT is slow

#43

> I was taught that to allocate memory was to summon death itself to ruin your performance. A single call to malloc() during any frame is likely to render your game unplayable. Any sort of allocations that needed to happen with any regularity required writing a custom, purpose-built allocator, usually either a fixed-size block allocator using a freelist, or a greedy allocator freed after the level ended. Where do peo…

Aside from the performance implications being very real (even today, the best first step to micro-optimize is usually to kill/merge/right-size as many allocations as possible), up through ~2015 the dominant consoles still had very little memory and no easy way to compact it. Every single non-deterministic malloc was a small step towards death by fragmentation. (And every deterministic malloc would see major performance gains with no usability loss if converted to e.g. a per-frame bump allocator, so in practice any malloc you were doing was non-deterministic.)

Re: The Windows malloc() implementation from MSVCRT is slow

#44

> I was taught that to allocate memory was to summon death itself to ruin your performance. A single call to malloc() during any frame is likely to render your game unplayable. Any sort of allocations that needed to happen with any regularity required writing a custom, purpose-built allocator, usually either a fixed-size block allocator using a freelist, or a greedy allocator freed after the level ended. Where do peo…

[deleted]

Re: The Windows malloc() implementation from MSVCRT is slow

#45
post #36
post #27

Earlier quoted context omitted.

malloc() isn't part of the Linux API which provides mmap().

Since we are getting pedantic, Linux isn't a UNIX.

I thought it was. Some distro (I forget the name) paid to be certified with The Open Group.

Re: The Windows malloc() implementation from MSVCRT is slow

#46

Not that it helps here, but Microsoft never considered the MSVCRT that ships with Windows to be public API. This is not the "Windows allocator", this is the (very) old MSVC runtime library's allocator. Of course that doesn't keep anyone from using this library because it's present on any Windows system, unlike the newer MSVC versions' runtime library. Using the allocator from a later MSVC's runtime library would prov…

There's also UCRT, which ships with the OS since Windows 10. The logic of this rant was a real head-scratcher. If you must blame one side, it's LLVM. Fragmentation of C runtimes is annoying but inescapable. Glibc for example isn't any better.

The UCRT has even been present since Windows 7, if users keep up with updates. Or if applications bundle the UCRT installer with their own.

Re: The Windows malloc() implementation from MSVCRT is slow

#47
post #33

Not that it helps here, but Microsoft never considered the MSVCRT that ships with Windows to be public API. This is not the "Windows allocator", this is the (very) old MSVC runtime library's allocator. Of course that doesn't keep anyone from using this library because it's present on any Windows system, unlike the newer MSVC versions' runtime library. Using the allocator from a later MSVC's runtime library would prov…

It's effectively mandatory. Microsoft provides about twelve different C Runtimes. But if you're building something like an open source library, you can't link two different C runtimes where you might accidentally malloc() memory with one and then free() with the other. If you want to be able to pass pointers around your dynamic link libraries, you have to link the one C runtime everyone else uses, which is MSVCRT. Al…

It isn't mandatory. I have never actively linked against MSVCRT on Windows. From my experience it's mostly software that isn't built with Visual Studio that uses MSVCRT, or software that that takes extreme care of its binary size (e.g. 64k intros). MSVCRT is not even an up-to-date C runtime library. You wouldn't be able to use it for writing software requiring C11 library features without implementing them somewhere on top of it.

It's true that you cannot just happily pass pointers around and expect someone else to be able to safely delete your pointer - but that is why any serious library with a C interface provides its own function to free objects you obtained from the library. Saying that this is impossible without MSVCRT implies that every software needs to be built with it, which is not even remotely the case. If I wanted, I could build all the C libraries I use with LLVM and still link against them in my application compiled with the latest MSVC runtime or UCRT.

The much bigger problem is mixing C++ runtimes in the same piece of software, there you effectively must guarantee that each library uses the same runtime, or chaos ensues.

Re: The Windows malloc() implementation from MSVCRT is slow

#48
post #20

Well... Who told you to link to MSVCRT (the one in System32)? Not Microsoft that's for sure. New software is supposed to link to the Visual Studio C runtime it was compiled with and then ship that library alongside the application itself. Even if you don't compile with VS you can distribute the runtime library (freely downloadable from some page on microsoft.com). Ostensibly, that library contains an efficient malloc…

there's a weird licensing thing for the VC runtime where you can't redistribute the dll alongside your code unless you have a special license. instead, you have to install it as an MSI. I have no idea why they created that restriction. I even work for MS and it baffles me.

At one point MS was very keen for people to use MSI's rather than building custom installers that jam things into system32. Perhaps that was why.

Re: The Windows malloc() implementation from MSVCRT is slow

#49
post #31

Earlier quoted context omitted.

Well, a pretty big part of the point of Linux is that it's not a Unix-descendant, just a Unix-clone.

What’s the distinction between those two?

Part of it is whether the code can be traced back to original AT&T code. Which would be true for e.g. BSD variants (which includes MacOS). https://i.redd.it/kgv4ckmz3zb51.jpg

Another part is the trademark and certification fee. https://www.opengroup.org/openbrand/Brandfees.htm

Re: The Windows malloc() implementation from MSVCRT is slow

#50
post #33

Not that it helps here, but Microsoft never considered the MSVCRT that ships with Windows to be public API. This is not the "Windows allocator", this is the (very) old MSVC runtime library's allocator. Of course that doesn't keep anyone from using this library because it's present on any Windows system, unlike the newer MSVC versions' runtime library. Using the allocator from a later MSVC's runtime library would prov…

It's effectively mandatory. Microsoft provides about twelve different C Runtimes. But if you're building something like an open source library, you can't link two different C runtimes where you might accidentally malloc() memory with one and then free() with the other. If you want to be able to pass pointers around your dynamic link libraries, you have to link the one C runtime everyone else uses, which is MSVCRT. Al…

Our programs ship their DLL dependencies in their own installer anyway, like most others on Windows. Just ship your FOSS library with a CMake configuration and let the users build it with whatever runtime they want.
Post reply on HN