Live data from Hacker News

The Windows malloc() implementation from MSVCRT is slow

erikmcclure.com

141–150 of 187 posts

Re: The Windows malloc() implementation from MSVCRT is slow

#141
post #46

Earlier quoted context omitted.

The UCRT has even been present since Windows 7, if users keep up with updates. Or if applications bundle the UCRT installer with their own.

> Or if applications bundle the UCRT installer with their own. But the UCRT is proprietary, so there are often legal issues with doing so.

There is an official Universal CRT Redistributable.

Re: The Windows malloc() implementation from MSVCRT is slow

#142
post #36

Earlier quoted context omitted.

Since we are getting pedantic, Linux isn't a UNIX.

It's absurd you would call someone pedantic for saying malloc is in a library on linux after trying to say that malloc in a library on windows.

No post body was provided.

Re: The Windows malloc() implementation from MSVCRT is slow

#143

Earlier quoted context omitted.

> Or if applications bundle the UCRT installer with their own. But the UCRT is proprietary, so there are often legal issues with doing so.

There is an official Universal CRT Redistributable.

And java ships it, and installs it into its own java bindir, not the global system dir.

So you have two competing copies, which leads to race conditions (different "global" locks!) and nice cashes, until you can analyze it with DrMemory or the kernel debugger. Nobody does that.

Re: The Windows malloc() implementation from MSVCRT is slow

#144
post #20

Well... Who told you to link to MSVCRT (the one in System32)? Not Microsoft that's for sure. New software is supposed to link to the Visual Studio C runtime it was compiled with and then ship that library alongside the application itself. Even if you don't compile with VS you can distribute the runtime library (freely downloadable from some page on microsoft.com). Ostensibly, that library contains an efficient malloc…

i'm not sure which part of the post gave you the impression that it's linking against the one in system32. the author even says:

> it's statically linking with the CRT shipped with Visual Studio

that is, unless you know for a fact that they're wrong and clang uses the msvcrt from system32. in which case this seems like clang's fault?

Re: The Windows malloc() implementation from MSVCRT is slow

#146

Earlier quoted context omitted.

There's also UCRT, which ships with the OS since Windows 10. The logic of this rant was a real head-scratcher. If you must blame one side, it's LLVM. Fragmentation of C runtimes is annoying but inescapable. Glibc for example isn't any better.

> Fragmentation of C runtimes is annoying but inescapable. Glibc for example isn't any better. Glibc very much is better. There cannot be more than a single version of a (tightly coupled) ld.so+libc.so pair in a given address space any more than there can be more than a single KERNEL32 version in a given address space, and given that some system services are exclusively accessible via dynamic linking (accelerated gra…

> There cannot be more than a single version of a (tightly coupled) ld.so+libc.so pair in a given address space

That's an odd restriction, and it is related to the fact that all symbols live in one global namespace in a process. It's annoying if you are trying to build something like a plugin system, or if you are using a dynamic language which by definition loads all libraries dynamically. This is also the reason that you cannot mix Glib (GTK+) versions in a process.

I think you should be able to dlopen some library, or heck just load some machine code, and be able to run it, just take care to only pass POD over the boundary and never `free` stuff you didn't `malloc`.

Re: The Windows malloc() implementation from MSVCRT is slow

#147

Earlier quoted context omitted.

If this person was taught game dev any time before about 2005, that would have still been relevant knowledge. Doing a large malloc or causing paging could have slaughtered game execution, especially during streaming. >If you want a good malloc impl just use tcmalloc or jemalloc and be done with it This wasn't applicable until relatively recently.

> Doing a large malloc or causing paging could have slaughtered game execution, especially during streaming. ... it still does ? I had a case a year or so ago (on then-latest Linux / GCC / etc.) where a very sporadic allocation of 40-something bytes (very exactly, inserting a couple of int64 in an unordered_map at the wrong time) in a real-time thread was enough to go from "ok" to "unuseable"

i suppose so.

modern engines generally have a memory handler, which means that mallocs are usually coached in some type of asset management. you are also discouraged from extending working memory of the scene suddenly. When I was doing gamedev, even then, there was no reason to big malloc because everything was already done for you with good guardrails

Re: The Windows malloc() implementation from MSVCRT is slow

#149
post #148
post #6

Just wait until you try to use it from multiple threads at the same time!

Not sure what's your usage was exactly, but Heap API works reallly well in this context. So much so that beating it with a custom allocator is a real challenge.

I had a system that was sped up by 30%+ on Windows by switching from HeapAlloc to jemalloc. Profiling showed that HeapAlloc was largly stuck in a single giant lock. (This was on Windows Server 2016, IIRC.) And that wasn't even that allocation-heavy in the large scale of it; most of memory was done through arena allocations, but a few larger buffers were not.

Re: The Windows malloc() implementation from MSVCRT is slow

#150
post #20

Well... Who told you to link to MSVCRT (the one in System32)? Not Microsoft that's for sure. New software is supposed to link to the Visual Studio C runtime it was compiled with and then ship that library alongside the application itself. Even if you don't compile with VS you can distribute the runtime library (freely downloadable from some page on microsoft.com). Ostensibly, that library contains an efficient malloc…

i'm not sure which part of the post gave you the impression that it's linking against the one in system32. the author even says: > it's statically linking with the CRT shipped with Visual Studio that is, unless you know for a fact that they're wrong and clang uses the msvcrt from system32. in which case this seems like clang's fault?

He updated the text after the article hit HN: https://web.archive.org/web/20220702095817/https://erikmcclu...

The C runtime shipped in Windows is called MSVCRT.DLL and the ones shipped in Visual Studio are called MSVCRxxx.DLL, where xxx is Visual Studio's version number. If he statically linked to MSVCRxxx.DLL (MSVCRxxx.LIB actually) then what version did he link to? The performance of malloc() differs between versions.

Clang doesn't ship its own C/C++ runtime and certainly can link to MSVCRT.DLL. That is how legacy applications are built.

Post reply on HN