Live data from Hacker News

The Windows malloc() implementation from MSVCRT is slow

erikmcclure.com

151–160 of 187 posts

Re: The Windows malloc() implementation from MSVCRT is slow

#151
post #89
post #23

There is no Windows malloc(). Only UNIXes have the C API as part of the OS API.

malloc() is defined by the C Standard. If you want to claim your compiler is ANSI or ISO certified, you need to support malloc() (as well as the rest of the C Standard library).

Quite right, except we are then talking about compilers and not OS APIs.

UNIXes are the only OSes were there is an overlapping between OS APIs and libc due to C's origin.

Re: The Windows malloc() implementation from MSVCRT is slow

#152

Earlier quoted context omitted.

VirtualAlloc is a better base for a custom memory allocator. It's closer to mmap + mprotect in functionality. There's also CoTaskMemAlloc (aka IMalloc::Alloc). And COM automation has a bunch of methods which allocate memory for dynamically sized data, which could be abused for memory allocation - SafeArrayCreate, SysAllocString.

It sounds like they've invented 15 different ways to allocate memory rather than have malloc. What did they do that for? (Not to say malloc is a perfect API, it’s definitely oversimplified, but they probably didn’t solve any of its problems.)

The latter three are for RPC / interop / automation scenarios, simplifying the programming model for things like Visual Basic.

HeapAlloc (and legacy routines GlobalAlloc and LocalAlloc which wrap it) is mostly a relic of 16-bit Windows.

VirtualAlloc is the one that matters for language runtimes on Windows since Win32 API in the 90s, and it's designed to allocate slabs which are suballocated by more sophisticated code.

Re: The Windows malloc() implementation from MSVCRT is slow

#153

Earlier quoted context omitted.

> It was in the past. At first msvcrt.dll was the runtime library used up to Visual C++ 6. At that time it was already a big mess, because at first it was the runtime library of Visual C++ 4 in fact! The gory details are here: https://devblogs.microsoft.com/oldnewthing/20140411-00/?p=12... > some system components still link to it Some system components themselves are very much ancient and unmaintained and only exist…

Ancient or not, I don't think it really matters for allocation performance: malloc in both msvcrt.dll and ucrtbase.dll after some indirection ends up calling RtlAllocateHeap in ntdll.dll

It does, because modern allocators do a lot of house keeping on top of the system heaps, e.g. to quickly reuse identically-sized memory chunks that were just freed.

Re: The Windows malloc() implementation from MSVCRT is slow

#154

Earlier quoted context omitted.

You shouldn't read too much into the PowerShell story. Creating your own programming language is in most cases a frivolous vanity project. Spending company resources on your own frivolous vanity projects is the sort of thing that can get you fired.

! I disagree. CMD badly needed replacing. MS needed a new shell language. A functional company would connect people with a passion for X with the resources to achieve X, if X has a chance of helping the company. Windows Terminal and WSL show how far MS has come from the PS days. (Disclaimer: I work for MS)

I didn't say that this project in particular was a vanity project. Just that when someone comes to you with a project and says "I'm going to solve our problems by using this new programming language that I'm in the process of inventing", then some skepticism is not unwarranted.

In hindsight, this particular one may be the best thing since sliced bread. But that's survivorship bias.

Re: The Windows malloc() implementation from MSVCRT is slow

#155

Earlier quoted context omitted.

There's also UCRT, which ships with the OS since Windows 10. The logic of this rant was a real head-scratcher. If you must blame one side, it's LLVM. Fragmentation of C runtimes is annoying but inescapable. Glibc for example isn't any better.

Eh, no, this is strictly a Windows problem. On Windows every DLL can be statically linked each with its very private copy of the MSVCRT, which means -for example- that you'd better never ever pass one DLL's malloc()'ed memory pointers to another DLL's free(). On Unix systems (and Linux) this sort of thing can only ever happen if you have a statically-linked application that is linked with libdl and then it dlopen()s…

AIX, which happens to be a UNIX, has similar dynamic linking model.

Then we have the surviving mainframes and embedded RTOS, which aren't UNIXes either.

Re: The Windows malloc() implementation from MSVCRT is slow

#156
post #136

Earlier quoted context omitted.

Another reasonable choice for some components would be to have a purely C api; everything extern "C", and only PODs (plain old datastructures) in the arguments.

Exactly. Using a pure C interface escapes COM's requirement to register every component (regsvr32) and its overblown "GUID for everything" model, and its baroque way of creating components (somewhat alleviated by various macros and templates, but still). Making an object oriented interface is slightly cumbersome, but you can do it by sending back and forth a cookie (void* or int) of your object as the first parameter…

Not kept up with the times?

Registration free COM exists for several years now.

Re: The Windows malloc() implementation from MSVCRT is slow

#157
post #2

I wonder how much of this is the development culture at MS. https://www.theregister.com/2022/05/10/jeffrey_snover_said_m... (“When I was doing the prototype for what became PowerShell, a friend cautioned me saying that was the sort of thing that got people fired.”) In that environment I can imagine nobody wants to be on the hook for messing with something fundamental like malloc(). The complete trash fire that is O36…

(I work for MS, though in core Azure rather than Office or Windows.) I think that PowerShell story was how old MS worked, back in the days of stack ranking, hatred of Linux and the Longhorn fiasco. things inside the company are a lot more functional now. I saw internal politics drama at my first position, but once I moved everything was chill, and experimentation and contributing across team boundaries was actively e…

Looking how things have shaped through the years with WinRT, UWP, WinUI and related projects, at least WinDev seems to still be living in the past culture while trying to pretend to actually have changed.

The rate PMs change, how they lack understanding that we are fed up with rewrites, how they seem to believe we would still care and hope that in a couple of years WinUI 3.0 will actually be usable, how bugs in public repos accumulate,....

Re: The Windows malloc() implementation from MSVCRT is slow

#158
post #99

Earlier quoted context omitted.

> Fragmentation of C runtimes is annoying but inescapable. Glibc for example isn't any better. Glibc very much is better. There cannot be more than a single version of a (tightly coupled) ld.so+libc.so pair in a given address space any more than there can be more than a single KERNEL32 version in a given address space, and given that some system services are exclusively accessible via dynamic linking (accelerated gra…

Glibc is major source of "binary from X years ago doesn't work" in my experience, to the point that if I want to run stuff older than let's say 5 years I start by getting a docker container - or a specially taylored set of libs starting with dynamic linker itself. Somehow proprietary OpenGL drivers still worked, it was glibc related files that caused crashes. Thanks to symbol versioning I never know if a binary will…

Glibc 2.34 broke FreePascal. It calls some unversioned symbol for initialization that was removed

Re: The Windows malloc() implementation from MSVCRT is slow

#159
post #61

Earlier quoted context omitted.

(I work for MS, though in core Azure rather than Office or Windows.) I think that PowerShell story was how old MS worked, back in the days of stack ranking, hatred of Linux and the Longhorn fiasco. things inside the company are a lot more functional now. I saw internal politics drama at my first position, but once I moved everything was chill, and experimentation and contributing across team boundaries was actively e…

It was interesting the see them switch back to Win32 after all of these greenfield alternatives that quickly died. (WPF, WinRT, etc.) Makes you wonder what was going on during that time. Contrast Apple which has been with Cocoa which is an evolution of Next Step.

Kind of, Windows 11 WinRT components are still based on UWP, as WinUI 3.0 and WinAppSDK aren't yet up to the job of replacing it.

WinRT hasn't died, that is what WinUI 3.0/WinAppSDK is all about, making that COM infrastructure available on the Win32 side, even though their progress is quite slow.

I think it will take them 2 years still to reach feature parity with UWP features.

Re: The Windows malloc() implementation from MSVCRT is slow

#160

Earlier quoted context omitted.

"Unix" implies paying annual fees for use of the "Unix" trademark, and/or at least direct descent from the original Unix code. According to: https://kb.iu.edu/d/agat "To use the Unix trademark, an operating system vendor must pay a licensing fee and annual trademark royalties to The Open Group. Officially licensed Unix operating systems (and their vendors) include macOS (Apple), Solaris (Oracle), AIX (IBM), IRIX (SGI…

It should be noted that Linux generally does actually conform to UNIX standards, though, and that two actual Linux distributions (Inspur K-UX and EulerOS) have in the past obtained UNIX certification. While this doesn't make all Linux distributions UNIX certified, it puts a rather large dent in the claim that they cannot be qualified as UNIX because of some claimed divergence from the standards. (It also seems odd fr…

Only if their code for the certification is fully available upstream.
Post reply on HN