Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Windows malloc() implementation from MSVCRT is slow

erikmcclure.com

Post reply on HN
Print view View original

71–80 of 187 posts

Re: The Windows malloc() implementation from MSVCRT is slow

#71

Post by 13of40 »

sterlind wrote: post #40
munch117 wrote: post #30

Earlier quoted context omitted.

You shouldn't read too much into the PowerShell story. Creating your own programming language is in most cases a frivolous vanity project. Spending company resources on your own frivolous vanity projects is the sort of thing that can get you fired.

! I disagree. CMD badly needed replacing. MS needed a new shell language. A functional company would connect people with a passion for X with the resources to achieve X, if X has a chance of helping the company. Windows Terminal and WSL show how far MS has come from the PS days. (Disclaimer: I work for MS)

The way I remember it, the need for a new shell language for system administration was something that lots of people in Windows Server were trying to solve. Ballmer talked about it, we had a push to add a handful of new command line tools (like tasklist.exe I think) that you could use under CMD, and there was a proof of concept where MMC could be used to output some kind of macro language when users did things in the UI. PowerShell was the thing that eventually won, and I think it was largely because it stood on the shoulders of .Net so had a ton of capability right out of the gate. (And TBH, I think it's a little bit weird that we have this mythos today where Snover sat down at his computer one morning and invented it out of thin air, when even the v1 feature team had something like 30 engineers and PMs on it.)

Re: The Windows malloc() implementation from MSVCRT is slow

#72

Post by kazinator »

TonyTrapp wrote: post #47
jart wrote: post #33

Earlier quoted context omitted.

It's effectively mandatory. Microsoft provides about twelve different C Runtimes. But if you're building something like an open source library, you can't link two different C runtimes where you might accidentally malloc() memory with one and then free() with the other. If you want to be able to pass pointers around your dynamic link libraries, you have to link the one C runtime everyone else uses, which is MSVCRT. Al…

It isn't mandatory. I have never actively linked against MSVCRT on Windows. From my experience it's mostly software that isn't built with Visual Studio that uses MSVCRT, or software that that takes extreme care of its binary size (e.g. 64k intros). MSVCRT is not even an up-to-date C runtime library. You wouldn't be able to use it for writing software requiring C11 library features without implementing them somewhere…

If you're writing in C++ on Windows, expose only COM (or at least COM-style) interface called through virtual functions on an object pointer. Then you can use whatever C++ run-time you want, internally. What you don't want is the other library calling C++ functions by name. Like you pass it some ostream object and it calls ostream::put or whatever, where that symbolically resolves to the wrong one.

Re: The Windows malloc() implementation from MSVCRT is slow

#73

Post by garaetjjte »

TonyTrapp wrote: post #56
garaetjjte wrote: post #51

Earlier quoted context omitted.

>but Microsoft never considered the MSVCRT that ships with Windows to be public API It was in the past. At first msvcrt.dll was the runtime library used up to Visual C++ 6. Later, VC++ moved to their own separate dlls, but you could still link with system msvcrt.dll using corresponding DDK/WDK up to Windows 7. I'm also not sure that this is just ancient library left for compatibility, some system components still lin…

> It was in the past. At first msvcrt.dll was the runtime library used up to Visual C++ 6. At that time it was already a big mess, because at first it was the runtime library of Visual C++ 4 in fact! The gory details are here: https://devblogs.microsoft.com/oldnewthing/20140411-00/?p=12... > some system components still link to it Some system components themselves are very much ancient and unmaintained and only exist…

Ancient or not, I don't think it really matters for allocation performance: malloc in both msvcrt.dll and ucrtbase.dll after some indirection ends up calling RtlAllocateHeap in ntdll.dll

Re: The Windows malloc() implementation from MSVCRT is slow

#74

Post by chrisseaton »

pjmlp wrote: post #36
jart wrote: post #27

Earlier quoted context omitted.

malloc() isn't part of the Linux API which provides mmap().

Since we are getting pedantic, Linux isn't a UNIX.

> Linux isn't a UNIX

I think this isn't quite right - I think some distributions are actually certified as UNIX.

https://www.opengroup.org/openbrand/register/

Re: The Windows malloc() implementation from MSVCRT is slow

#75

Post by evmar »

The other inaccuracies in this article have already been covered. I noticed there was also a weird rant about mimalloc in there ("For some insane reason, mimalloc is not shipped in Visual Studio").

My understanding is mimalloc is basically a one-person project[1] from an MSR researcher in support of his research programming languages. It sounds like it's pretty nice, but I also wouldn't expect it to be somehow pushed as the default choice for Windows allocators.

[1]: https://github.com/microsoft/mimalloc/graphs/contributors

Re: The Windows malloc() implementation from MSVCRT is slow

#76

Post by naniwaduni »

leajkinUnk wrote: post #70
plorkyeran wrote: post #39

Earlier quoted context omitted.

Libc being just a library is indeed one of the ways that Linux is unlike Unix.

What do you mean by "Unix"? Are you talking about some specific Unix version, or is there something in the POSIX spec that says that libc isn't a library?

It's not that libc is supposed to not be a library, but those functions are the POSIX-defined interfaces to the OS. Linux is unusual in that it defines its stable interfaces in terms of the syscall ABI, enabling different implementations of the libc that can work semi-reliably across kernel versions.

Re: The Windows malloc() implementation from MSVCRT is slow

#77

Post by spatulon »

moonchild wrote: post #18

> it basically represents control flow as a gigantic DAG Control flow is not a DAG.

You're not wrong.

I guess they're just trying to say that LLVM's control-flow graph is implemented as individually heap-allocated objects for nodes, and pointers for edges. (I haven't looked at the LLVM code, but that sounds plausible).

Even if those allocations are fast on Linux/Mac, I wonder whether there are other downsides of that representation, for example in terms of performance issues from cache misses when walking the graph. Could you do better, e.g. with a bump allocator instead of malloc? But who knows, maybe graph algorithms are just inherently cache-unfriendly, no matter the representation.

Re: The Windows malloc() implementation from MSVCRT is slow

#78

Post by usrn »

leajkinUnk wrote: post #68
FreakLegion wrote: post #35

Earlier quoted context omitted.

There's also UCRT, which ships with the OS since Windows 10. The logic of this rant was a real head-scratcher. If you must blame one side, it's LLVM. Fragmentation of C runtimes is annoying but inescapable. Glibc for example isn't any better.

Could you elaborate why Glibc isn't any better? I remember some funny problems with Glibc, like, 20 years ago, but it's been invisible to me (as a user) since then. You get a new Glibc, old binaries still work, it's fine.

I'm pretty sure I've run into binaries breaking on new versions of Glibc but maybe it's because the architecture or calling convention changed. I've never really gotten the sense that GNU cares much about binary compatibility (which makes sense, they argue that sharing binaries is mostly counter productive.)

Re: The Windows malloc() implementation from MSVCRT is slow

#79

Post by garaetjjte »

softwaredoug wrote: post #11

My knowledge is like 10 years old - For a long time, Microsoft's stl implementation was based on their licensning of dinkumware's STL ( https://www.dinkumware.com/ ). Not something maintained in house. It seemed to work OK'ish - giving lowest common denominator functionality. However, it was pretty easy to create higher performing specialized data structures for your use case then what seemed like simple uses of dink…

malloc is not related to STL. But about it, big issue with Microsoft STL is that it is atrociously slow on debug builds.
Post reply on HN