Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Windows malloc() implementation from MSVCRT is slow

erikmcclure.com

Post reply on HN
Print view View original

101–110 of 187 posts

Re: The Windows malloc() implementation from MSVCRT is slow

#101

Post by Dylan16807 »

oddity wrote: post #91
dzaima wrote: post #64

Earlier quoted context omitted.

Except that I'd guess that there is no "good" case in the case for MSVCRT's malloc. You shouldn't assume malloc is free, but you should also be able to assume it won't be horrifyingly slow. Just as much as you should be able to rely on "x*y" not compiling to an addition loop over 0..y (which might indeed be very fast when y is 0). Yes, this unfortunately isn't the reality MSVCRT is in, but it is quite a reasonable ex…

It's unreasonable to assume that an stdlib must be designed around performance to any capacity. For most software, the priorities for the stdlib are 1) existing, 2) being bug/vulnerability free, and likely, in the Windows case given Microsoft's tradition, 3) being functionally identical to the version they shipped originally. Linux and macOS have much more flexibility to choose a different set of priorities (the form…

> It's unreasonable to assume that an stdlib must be designed around performance to any capacity.

To any capacity? That's insane.

It's not a reference implementation to show you what the correct results should be. It's the standard. The default.

Re: The Windows malloc() implementation from MSVCRT is slow

#102

Post by wvenable »

rayiner wrote: post #61
sterlind wrote: post #37

Earlier quoted context omitted.

(I work for MS, though in core Azure rather than Office or Windows.) I think that PowerShell story was how old MS worked, back in the days of stack ranking, hatred of Linux and the Longhorn fiasco. things inside the company are a lot more functional now. I saw internal politics drama at my first position, but once I moved everything was chill, and experimentation and contributing across team boundaries was actively e…

It was interesting the see them switch back to Win32 after all of these greenfield alternatives that quickly died. (WPF, WinRT, etc.) Makes you wonder what was going on during that time. Contrast Apple which has been with Cocoa which is an evolution of Next Step.

Apple wanted to force Cocoa on everyone but had to delay their OS an entire year to build Carbon so that developers would actually port their Mac apps to OS X.

Re: The Windows malloc() implementation from MSVCRT is slow

#103

Post by Dylan16807 »

dzaima wrote: post #95
oddity wrote: post #91

Earlier quoted context omitted.

It's unreasonable to assume that an stdlib must be designed around performance to any capacity. For most software, the priorities for the stdlib are 1) existing, 2) being bug/vulnerability free, and likely, in the Windows case given Microsoft's tradition, 3) being functionally identical to the version they shipped originally. Linux and macOS have much more flexibility to choose a different set of priorities (the form…

I don't see a reason for the stdlib to be considered in a different way from the base language is all I'm saying. For most C programmers, the distinction between the stdlib and the base language isn't even a consideration. Thinking most software doesn't heavily rely on malloc (and the rest of the stdlib) being fast is stupid. Even on hardware without a multiplier you'd do a shift-based version, with log_2(max_value)…

> If there truly were no way to do a shift, then, sure, I'd accept the loop

Not even then. You can just use an addition instead of a shift.

Re: The Windows malloc() implementation from MSVCRT is slow

#104

Post by oddity »

dzaima wrote: post #95
oddity wrote: post #91

Earlier quoted context omitted.

It's unreasonable to assume that an stdlib must be designed around performance to any capacity. For most software, the priorities for the stdlib are 1) existing, 2) being bug/vulnerability free, and likely, in the Windows case given Microsoft's tradition, 3) being functionally identical to the version they shipped originally. Linux and macOS have much more flexibility to choose a different set of priorities (the form…

I don't see a reason for the stdlib to be considered in a different way from the base language is all I'm saying. For most C programmers, the distinction between the stdlib and the base language isn't even a consideration. Thinking most software doesn't heavily rely on malloc (and the rest of the stdlib) being fast is stupid. Even on hardware without a multiplier you'd do a shift-based version, with log_2(max_value)…

C and C++ are some of the few languages where the spec goes out of its way to not depend on an allocator, for good reason, and this is well after you've accounted for the majority of the code that, hopefully, doesn't need to do memory allocation at all. The fact that many programmers don't care is an indication that most code in most C or C++ software is not written with performance in mind. And that's (sometimes) fine. LLVM has a good ecosystem reason to use C++, for example, and it's well known in the compiler space that LLVM is not fast. Less recently, for a long time C and C++ were considered high level languages, meaning lots of software was written in it without consideration of performance. But criticizing the default implementation's performance absent a discussion of its priorities when you have all the power to not be bottlenecked in it anyway is just silly.

Re: The Windows malloc() implementation from MSVCRT is slow

#105

Post by assttoasstmgr »

rayiner wrote: post #2

I wonder how much of this is the development culture at MS. https://www.theregister.com/2022/05/10/jeffrey_snover_said_m... (“When I was doing the prototype for what became PowerShell, a friend cautioned me saying that was the sort of thing that got people fired.”) In that environment I can imagine nobody wants to be on the hook for messing with something fundamental like malloc(). The complete trash fire that is O36…

> people protecting fiefdoms than delivering the best product.

Apparently you have never worked at any company with more than 50 employees because that's literally how every large company works. Career-obsessed managers who can't see the forest for the trees not giving a single shit about the overall product as long as their goals were met. They're off to the next promotion before all hell breaks loose.

Re: The Windows malloc() implementation from MSVCRT is slow

#106

Post by legalcorrection »

sterlind wrote: post #29
bjourne wrote: post #20

Well... Who told you to link to MSVCRT (the one in System32)? Not Microsoft that's for sure. New software is supposed to link to the Visual Studio C runtime it was compiled with and then ship that library alongside the application itself. Even if you don't compile with VS you can distribute the runtime library (freely downloadable from some page on microsoft.com). Ostensibly, that library contains an efficient malloc…

there's a weird licensing thing for the VC runtime where you can't redistribute the dll alongside your code unless you have a special license. instead, you have to install it as an MSI. I have no idea why they created that restriction. I even work for MS and it baffles me.

It’s so the system can update the runtime when security vulnerabilities and such are discovered.

Re: The Windows malloc() implementation from MSVCRT is slow

#107

Post by assttoasstmgr »

wvenable wrote: post #102
rayiner wrote: post #61

Earlier quoted context omitted.

It was interesting the see them switch back to Win32 after all of these greenfield alternatives that quickly died. (WPF, WinRT, etc.) Makes you wonder what was going on during that time. Contrast Apple which has been with Cocoa which is an evolution of Next Step.

Apple wanted to force Cocoa on everyone but had to delay their OS an entire year to build Carbon so that developers would actually port their Mac apps to OS X.

A number of Apple's original core apps (like iTunes, AppleWorks, etc.) were Carbonized apps.

Re: The Windows malloc() implementation from MSVCRT is slow

#108

Post by skissane »

sterlind wrote: post #40
munch117 wrote: post #30

Earlier quoted context omitted.

You shouldn't read too much into the PowerShell story. Creating your own programming language is in most cases a frivolous vanity project. Spending company resources on your own frivolous vanity projects is the sort of thing that can get you fired.

! I disagree. CMD badly needed replacing. MS needed a new shell language. A functional company would connect people with a passion for X with the resources to achieve X, if X has a chance of helping the company. Windows Terminal and WSL show how far MS has come from the PS days. (Disclaimer: I work for MS)

> CMD badly needed replacing

CMD is nasty but there are lots of little ways in which it could have been improved. For example, provide an option to disable that useless "Terminate batch job (Y/N)?" prompt.

I wish Microsoft would open-source CMD.EXE. I dislike how slow PowerShell is (especially its startup). Maybe nobody at Microsoft cares about CMD.EXE enough to fix those long-standing little annoyances like the above, but if it was open-source other people out there would.

Also, I wonder why nobody ever seemed to have thought of integrating CSCRIPT into CMD, so that you could have seamlessly mixed VBScript (or other WSH languages) into batch files.

Re: The Windows malloc() implementation from MSVCRT is slow

#109

Post by mananaysiempre »

p_l wrote: post #99
mananaysiempre wrote: post #84

Earlier quoted context omitted.

> Fragmentation of C runtimes is annoying but inescapable. Glibc for example isn't any better. Glibc very much is better. There cannot be more than a single version of a (tightly coupled) ld.so+libc.so pair in a given address space any more than there can be more than a single KERNEL32 version in a given address space, and given that some system services are exclusively accessible via dynamic linking (accelerated gra…

Glibc is major source of "binary from X years ago doesn't work" in my experience, to the point that if I want to run stuff older than let's say 5 years I start by getting a docker container - or a specially taylored set of libs starting with dynamic linker itself. Somehow proprietary OpenGL drivers still worked, it was glibc related files that caused crashes. Thanks to symbol versioning I never know if a binary will…

> Given that glibc backward compatibility for all practical purposes is shorter than 10 years

As far as I’m able to tell[1], Glibc has never removed a versioned symbol as long as symbol versioning has existed, so the intended backwards compatibility is “forever”. No binary built against a(n unpatched) Glibc should fail to load on a later Glibc due to an absent Glibc symbol version. (Of course, if you play non-version-aware dynamic linker and dlsym() not dlvsym() things in libc, you lose; if you bundle an “updated” part of libc with the application[2], you lose; etc., so this may be true but still not entirely foolproof. And given how many things there are in libc, if you’re going to lose it’s probably going to be there.)

[1] E.g. https://sourceware.org/git/?p=glibc.git;a=history;f=stdlib/V...

[2] https://news.ycombinator.com/item?id=29479903

Re: The Windows malloc() implementation from MSVCRT is slow

#110

Post by password4321 »

p_l wrote: post #99
mananaysiempre wrote: post #84

Earlier quoted context omitted.

> Fragmentation of C runtimes is annoying but inescapable. Glibc for example isn't any better. Glibc very much is better. There cannot be more than a single version of a (tightly coupled) ld.so+libc.so pair in a given address space any more than there can be more than a single KERNEL32 version in a given address space, and given that some system services are exclusively accessible via dynamic linking (accelerated gra…

Glibc is major source of "binary from X years ago doesn't work" in my experience, to the point that if I want to run stuff older than let's say 5 years I start by getting a docker container - or a specially taylored set of libs starting with dynamic linker itself. Somehow proprietary OpenGL drivers still worked, it was glibc related files that caused crashes. Thanks to symbol versioning I never know if a binary will…

For example: https://github.com/microsoft/vscode-remote-release/issues/10...
Post reply on HN