Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why does calloc exist?

vorpus.org

Post reply on HN
Print view View original

11–20 of 141 posts

Re: Why does calloc exist?

#11

Post by rcfox »

valarauca1 wrote: post #6
wyldfire wrote: post #2

> So basically, calloc exists because it lets the memory allocator and kernel engage in a sneaky conspiracy to make your code faster and use less memory. You should let it! Don't use malloc+memset! On the flip side, if your critical metric is latency then these tricks of calloc's and the OS's are exactly what you try to avoid. memset() the buffer, and if you have the privileges you should mlock() it to prevent it fro…

Best to change your design to leverage a long-lived resource if possible. On the flip side, if your critical metric is latency then these tricks [...] are exactly what you try to avoid If you keep the buffer alive as long as possible with a slab allocator, or just smart/good memory management strategy. How you acquire the buffer will ultimately be trivial, likely dwarfed your other startup tasks (reading config, open…

I think he was referring to the part where a calloc'd memory page will be zeroed the first time it's used, rather than all at once at the beginning.

In a real-time system, the start-up time matters less than having predictable response times.

Re: Why does calloc exist?

#12

Post by ben_bai »

> Plus, if we wanted to, we could certainly write our own wrapper for malloc that took two arguments and multiplied them together with overflow checking. And in fact if we want an overflow-safe version of realloc, or if we don't want the memory to be zero-initialized, then... we still have to do that.

Like reallocarray(3) does?

    buf = malloc(x * y);
    // becomes
    buf = reallocarray(NULL, x, y);
    
    newbuf = realloc(buf, (x * y));
    // becomes
    newbuf = reallocarray(buf, x, y);

Re: Why does calloc exist?

#13

Post by viraptor »

colanderman wrote: post #5
bluefox wrote: post #3

That's a nice alternative history fiction. Here's an early implementation: https://github.com/dspinellis/unix-history-repo/blob/Researc...

It's a good explanation of why calloc still exists and is useful. Otherwise it would have been dropped from the standard like cfree was.

I think there's a big difference between "why does ... exist" and "why does ... still exist". calloc() may be useful today for reasons completely different from why it existed in the first place. And the difference #2 is just an implementation-specific optimisation. There's nothing in the standard that forces calloc to use lazy allocation / virtual memory. Actually, it may be implemented on platforms which can't provide this.

Re: Why does calloc exist?

#14

Post by stouset »

notacoward wrote: post #7

I always thought it was because of padding. An array of M structures each N bytes long could require more than M*N bytes (certainly has on some architectures I've worked with). But I guess that's not it after all.

This would be very, very bad for performance in a lot of cases do to non-aligned struct reads, were it true.

Re: Why does calloc exist?

#16

Post by greglindahl »

viraptor wrote: post #13
colanderman wrote: post #5

Earlier quoted context omitted.

It's a good explanation of why calloc still exists and is useful. Otherwise it would have been dropped from the standard like cfree was.

I think there's a big difference between "why does ... exist" and "why does ... still exist". calloc() may be useful today for reasons completely different from why it existed in the first place. And the difference #2 is just an implementation-specific optimisation. There's nothing in the standard that forces calloc to use lazy allocation / virtual memory. Actually, it may be implemented on platforms which can't prov…

Thank you for bringing up the implementation-specific nature of #2! If the author is running Linux, then perhaps they've never checked out overcommit vs. not overcommit.

Re: Why does calloc exist?

#17

Post by radarsat1 »

rcthompson wrote: post #15

I you calloc some memory and then the first thing you do is write to it, can the compiler optimize away the initial write of zeros since they will just be overwritten?

I would imagine this might work in cases where the compiler can prove that all addresses are written, but probably this is a limited number of cases. Likely it would do so simply by noticing two successive writes to the same location, instead of doing anything special related to calloc.

Re: Why does calloc exist?

#19

Post by smegel »

> But calloc lives inside the memory allocator, so it knows whether the memory it's returning is fresh from the operating system, and if it is then it skips calling memset. And this is why calloc has to be built into the standard library, and you can't fake it yourself.

Err...mmap(2)?

Re: Why does calloc exist?

#20

Post by Serplat »

rcthompson wrote: post #15

I you calloc some memory and then the first thing you do is write to it, can the compiler optimize away the initial write of zeros since they will just be overwritten?

For smaller memory allocations that don't go directly to the OS, I suppose it's theoretically possible (though I'm not sure any compilers do it). For the larger allocations that the author mentions that go directly to the OS to fulfill, however, the compiler wouldn't be able to optimize this away. In that case, the zero'ing occurs in the kernel, which is something that the compiler has no control over.

The zero'ing is done significantly for security reasons anyway. If a program could somehow disable that feature and get leftover memory from the kernel it could very easily contain password, secret keys, and other important bits of data that you wouldn't want random programs on your computer to have access to.

Post reply on HN