I you calloc some memory and then the first thing you do is write to it, can the compiler optimize away the initial write of zeros since they will just be overwritten?
[deleted]
Why does calloc exist?
21–30 of 141 posts
Re: Why does calloc exist?
#22It's harder to forget to multiply by sizeof(T) when calloc-ing as well.
Re: Why does calloc exist?
#23> But calloc lives inside the memory allocator, so it knows whether the memory it's returning is fresh from the operating system, and if it is then it skips calling memset. And this is why calloc has to be built into the standard library, and you can't fake it yourself. Err...mmap(2)?
You can't fake it yourself from within the C stdlib is what the author means.
Re: Why does calloc exist?
#24> Plus, if we wanted to, we could certainly write our own wrapper for malloc that took two arguments and multiplied them together with overflow checking. And in fact if we want an overflow-safe version of realloc, or if we don't want the memory to be zero-initialized, then... we still have to do that. Like reallocarray(3) does? buf = malloc(x * y); // becomes buf = reallocarray(NULL, x, y); newbuf = realloc(buf, (x *…
reallocarray(3) looks nifty, but until it's available on a wider, ideally more standard-driven basis than just OpenBSD and FreeBSD, it's likely to not see wide uptake.
Re: Why does calloc exist?
#25Earlier quoted context omitted.
Best to change your design to leverage a long-lived resource if possible. On the flip side, if your critical metric is latency then these tricks [...] are exactly what you try to avoid If you keep the buffer alive as long as possible with a slab allocator, or just smart/good memory management strategy. How you acquire the buffer will ultimately be trivial, likely dwarfed your other startup tasks (reading config, open…
I think he was referring to the part where a calloc'd memory page will be zeroed the first time it's used, rather than all at once at the beginning. In a real-time system, the start-up time matters less than having predictable response times.
In a realtime system you can't use virtual memory because the access times are unpredictable.
Re: Why does calloc exist?
#26I always thought it was because of padding. An array of M structures each N bytes long could require more than M*N bytes (certainly has on some architectures I've worked with). But I guess that's not it after all.
C accounts for padding in the size of the individual type. By the time you do sizeof, it's already rounded up to where you can just do M*N. For example: struct S { long a; char b; }; On my computer (64-bit Mac), sizeof(struct S) is 16, due to 7 bytes of padding after b. Since the compiler handles the padding, that means calloc doesn't have to.
Which can lead to sadness if you're not careful with #pragma pack and the like.
Re: Why does calloc exist?
#27Earlier quoted context omitted.
I think there's a big difference between "why does ... exist" and "why does ... still exist". calloc() may be useful today for reasons completely different from why it existed in the first place. And the difference #2 is just an implementation-specific optimisation. There's nothing in the standard that forces calloc to use lazy allocation / virtual memory. Actually, it may be implemented on platforms which can't prov…
Thank you for bringing up the implementation-specific nature of #2! If the author is running Linux, then perhaps they've never checked out overcommit vs. not overcommit.
You're likely to still see a performance improvement without overcommit. The OS will try to zero free pages in the background, so there's a good chance that it'll have pre-zeroed pages to hand you when you ask for them, rather than making you wait for them to be zeroed on demand.
Of course, there are plenty of systems where this doesn't happen, or there are no pages in the first place, or there's no kernel zeroing stuff for you.
Re: Why does calloc exist?
#28That's a nice alternative history fiction. Here's an early implementation: https://github.com/dspinellis/unix-history-repo/blob/Researc...
Good point, and it makes we wonder why the code is attributed to dmr...it looks like written by ken. I suppose recreating such a repo can only be so accurate.
Re: Why does calloc exist?
#29No, the 2 GB array should still take a quota of 2 GB. It just wouldn't take 2 GB's worth of time to initialize. The overcommit "feature" in Linux is a bug that crashes C programs in places that violate the language's guarantees (such as when a write occurs to a location in memory that was allocated correctly).
Re: Why does calloc exist?
#30Sorry, but this is just goofy and bad.
If you depend on copy-on-write functionality, then you need to use an API that is specced to guarantee copy-on-write functionality. If that means you use an #ifdef per platform and do OS-specific stuff, then that is what you do.
Anything else is amateur hour.
If copy-on-write is a desirable feature, then as the API creator, your job is to expose this functionality in the clearest and simplest way possible, not to hack it in obscurely via the implementation details of some random routine. (And then surprise people who didn't expect copy-on-write with the associated performance penalties.)
This is why we can't have nice things.