Live data from Hacker News

Why does calloc exist?

vorpus.org

91–100 of 141 posts

Re: Why does calloc exist?

#91
post #86
post #82

Earlier quoted context omitted.

That's even stronger case for not relying on errno to catch errors. The code should be something like this: buf = calloc(huge, huge); if (!buf) perror("calloc failed"); printf("calloc(huge, huge) returned: %p\n", buf); free(buf);

Yes, my post above was agreeing that you should check the error condition of the return value, rather than relying on errno to be cleared on success.

Ah, my bad.

Re: Why does calloc exist?

#92
post #33
post #3

That's a nice alternative history fiction. Here's an early implementation: https://github.com/dspinellis/unix-history-repo/blob/Researc...

You haven't proven it wrong. Here's the earliest implementation in that repo (in Research UNIX V6; your link in V7): https://github.com/dspinellis/unix-history-repo/blob/Researc... calloc(n, s) { return(alloc(n*s)); } There are several interesting things we learn from poking around V6 though: - `calloc` originated not on UNIX, but as part of Mike Lesk's "iolib", which was written to make it easier to write C programs…

So the (slightly modified) question still stands: Why do calloc and malloc exist? Indeed it looks like calloc was originally intended as a portable way to allocate memory. It used the function alloc which apparently was not meant to be used directly; most iolib functions have a 'c' tacked on. So when iolib was reworked into the stdlib why was calloc kept? saretired suspects backward compatibility but I don't believe this, because no other c-prefixed iolib function was kept and i couldn't find any code that actually used calloc in the v6 distribution either. So maybe whoever is responsible for malloc/calloc in v7 (I think it was ken, not dmr) thought malloc should be a public function but saw a use for calloc and changed the semantics to be a bit more predictable.

Re: Why does calloc exist?

#93
post #64

Earlier quoted context omitted.

CPUs are pretty great at running nonmagical for loops, so you'd have be zeroing a pretty giant block of memory before it made sense to get the OS involved at all.

Of course :) But the OS could get involved for larger blocks of memory. Also, I wonder if zeroing large chunks of memory would be faster to do in kernel space using real addresses. You can avoid the multiple real memory lookups involved in a single virtual write. (Of course, we already avoid those often, but it could be useful to avoid entirely. Not sure what the tradeoffs are here)

For linear access patterns the TLB does its job perfectly. The overhead is negligible.

Re: Why does calloc exist?

#94
Originally, calloc was the function Unix programmers were expected to use by default, since it avoids any sort of intermittent bugs due to your forgetting to initialize some field in the data structure you're allocating. But clearing the memory to all zeros took precious time, so if you were an advanced programmer, and knew for a fact that you were going to fill it all in yourself, you could optimize by calling malloc.

Re: Why does calloc exist?

#95
post #30

Sorry, but this is just goofy and bad. If you depend on copy-on-write functionality, then you need to use an API that is specced to guarantee copy-on-write functionality. If that means you use an #ifdef per platform and do OS-specific stuff, then that is what you do. Anything else is amateur hour. If copy-on-write is a desirable feature, then as the API creator, your job is to expose this functionality in the cleares…

>Anything else is amateur hour. Such as writing the optimizing compilers that make it feasible for you to use C at all?

Well his argument is that if you need e.g. CoW you shouldn't rely on the OS doing that implicitly for you and instead you should explicitly use the CoW features of the OS.

There should be a way to explicitly demand the optimization instead of relying on the behavior of a specific compiler to implicitly optimize the code.

Shifting beyond the bitwidth is undefined behavior. Why can't shifts be checked by default. Let me explicitly demand the undefined behavior when I really need it.

Re: Why does calloc exist?

#96
post #33

Earlier quoted context omitted.

You haven't proven it wrong. Here's the earliest implementation in that repo (in Research UNIX V6; your link in V7): https://github.com/dspinellis/unix-history-repo/blob/Researc... calloc(n, s) { return(alloc(n*s)); } There are several interesting things we learn from poking around V6 though: - `calloc` originated not on UNIX, but as part of Mike Lesk's "iolib", which was written to make it easier to write C programs…

OpenBSD added calloc overflow checking on July 29th, 2002. glibc added calloc overflow checking on August 1, 2002. Probably not a coincidence. I'm going to say nobody checked for overflow prior to the August 2002 security advisory. https://github.com/openbsd/src/commit/c7b2af4b3f7e78424f8943... https://github.com/bminor/glibc/commit/0950889b810736fe7ad34... http://cert.uni-stuttgart.de/ticker/advisories/calloc.html

[deleted]

Re: Why does calloc exist?

#97
post #32

buf = calloc(huge, huge); if (errno) perror("calloc failed"); printf("calloc(huge, huge) returned: %p\n", buf); free(buf); This has a flaw. errno doesn't magically get reset to zero. You should check the return value of calloc, then use errno. Checking if(errno) is not the right way to determine if there was an error.

wow, yeah, that's totally and obviously buggy ...

Re: Why does calloc exist?

#98
post #40

Earlier quoted context omitted.

It's already gaining adoption outside of the BSDs. OS X/iOS seem to have it as part of their libmalloc. Android Bionic libc has it as part of the code they sync from upstream OpenBSD. Many open source projects include their own, or simply bundle the OpenBSD implementation: * mandoc * flex * unbound and nsd * tor * tmux * libbsd * libressl * xorg-xserver * ... The list only continues to grow, several more examples to…

There's code for it in Darwin's libmalloc, but it's not exposed as API. reallocarray() has some difficulties as an interface, mostly inherited from realloc(). I'm a bigger fan of reallocarr(), but that's NetBSD only. We (the operating systems community) need to find a consensus here, but I'm not convinced that reallocarray() is that consensus yet.

reallocarray is a very thin layer around realloc. No surprises there. Simple find and replace to bring overflow checking into your code.

reallocarr changes the semantics. Equally easy in new pieces of code and a little harder when converting existing code.

Re: Why does calloc exist?

#99
post #40

Earlier quoted context omitted.

It's already gaining adoption outside of the BSDs. OS X/iOS seem to have it as part of their libmalloc. Android Bionic libc has it as part of the code they sync from upstream OpenBSD. Many open source projects include their own, or simply bundle the OpenBSD implementation: * mandoc * flex * unbound and nsd * tor * tmux * libbsd * libressl * xorg-xserver * ... The list only continues to grow, several more examples to…

There's code for it in Darwin's libmalloc, but it's not exposed as API. reallocarray() has some difficulties as an interface, mostly inherited from realloc(). I'm a bigger fan of reallocarr(), but that's NetBSD only. We (the operating systems community) need to find a consensus here, but I'm not convinced that reallocarray() is that consensus yet.

[deleted]

Re: Why does calloc exist?

#100
post #33

Earlier quoted context omitted.

You haven't proven it wrong. Here's the earliest implementation in that repo (in Research UNIX V6; your link in V7): https://github.com/dspinellis/unix-history-repo/blob/Researc... calloc(n, s) { return(alloc(n*s)); } There are several interesting things we learn from poking around V6 though: - `calloc` originated not on UNIX, but as part of Mike Lesk's "iolib", which was written to make it easier to write C programs…

OpenBSD added calloc overflow checking on July 29th, 2002. glibc added calloc overflow checking on August 1, 2002. Probably not a coincidence. I'm going to say nobody checked for overflow prior to the August 2002 security advisory. https://github.com/openbsd/src/commit/c7b2af4b3f7e78424f8943... https://github.com/bminor/glibc/commit/0950889b810736fe7ad34... http://cert.uni-stuttgart.de/ticker/advisories/calloc.html

It is embarrassing for glibc not to check for overflow in calloc implementation prior to 2002. It is not only a security flaw but also violation of C Standards (even the first version ratified in 1989, usually referred to as C89).

The standard reads as follows:

  void *calloc(size_t nmemb, size_t size);

  The calloc function allocates space for an array of nmemb objects, each of whose size is size.[...]
and,

  The calloc function returns either a null pointer or a pointer to the allocated space.
So if it cannot allocate space for an array of nmemb objects, each of whose size is size, then it has to return null pointer.
Post reply on HN