Live data from Hacker News

Why does calloc exist?

vorpus.org

41–50 of 141 posts

Re: Why does calloc exist?

#41
post #4

The real reason "calloc" exists was that it was really easy to hit 16-bit overflow back in the PDP-11 days.

Historically, not quite true.

No version of Research UNIX V1 through V7, nor any of BSD 1, 2, 3, 4, or 4.4 did overflow checking. They all just did `m * n` or `m *= n`.

Re: Why does calloc exist?

#42
post #41
post #4

The real reason "calloc" exists was that it was really easy to hit 16-bit overflow back in the PDP-11 days.

Historically, not quite true. No version of Research UNIX V1 through V7, nor any of BSD 1, 2, 3, 4, or 4.4 did overflow checking. They all just did `m * n` or `m *= n`.

If you look through the history of CVEs, you'll find that pretty much every implementation of calloc or a calloc-like function starts with m * n and ends up only changing after someone points out the security flaw.

Re: Why does calloc exist?

#43
post #40

Earlier quoted context omitted.

reallocarray(3) looks nifty, but until it's available on a wider, ideally more standard-driven basis than just OpenBSD and FreeBSD, it's likely to not see wide uptake.

It's already gaining adoption outside of the BSDs. OS X/iOS seem to have it as part of their libmalloc. Android Bionic libc has it as part of the code they sync from upstream OpenBSD. Many open source projects include their own, or simply bundle the OpenBSD implementation: * mandoc * flex * unbound and nsd * tor * tmux * libbsd * libressl * xorg-xserver * ... The list only continues to grow, several more examples to…

Darwin (macOS / iOS) is often counted as "one of the BSDs", just a fairly weird one. Big chunks of the standard library are copied from FreeBSD with changes to work on top of Mach.

Re: Why does calloc exist?

#44
There are some unfortunate statements in there (if taken out of context) that requires you to read the whole thing for it to make sense. Like "...but most of the array is still zeros, so it isn't actually taking up any memory..." which is a bit ambigious if not read in the complete context, then it makes sense.

Re: Why does calloc exist?

#45

Earlier quoted context omitted.

In a realtime system you can't use virtual memory because the access times are unpredictable.

I specifically avoided that word because it triggers particular deadlines that people have in mind. If my application requires no more than X ms latency I don't care to handwring over realtime vs soft realtime vs whatever, but it's still critical to fit in the budget. But indeed you can get reliable low-latency products to work on linux, with virtual memory. But like I said pinning is a great way to keep those peaks…

The division you are looking for is

Hard Realtime: Embedded system, no virtual memory/OS. Or special OS provisions to let them run.

Soft Realtime: Responsive.

In the case you are aiming for the second. There are several million things that'll net greater performance. We're talking about saving a matter of nano-seconds in C/C++. How you load your config will have more effect then this.

If you want to save $1,000,000 rolling pennies a start. But there are likely way bigger savings elsewhere, worth way more time, and less effort.

Re: Why does calloc exist?

#46
This is a great example of why _alloc is an abstraction over virtual memory.

What this doesn't express is that dealing with page allocation directly can be quite annoying to get correct cross platform. You generally don't want to do that unless a) you're optimizing past the "knuth level" and know you need to for performance (e.g. mapping files to memory), b) you're writing something where you run dynamic code (JIT or dynamic recompilation) or c) you're writing your own allocator and/or using page faults to get some functionality, ala Go's stop-the-world hack.

Basically, don't bypass _alloc unless you have a reason.

Re: Why does calloc exist?

#47
post #36
post #30

Sorry, but this is just goofy and bad. If you depend on copy-on-write functionality, then you need to use an API that is specced to guarantee copy-on-write functionality. If that means you use an #ifdef per platform and do OS-specific stuff, then that is what you do. Anything else is amateur hour. If copy-on-write is a desirable feature, then as the API creator, your job is to expose this functionality in the cleares…

Neither copy on write nor size checking are specced as part of the calloc() definition. Here's the specification of calloc from the ISO standard: 7.22.3.2 The calloc function Synopsis #include void *calloc(size_t nmemb, size_t size); Description The calloc function allocates space for an array of nmemb objects, each of whose size is size. The space is initialized to all bits zero. Returns The calloc function returns…

Yes, and that is exactly my point.

The article says you should use calloc because it provides these optimizations. I am saying no, that's goofy, because it is not specced to provide these optimizations.

Re: Why does calloc exist?

#48
> And at least we aren't trashing the cache hierarchy up front – if we delay the zero'ing until we were going to write to the pages anyway, then that means both writes happen at the same time, so we only have to pay one set of TLB / L2 cache / etc. misses.

Ooh, nice one. My first impression was that calloc was just lazy-allocating, which is fine in most cases but when you want precise control over timing, maybe you want to be sure that memory is zero'd at allocating time rather than pay the cost unexpectedly at use time.

But the cache-awareness makes that a moot point. You'd be paying double cache-eviction costs if you were clearing that memory up front: once at clearing time, and once at actual-writing time. This implementation of calloc avoids that.

Re: Why does calloc exist?

#49
post #34
post #30

Sorry, but this is just goofy and bad. If you depend on copy-on-write functionality, then you need to use an API that is specced to guarantee copy-on-write functionality. If that means you use an #ifdef per platform and do OS-specific stuff, then that is what you do. Anything else is amateur hour. If copy-on-write is a desirable feature, then as the API creator, your job is to expose this functionality in the cleares…

I think the author's point is opportunistic optimization. He didn't ask us to rely on this behavior.

If you don't know whether or not you are really getting an optimization, then how much do you really care?

If you really care, then you actually profile your system and see what takes how much time, under which circumstances. The results of such a profile are almost always surprising.

I guess this is a basic cultural difference -- almost nobody in the HN crowd really cares whether their software runs quickly; there is just a bunch of lip service and wanting-to-feel-warm-fuzzies, with very little actual work.

In video games (for example) we need to hit the frame deadline or else there is a very clear and drastic loss in quality. This makes this kind of issue a lot more real to us. If you look at the kinds of things we do to make sure we run quickly ... they are of a wholly different character than "guess that calloc is going to do copy-on-write maybe."

Post reply on HN