Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In C, how do you know if the dynamic allocation succeeded?

lemire.me

Post reply on HN
Print view View original

191–198 of 198 posts

Re: In C, how do you know if the dynamic allocation succeeded?

#191

Post by marcan_42 »

tsimionescu wrote: post #123
marcan_42 wrote: post #110

Earlier quoted context omitted.

Are there any things that are illegal between fork and exec? It is perfectly legal to exec whenever you want, and it it perfectly legal to fork whenever you want. I'm not aware of any requirements around the fork/exec sequence.

For a single-threaded process, sure. But in a multi threaded context, almost any operation done in the child after fork() can royally mess up the system. There are many versions of libcs where malloc() after fork() is likely to deadlock between the parent and child processes, as they share the internal malloc() locks.

Ah, yes, multithreading. That always interacts poorly with old UNIX APIs...

Re: In C, how do you know if the dynamic allocation succeeded?

#192

Post by underdeserver »

dekhn wrote: post #48
underdeserver wrote: post #39

Earlier quoted context omitted.

TIL, unless you explicitly disable memory overcommit, it can and will overcommit. This is crazy to me.

So, uh, how do you feel about fractional reserve banking? Nearly all banks worldwide practice it. Statistically, it's not impossible that the entire world financial system could collapse due to uncorrelated bank runs.

It's not impossible, no, but in the case of my computer a single user - me - controls memory usage.

I guess it's beneficial in >99% of use-cases, and the <1% of other cases can turn it off. Still I guess I'm naive enough to hope a correct program would not crash.

Re: In C, how do you know if the dynamic allocation succeeded?

#193

Post by Anunayj »

dekhn wrote: post #6

I once complained about malloc happily allocating memory that the physical memory system couldn't satisfy (never let your hand write a check your ass can't cash?) but the more experienced programmer asked me if I'd heard of fractional reserve banking, and if not, whether it bothered me too.

Oh. but is there a scenario where it might be useful to check if a certain amount of memory can be available? Like say you know a certain process uses 6 Gigs for memory but will take a while to get to that point.. and then fail, is it not safe to just error out earlier?

Re: In C, how do you know if the dynamic allocation succeeded?

#194

Post by yc12340 »

josefx wrote: post #166
dekhn wrote: post #67

Earlier quoted context omitted.

if overcommit is disabled, then the system drops its internal caches (dentry cache, pagecache) to satisfy a memory allocation. Since most applications don't touch pages they allocate, that means the OS could have avoided dropping the caches. Since it did drop the caches, other parts of the system will then have to do more work to reconstruct the caches (looking up an dentry explicitly, or loading a page from disk ins…

> if overcommit is disabled, then the system drops its internal caches (dentry cache, pagecache) to satisfy a memory allocation. Couldn't the system just reserve the pages for future use by the application but still use them for caching until the application actually tries to use them?

This sounds like it would cause LRU caches to lose their LRU properties...

Re: In C, how do you know if the dynamic allocation succeeded?

#195

Post by kccqzy »

atfzl wrote: post #158

This made be curious to check the real and virtual memory of some processes on my laptop (MacBook Air M1). The real memory size of Safari is ~160MB but virtual memory size is 392GB which doesn't look right. I checked other processes and all the processes have similar virtual memory size which is around ~390GB. I wonder if this is a bug in Activity Monitor or the virtual memory allocations really are this big for each…

That sounds about right. The runtime of a certain language actually allocates 1TB of virtual memory at startup, and then hands out memory from that pool. It's just reserving 1TB of virtual address space.

Re: In C, how do you know if the dynamic allocation succeeded?

#196

Post by josefx »

yc12340 wrote: post #194
josefx wrote: post #166

Earlier quoted context omitted.

> if overcommit is disabled, then the system drops its internal caches (dentry cache, pagecache) to satisfy a memory allocation. Couldn't the system just reserve the pages for future use by the application but still use them for caching until the application actually tries to use them?

This sounds like it would cause LRU caches to lose their LRU properties...

It doesn't need to reserve a specific page during allocation, just ensure that the number of reserved pages is smaller than the total amount of pages. I would expect at least read only caches to mostly behave like they currently do on a system with over commit enabled.

Re: In C, how do you know if the dynamic allocation succeeded?

#197

Post by angry_octet »

trhway wrote: post #139
angry_octet wrote: post #117

Earlier quoted context omitted.

Virtual memory is always on... If you actually hit swap the system effectively deadlocks anyway because the swap daemon is too dumb by being too fair. Persistent memory might change this but it needs so much work.

it was a throwback down the memory lane and the years before my first Linux in 1995 i had been dealing with Windows where configuration of swap was "configuring virtual memory", so i automatically used that term. > If you actually hit swap the system effectively deadlocks anyway that depends. In many cases in the past the options would be either with swap and thus slow or pretty much not at all. And again these days…

OOM kill exists because swap is not being useful. A process failure is something you can deal with, livelock isn't. If you have an algorithm that works out of core then you'll never hit swap because you maintain the working set below memory limits.

Re: In C, how do you know if the dynamic allocation succeeded?

#198

Post by andi999 »

chrsig wrote: post #96
joosters wrote: post #41

Earlier quoted context omitted.

Please don’t blindly declare it ‘totally dumb’. If you disallow overcommit, you can end up with a system that can’t fork and run /bin/true, even if there are gigabytes of memory left. Both styles of memory allocation have their uses, and their drawbacks, but please understand them before declaring many OS designers as stupid and dumb.

I agree with you on all counts, but it's worth highlighting that the parent did not call the designers dumb, but rather the situation and implementation. I know it can seem like a distinction without a difference, but I think it's fair to critique work. I think they could have been more articulate and considerate towards the designers, and have some empathy that many people have worked very hard on it, and did their…

But don't you need to check the return value of malloc to then reliable crash/terminate the program? It is like if your car runs out of gas you need to stop, but you don't abandon the steering wheel at high speed and hope for the best.
Post reply on HN