A Story Of realloc (And Laziness) • Hacker News Board

Re: A Story Of realloc (And Laziness)

#61

Post by BugBrother » Mon, Apr 07, 2014, 7:43 AM UTC

stormbrew wrote: post #39

BugBrother wrote: post #22

Earlier quoted context omitted.

Huh, is it really implemented like that?! Why use realloc when growing an array you have an interface to? Just add another allocated buffer to the previous allocated areas. When there are too many small areas, consolidate with realloc/free. Much faster (yes yes, almost always). (Disclaimer: Last time I used C++ I had hair. :-) ) Edit: OK, thanks plorkyeran.

> Huh, is it really implemented like that?! Well, there is no grow() method on std::vector, so ... no? But generally speaking, std::vector implementations are basically required to[1] grow the backing store exponentially so that adding elements to them absolutely does not call malloc on every growth of the vector. You can make a vector do this kind of pessimistic allocation by calling reserve() for every element you…

I was talking of the standard way of sllocating extra buffers, to keep new data. With a layer on top of this, using a series of pointers (8-16) to the underlaying memory areas.

(When you fill up the sub-buffers, you do a realloc to all of them. Future sub-buffers get the new size.)

This needs less copying and (potentially) free calls. On the other hand: An access would need some extra operation to find the right buffer and get an offset into it.

Edit: Ok, thanks. I explained if I was unclear, since you went off on a tangent. It was informative, so it is all good. (I'm not going back to C++ et al anyway. :-) )

Re: A Story Of realloc (And Laziness)

#62

Post by greatsuccess » Mon, Apr 07, 2014, 7:55 AM UTC

Pretty sad that I needed to read through 20 minutes of code when he could have just said "It reallocs the paging table"

What do you think we are a bunch of fucking idiots? Must be nice to be 22 and think you have educational value.

Re: A Story Of realloc (And Laziness)

#63

Post by stormbrew » Mon, Apr 07, 2014, 8:12 AM UTC

BugBrother wrote: post #61

stormbrew wrote: post #39

Earlier quoted context omitted.

> Huh, is it really implemented like that?! Well, there is no grow() method on std::vector, so ... no? But generally speaking, std::vector implementations are basically required to[1] grow the backing store exponentially so that adding elements to them absolutely does not call malloc on every growth of the vector. You can make a vector do this kind of pessimistic allocation by calling reserve() for every element you…

I was talking of the standard way of sllocating extra buffers, to keep new data. With a layer on top of this, using a series of pointers (8-16) to the underlaying memory areas. (When you fill up the sub-buffers, you do a realloc to all of them. Future sub-buffers get the new size.) This needs less copying and (potentially) free calls. On the other hand: An access would need some extra operation to find the right buff…

I know. I wasn't really replying to that, though.

The standard library actually does have a built in container that's (almost) exactly as you describe, though. It's called std::deque (http://www.cplusplus.com/reference/deque/deque/).

Re: A Story Of realloc (And Laziness)

#64

Post by ithkuil » Mon, Apr 07, 2014, 3:54 PM UTC

srean wrote: post #20

rfrey wrote: post #6

Earlier quoted context omitted.

Serious question from a guy made soft by garbage collection: how frequent is memory allocation failure nowadays, with large memories and virtual memory? Were I to guess from my state of ignorance I'd think that if allocs began to fail, there was no recovery anyhow... so leaking in this case would be one leak right before a forced quit. Wrong? Are there lots of ways allocation can fail besides low memory conditions?

Failure or not, this is the highway to shitty software with bad user experience (except for very special cases where it makes sense). For me the funniest part has been that the people who seem entitled to write sloppy software are the exact same set who would have the shrillest voices complaining that firefox is so slow and bloated (although its not anymore) Many believe that its OK to hog memory, that it is an infin…

many people believe that over engineering is bad. It doesn't mean that it's OK to have crappy software, but that you should focus on things that matter. In other words, it's OK to do X until it's not.

On the other hand that's not an excuse for not understanding how things work and just having faith in some magic layer that somehow would just handle things for you. Doing so it could make it impossible to improve those parts of the system that are important without rewriting everything.

Re: A Story Of realloc (And Laziness)

#65

Post by asveikau » Mon, Apr 07, 2014, 4:12 PM UTC

fprawn wrote: post #50

asveikau wrote: post #42

Earlier quoted context omitted.

> In the general case, however, if allocating 100 bytes fails, reporting that error is also likely to fail. An actual memory allocation failure on a modern computer running a modern OS is a very rare and very bad situation. It's rarely recoverable. I call BS on this. First of all, it's not the 100 byte allocation that is likely to fail; chances are it's going to be bigger than 100 bytes and the 100 byte allocation wi…

If you have time, can you expound on this? Is there, perhaps, an open source project that handles NULL returns from malloc in this way you could point me to?

Adding a bit more after the fact: most well-written libraries in C are also like this. It's not a library's business to decide to exit the process at any time. The library doesn't know if it's some long-running process that absolutely must keep going, for example.