Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everything old is new again: memory optimization

nibblestew.blogspot.com

Post reply on HN
Print view View original

51–60 of 168 posts

Re: Everything old is new again: memory optimization

#51

Post by rcxdude »

dgb23 wrote: post #12
pjc50 wrote: post #7

Earlier quoted context omitted.

>> Peak memory consumption is 1.3 MB. At this point you might want to stop reading and make a guess on how much memory a native code version of the same functionality would use. At this point I'd make two observations: - how big is the text file? I bet it's a megabyte, isn't it? Because the "naive" way to do it is to read the whole thing into memory. - all these numbers are way too small to make meaningful distinctio…

> all these numbers are way too small to make meaningful distinctions. Come back when you have a gigabyte. I have to disagree. Bad performance is often a result of a death of a thousands cuts. This function might be one among countless similarly inefficient library calls, programs and so on.

If you're not putting a representative amount of data through the test, you have no idea if the resource usage you're seeing scales with the amount of data or is just a fixed overhead if the runtime.

Re: Everything old is new again: memory optimization

#52

Post by pjc50 »

gwbas1c wrote: post #43

A lot of frameworks that use variants of "mark and sweep" garbage collection instead of automatic reference counting are built with the assumption that RAM is cheap and CPU cycles aren't, so they are highly optimized CPU-wise, but otherwise are RAM inefficient. I wonder if frameworks like dotnet or JVM will introduce reference counting as a way to lower the RAM footprint?

Reference counting in multithreaded systems is much more expensive than it sounds because of the synchronization overhead. I don't see it coming back. I don't think it saves massive amounts of memory, either, especially given my observation with vmmap upthread that in many cases the code itself is a dominant part of the (virtual) memory usage.

Re: Everything old is new again: memory optimization

#53

Post by dgb23 »

VorpalWay wrote: post #17
dgb23 wrote: post #15

Not a C++ programmer and I think the solution is neat. But it's not necessarily an apples to apples comparison. It's not unfair to python because of the runtime overhead. It's unfair because it's a different algorithm with fundamentally different memory characteristics. A fairer comparison would be to stream the file in C++ as well and maintain internal state for the count. For most people that would be the first/nai…

> A fairer comparison would be to stream the file in C++ as well and maintain internal state for the count. Wouldn't memory mapping the data in Python be the more fair comparison? If the language doesn't support that, then this seems to absolutely be a fair comparison. > For most people that would be the first/naive approach as well when they programmed something like this I think. I disagree, my mind immediately goe…

I'd make the bet that "most people" (who can program) would not think of mmap, but either about streaming or would even just load the whole thing into memory.

Ask a bunch of coding agents and they will give you these two versions, which means it's likely that the LLMs have seen these way more often than the mmap version. Both Opus and GPT even pushed back when I asked for mmap, both said it would "add complexity".

Re: Everything old is new again: memory optimization

#54

Post by zozbot234 »

pjc50 wrote: post #52
gwbas1c wrote: post #43

A lot of frameworks that use variants of "mark and sweep" garbage collection instead of automatic reference counting are built with the assumption that RAM is cheap and CPU cycles aren't, so they are highly optimized CPU-wise, but otherwise are RAM inefficient. I wonder if frameworks like dotnet or JVM will introduce reference counting as a way to lower the RAM footprint?

Reference counting in multithreaded systems is much more expensive than it sounds because of the synchronization overhead. I don't see it coming back. I don't think it saves massive amounts of memory, either, especially given my observation with vmmap upthread that in many cases the code itself is a dominant part of the (virtual) memory usage.

If you use an ownership/lifetime system under the hood you only pay that synchronization overhead when ownership truly changes, i.e. when a reference is added or removed that might actually impact the object's lifecycle. That's a rare case with most uses of reference counting; most of the time you're creating a "sub"-reference and its lifetime is strictly bounded by some existing owning reference.

Re: Everything old is new again: memory optimization

#55

Post by Capricorn2481 »

muskstinks wrote: post #21

I'm always confused as hell how little insight we have in memory consumption. I look at memory profiles of rnomal apps and often think "what is burning that memory". Modern compression works so well, whats happening? Open your taskmaster and look through apps and you might ask yourself this. For example (lets ignore chrome, ms teams and all the other bloat) sublime consumes 200mb. I have 4 text files open. What is it…

> sublime consumes 200mb. I have 4 text files open. What is it doing?

Huh? Sublime Text? I have like 100 files open and it uses 12mb. Sublime is extremely lean.

Do you have plugins installed?

Re: Everything old is new again: memory optimization

#56

Post by Capricorn2481 »

pjc50 wrote: post #49
muskstinks wrote: post #21

I'm always confused as hell how little insight we have in memory consumption. I look at memory profiles of rnomal apps and often think "what is burning that memory". Modern compression works so well, whats happening? Open your taskmaster and look through apps and you might ask yourself this. For example (lets ignore chrome, ms teams and all the other bloat) sublime consumes 200mb. I have 4 text files open. What is it…

https://learn.microsoft.com/en-us/sysinternals/downloads/vmm... for an empty sublime text window gives me: - 100MB 'image' (ie executable code; the executable itself plus all the OS libraries loaded.) - 40MB heap - 50MB "mapped file", mostly fonts opened with mmap() or the windows equivalent - 45MB stack (each thread gets 2MB) - 40MB "shareable" (no idea) - 5MB "unusable" (appears to be address space that's not usabl…

But I have sublime text open with a hundred files and it's using 12mb.

Re: Everything old is new again: memory optimization

#57

Post by saintfire »

MrBuddyCasino wrote: post #26
biorach wrote: post #4

"copyright infringement factories"

I don't know how anyone can call the most amazing invention in computer science of the last 20 years "copyright infringement factories". We went from the ST:NG ship computer being futuristic tech to "we kinda have this now". Its like calling cars "air pollution factories", as if that was their only purpose and use. A fundamentally anti-civilisational mindset.

The people pushing this technology, that accelerates climate change, have lobbied the government to circumvent typical roadblocks created by society to limit sensationalist development. Incidentally, the same people who talk about how dangerous AI will be for society, but don't worry, they're going to be the one to deliver it safely.

Now, I don't believe AI will ever amount to enough to be a critical threat to human life, you know, beyond the immense amounts of wasted energy they propose to convert into something more useful, like a market crash or heat and noise, or both.

Not sure how you can call someone opposed to any of that "anti-civilisational" matter-of-factly.

Re: Everything old is new again: memory optimization

#58

Post by zozbot234 »

griffindor wrote: post #3

Nice! > Peak memory consumption is 1.3 MB. At this point you might want to stop reading and make a guess on how much memory a native code version of the same functionality would use. I wish I knew the input size when attempting to estimate, but I suppose part of the challenge is also estimating the runtime's startup memory usage too. > Compute the result into a hash table whose keys are string views, not strings If t…

> If the file is mmap'd, and the string view points into that, presumably decent performance depends on the page cache having those strings in RAM.

Not so much, because you only need some fraction of that memory when the program is actually running; the OS is free to evict it as soon as it needs the RAM for something else. Non-file-backed memory can only be evicted by swapping it out and that's way more expensive,

Re: Everything old is new again: memory optimization

#59

Post by muskstinks »

MrBuddyCasino wrote: post #26
biorach wrote: post #4

"copyright infringement factories"

I don't know how anyone can call the most amazing invention in computer science of the last 20 years "copyright infringement factories". We went from the ST:NG ship computer being futuristic tech to "we kinda have this now". Its like calling cars "air pollution factories", as if that was their only purpose and use. A fundamentally anti-civilisational mindset.

You can see both sides, critzise how its done and still wanting to have the result of it.

Its a little bit hypocritic which often enough ends in realism aka "okay we clearly can't fight their copyright infridgments because they are too powerful and too rich but at least we can use the good side of it".

Nothing btw. enforces all of this to happen THAT fast besides capitalism. We could slow down, we could do it better or more right.

Re: Everything old is new again: memory optimization

#60

Post by Filligree »

pjc50 wrote: post #40
kloop wrote: post #32

Earlier quoted context omitted.

> how big is the text file? I bet it's a megabyte, isn't it? The edit in the article says ~1.5kb

Single page on many systems, which makes using mmap() for it even funnier.

Not to mention inefficient in memory use. I would have expected a mention of interning; using string-views is fine, but making it a view of 4kB cache pages is not really.

Though I believe the “naive” streaming read could very well be superior here.

Post reply on HN