Not "C++ everywhere again" but maybe "understanding memory again"
Everything old is new again: memory optimization
31–40 of 168 posts
Re: Everything old is new again: memory optimization
#32Nice! > Peak memory consumption is 1.3 MB. At this point you might want to stop reading and make a guess on how much memory a native code version of the same functionality would use. I wish I knew the input size when attempting to estimate, but I suppose part of the challenge is also estimating the runtime's startup memory usage too. > Compute the result into a hash table whose keys are string views, not strings If t…
>> Peak memory consumption is 1.3 MB. At this point you might want to stop reading and make a guess on how much memory a native code version of the same functionality would use. At this point I'd make two observations: - how big is the text file? I bet it's a megabyte, isn't it? Because the "naive" way to do it is to read the whole thing into memory. - all these numbers are way too small to make meaningful distinctio…
> how big is the text file? I bet it's a megabyte, isn't it?
The edit in the article says ~1.5kb
Re: Everything old is new again: memory optimization
#33I'm always confused as hell how little insight we have in memory consumption. I look at memory profiles of rnomal apps and often think "what is burning that memory". Modern compression works so well, whats happening? Open your taskmaster and look through apps and you might ask yourself this. For example (lets ignore chrome, ms teams and all the other bloat) sublime consumes 200mb. I have 4 text files open. What is it…
Part of the problem is that modern apps aren't really "one thing" anymore
Re: Everything old is new again: memory optimization
#34Nice! > Peak memory consumption is 1.3 MB. At this point you might want to stop reading and make a guess on how much memory a native code version of the same functionality would use. I wish I knew the input size when attempting to estimate, but I suppose part of the challenge is also estimating the runtime's startup memory usage too. > Compute the result into a hash table whose keys are string views, not strings If t…
I suspect it'll be selective
Re: Everything old is new again: memory optimization
#35String views were a solid addition to C++. Still underutilized. It does not matter which language you are using when you make thousands of tiny memory allocations during parsing. https://en.cppreference.com/w/cpp/string/basic_string_view.h...
In C you have char*
Re: Everything old is new again: memory optimization
#36I'm always confused as hell how little insight we have in memory consumption. I look at memory profiles of rnomal apps and often think "what is burning that memory". Modern compression works so well, whats happening? Open your taskmaster and look through apps and you might ask yourself this. For example (lets ignore chrome, ms teams and all the other bloat) sublime consumes 200mb. I have 4 text files open. What is it…
It's partly because there are layers of abstractions (frameworks, libraries / runtimes / VM, etc). Also, today's software often has other pressures, like development time, maintainability, security, robustness, accessibility, portability (OS / CPU architecture), etc. It's partly because the complexity / demand has increased.
https://waspdev.com/articles/2025-11-04/some-software-bloat-...
Re: Everything old is new again: memory optimization
#37String views were a solid addition to C++. Still underutilized. It does not matter which language you are using when you make thousands of tiny memory allocations during parsing. https://en.cppreference.com/w/cpp/string/basic_string_view.h...
In C you have char*
And the type system does not tell you if you need to call free on this char* when you’re done with it.
Re: Everything old is new again: memory optimization
#38Well, we can use memoryview for the dict generation avoiding creation of string objects until the time for the output: import re, operator def count_words(filename): with open(filename, 'rb') as fp: data= memoryview(fp.read()) word_counts= {} for match in re.finditer(br'\S+', data): word= data[match.start(): match.end()] try: word_counts[word]+= 1 except KeyError: word_counts[word]= 1 word_counts= sorted(word_counts.…
This doesn't do the same thing though, since it's not Unicode aware. >>> 'x\u2009 a'.split() ['x', 'a'] # incorrect; in bytes mode, `\S` doesn't know about unicode whitespace >>> list(re.finditer(br'\S+', 'x\u2009 a'.encode())) [ , ] # correct, in unicode mode >>> list(re.finditer(r'\S+', 'x\u2009 a')) [ , ]
There's bound to be a way to turn a stream of bytes into a stream of unicode code points (at least I think that's what python is doing for strings). Though I'm explicitly not volunteering to write the code for it.
Re: Everything old is new again: memory optimization
#39Well, we can use memoryview for the dict generation avoiding creation of string objects until the time for the output: import re, operator def count_words(filename): with open(filename, 'rb') as fp: data= memoryview(fp.read()) word_counts= {} for match in re.finditer(br'\S+', data): word= data[match.start(): match.end()] try: word_counts[word]+= 1 except KeyError: word_counts[word]= 1 word_counts= sorted(word_counts.…
For reasons I never quite understood python has a collections.Counter for the purpose of counting things. It's a bit cleaner.
Re: Everything old is new again: memory optimization
#40Earlier quoted context omitted.
>> Peak memory consumption is 1.3 MB. At this point you might want to stop reading and make a guess on how much memory a native code version of the same functionality would use. At this point I'd make two observations: - how big is the text file? I bet it's a megabyte, isn't it? Because the "naive" way to do it is to read the whole thing into memory. - all these numbers are way too small to make meaningful distinctio…
> how big is the text file? I bet it's a megabyte, isn't it? The edit in the article says ~1.5kb
Single page on many systems, which makes using mmap() for it even funnier.