Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everything old is new again: memory optimization

nibblestew.blogspot.com

Post reply on HN
Print view View original

71–80 of 168 posts

Re: Everything old is new again: memory optimization

#71

Post by usrnm »

adrian_b wrote: post #67
pjc50 wrote: post #52

Earlier quoted context omitted.

Reference counting in multithreaded systems is much more expensive than it sounds because of the synchronization overhead. I don't see it coming back. I don't think it saves massive amounts of memory, either, especially given my observation with vmmap upthread that in many cases the code itself is a dominant part of the (virtual) memory usage.

Incrementing or decrementing a shared counter is done with an atomic instruction, not with a locked critical section. This has negligible overhead in most cases. For instance, if the shared counter is already in some cache memory the overhead is smaller than a normal non-atomic access to the main memory. The intrinsic overhead of an atomic instruction is typically about the same as that of a simple memory access to d…

Atomic operations, especially RMW operations are very expensive, though. Not as expensive as a syscall, of course, but still a lot more expensive than non-atomic ones. Exactly because they break things like caches

Re: Everything old is new again: memory optimization

#72

Post by bcjdjsndon »

A few things

- since GC languages became prevalent, and maybe high level programming in general, coders arent as economic with their designs. Memory isn't something a coder should worry about apparently.

- far more people code apps in web languages because they don't know anything else. These are anywhere from 5-10 levels of abstraction away from the metal, naturally inefficient.

- increasing scope... I can only describe this one by example, web browsers must implement all manner of standards etc that it's become a mammoth task, especially compared to 90s. Same for compilers, oses, heck even computers thenselves were all one-man jobs at some point because things were simpler cos we knew less.

Re: Everything old is new again: memory optimization

#73

Post by Twirrim »

Capricorn2481 wrote: post #66
muskstinks wrote: post #61

Earlier quoted context omitted.

I do not have plugins installed and i have only a handful of files open on macos. Memroy statistics says 200mb and a peak of 750mb in the past (for whatever reason)

Is that in Task Manager, or is that not a reliable place to look for these statistics? Edit: From what I can tell, Sublime is allocated 100mb of virtual memory even if it's only using about 10mb in practice.

A lot of programs over-allocate on virtual memory, but don't actually use it, and the OS is smart enough to just pretend like it allocated it. I'm sure there's probably some justification for it somewhere, but it's hard not to see it as some absurd organically achieved agreement. Developers used to ask for more memory than their application actually needed and caused all sorts of OOM problems for end users. OS developers realised this and made the OS lie to the app to tell it it got what it asked for, and only give it memory as needed. Now developers just can't be bothered to set any realistic amount of memory, because what's the point, the OS is going to ignore it anyway.

Electron really loves to claim absurd amounts of memory, e.g. slack has claimed just over 1TB of virtual memory, but is only using just north of 200MB.

Re: Everything old is new again: memory optimization

#74

Post by VorpalWay »

Filligree wrote: post #64
dgb23 wrote: post #53

Earlier quoted context omitted.

I'd make the bet that "most people" (who can program) would not think of mmap, but either about streaming or would even just load the whole thing into memory. Ask a bunch of coding agents and they will give you these two versions, which means it's likely that the LLMs have seen these way more often than the mmap version. Both Opus and GPT even pushed back when I asked for mmap, both said it would "add complexity".

It does add complexity, and the optimal solution is probably not to use it. Consider what happens if a 4kB page has only a single unique word in it—you’d still need to load it to memory to read the string, it just isn’t accounted against your process (maybe). I would have expected something like this: - Scan the file serially. - For each word, find and increment a hash table entry. - Sort and print. In theory, techni…

That is a valid solution, but what IO block size should you use for the best performance? What if you end up reading half a word at the end of a chunk?

Handling that is in my opinion way more complex than letting the kernel figure it out via mmap. The kernel knows way more than you about the underlying block devices, and you can use madvise with MADV_SEQUENTIAL to indicate that you will read the whole file sequentially. (That might free pages prematurely if you keep references into the data rather than copy the first occurance of each word though, so perhaps not ideal in this scenario.)

Re: Everything old is new again: memory optimization

#75

Post by ben-schaaf »

muskstinks wrote: post #21

I'm always confused as hell how little insight we have in memory consumption. I look at memory profiles of rnomal apps and often think "what is burning that memory". Modern compression works so well, whats happening? Open your taskmaster and look through apps and you might ask yourself this. For example (lets ignore chrome, ms teams and all the other bloat) sublime consumes 200mb. I have 4 text files open. What is it…

Completely agree, it would be very helpful to get even just a breakdown of what the ram is being used for. It's unfortunately a lot of work to instrument.

> sublime consumes 200mb. I have 4 text files open. What is it doing?

To add to what others have said: Depending on the platform a good amount will be the system itself, various buffers and caches. If you have a folder open in the side bar, Sublime Text will track and index all the files in there. There's also no limit to undo history that is kept in RAM.

There's also the possibility that that 200MB includes the subprocesses, meaning the two python plugin hosts and any processes your plugins spawn - which can include heavy LSP servers.

Re: Everything old is new again: memory optimization

#76

Post by VorpalWay »

menaerus wrote: post #47
VorpalWay wrote: post #16

Earlier quoted context omitted.

The issue with retrofitting things to an existing well established language is that those new features will likely be underutilized. Especially in other existing parts of the standard library, since changing those would break backwards compatibly. std::optional is another example of this, which is not used much in the c++ standard library, but would be much more useful if used across the board. Contrast this with Rus…

Retrofitting new patterns or ideas is underutilized only when it is not worth the change. string_view example is trivial and anyone who cared enough about the extra allocations that could have happened already (no copy-elision taking place) rolled their own version of string_view or simply used char+len pattern. Those folks do not wait for the new standard to come along when they can already have the solution now. st…

Existing APIs for file IO in STL don't return string views into the file buffer of the library (when using buffered IO). That is something you could do, as an example.

Optional being opinionated I don't think I agree with. It is better to have an optional of something that can't be null (such as a reference) than have everything be implicitly nullable (such as raw pointers). This means you have to care about the nullable case when it can happen, and only when it can happen.

There is a caveat for C++ though: optional is larger in memory than a rae pointer. Rust optimises this case to be the same size (one pointer) by noting that the zero value can never be valid, so it is a "niche" that can be used for something else, such as the None variant of the Option. Such niche optimisation applies widely across the language, to user defined types as well. That would be impossible tp retrofit on C++ without at the very least breaking ABI, and probably impossible even on a language level. Maybe it could be done on a type by type basis with an attribute to opt in.

Re: Everything old is new again: memory optimization

#77

Post by ThrowawayR2 »

inetknght wrote: post #68
muskstinks wrote: post #21

I'm always confused as hell how little insight we have in memory consumption. I look at memory profiles of rnomal apps and often think "what is burning that memory". Modern compression works so well, whats happening? Open your taskmaster and look through apps and you might ask yourself this. For example (lets ignore chrome, ms teams and all the other bloat) sublime consumes 200mb. I have 4 text files open. What is it…

> I look at memory profiles of rnomal apps and often think "what is burning that memory". As a corrolary to this: I look at CPU utilization graphs. Programs are completely idle. "What is burning all that CPU?!" I remember using a computer with RAM measured in two-digit amounts of MiB. CPU measured in low hundreds of MHz. It felt just as fast -- sometimes faster -- as modern computers. Where is all of that extra RAM b…

Next time you see someone on HN blithely post "CPU / RAM is cheaper than developer time", it's them. That is the sort of coder who are collectively wasting our CPU and RAM.

Re: Everything old is new again: memory optimization

#78

Post by pjmlp »

muskstinks wrote: post #21

I'm always confused as hell how little insight we have in memory consumption. I look at memory profiles of rnomal apps and often think "what is burning that memory". Modern compression works so well, whats happening? Open your taskmaster and look through apps and you might ask yourself this. For example (lets ignore chrome, ms teams and all the other bloat) sublime consumes 200mb. I have 4 text files open. What is it…

It is a matter of tooling.

Visual Studio runs the memory profiler in debug mode right from the start, it is the default configuration, you need to disable it.

https://learn.microsoft.com/en-us/visualstudio/profiling/mem...

Re: Everything old is new again: memory optimization

#79

Post by est »

contravariant wrote: post #38
akx wrote: post #29

Earlier quoted context omitted.

This doesn't do the same thing though, since it's not Unicode aware. >>> 'x\u2009 a'.split() ['x', 'a'] # incorrect; in bytes mode, `\S` doesn't know about unicode whitespace >>> list(re.finditer(br'\S+', 'x\u2009 a'.encode())) [ , ] # correct, in unicode mode >>> list(re.finditer(r'\S+', 'x\u2009 a')) [ , ]

There's bound to be a way to turn a stream of bytes into a stream of unicode code points (at least I think that's what python is doing for strings). Though I'm explicitly not volunteering to write the code for it.

    import mmap, codecs

    from collections import Counter

    def word_count(filepath):

        freq = Counter()
    
        decode = codecs.getincrementaldecoder('utf-8')().decode
    
        with open(filepath, 'rb') as f, mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as mm:
        
                for chunk in iter(lambda: mm.read(65536), b''):
            
                        freq.update(decode(chunk).split())
            
                    freq.update(decode(b'', final=True).split())
        
                return freq

Re: Everything old is new again: memory optimization

#80

Post by 1vuio0pswjnm7 »

Been waiting for online commentary about programming to start acknowledging this situation as it pertains to writing programs

Memory and storage are not "cheap" anymore. Power may also rise in cost

Under these conditions, memory usage and binary size are irrefutably relevant^1

To some, this might feel like going backwards in time toward the mainframe era. Another current HN item with over 100 points, "Hold on to your hardware", reflects on how consumer hardware may change as a result

To me, the past was a time of greater software efficiency; arguably this was necessitated by cost. Perhaps higher costs in the present and future could lead to better software quality. But whether today's programmers are up for the challenge is debatable. It's like young people in finance whose only experience is in a world with "zero" interest rates. It's easier to whine about lowering rates than to adapt

With the money and poltical support available to "AI" companies, the incentive for efficiency of any kind is lacking. Perhaps their "no limits" operations, e.g., its effects on supply, may provide an incentive for others' efficiency

1. As an underpowered computer user that compiles own OS and writes own simple programs, I've always rejected large binary size and excessive memory use, even in times of "abundance"

Post reply on HN