Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory Allocation

samwho.dev

Post reply on HN
Print view View original

111–120 of 182 posts

Re: Memory Allocation

#111

Post by Stratoscope »

derefr wrote: post #106
ilyt wrote: post #94

Earlier quoted context omitted.

I think the far weirder part of this was the kernel-side handling of scrollbars

Until Windows 95, Windows was essentially just a DOS application that grabbed the framebuffer and ran an event loop where it drew "controls" (which includes windows, buttons, text views, and yes, scrollbars.) That was the whole point of it. It wasn't an "OS" per se; DOS was the OS. Windows was what a Linux-head would think of as a combination of an X server and window manager. And Windows loaded your "application" as…

> This might make you realize why MDI (or Multiple Document Interface, where there are multiple small per-document "windows" inside one big window) was so popular back then. The MDI "windows" weren't actually windows — they didn't have their own WndProc. They were just controls, like a tab view is a control. Only the big container window was a real window, and so all the resources within that big window were shared between all the virtual windows. MDI was a memory-saving trick!

MDI may have saved some memory - I can't say one way or the other on that - but the mechanism you describe is incorrect.

Every MDI child window was a window of its own with its own WndProc. Every control inside those windows was also a window with its own WndProc. Every dialog box was also - yes - a window with its own WndProc.

You wouldn't always be aware of the WndProc in your code, but it was there.

If you ran WinSight or Spy++, you could see the entire window hierarchy including all the child windows, child control windows, and so on.

Later on, a few applications implemented "windowless controls" to save memory, but this was uncommon, especially in the early days. For example, there was an optional windowless version of the Rich Edit control:

https://learn.microsoft.com/en-us/windows/win32/controls/win...

Fun fact: an early informal name for MDI was "Mac in a box", because if you maximized the top level window, you had a somewhat Mac-like environment, with one menu bar at the top that was shared by the child windows.

Source: I was the author of WinSight and the VBX (Visual Basic eXtension) API.

Re: Memory Allocation

#112

Post by bambax »

Excellent, excellent article! I have a question though.

> Couldn't we rearrange the memory to get a block of 6 contiguous bytes? Some sort of defragmentation process?

> Sadly not. Remember earlier we talked about how the return value of malloc is the address of a byte in memory? Moving allocations won't change the pointers we have already returned from malloc. We would change the value those pointers are pointed at, effectively breaking them. This is one of the downsides of the malloc/free API.

But why not? Couldn't we store information about old pointers somewhere and match them with new addresses when defragmenting? Some kind of virtual memory driver that would map old pointers to new adresses transparently for the programs? Or would it be too much overhead for too little benefit?

Re: Memory Allocation

#113

Post by alex7734 »

bambax wrote: post #112

Excellent, excellent article! I have a question though. > Couldn't we rearrange the memory to get a block of 6 contiguous bytes? Some sort of defragmentation process? > Sadly not. Remember earlier we talked about how the return value of malloc is the address of a byte in memory? Moving allocations won't change the pointers we have already returned from malloc. We would change the value those pointers are pointed at,…

To do that you either need a structure that you update every time a pointer is created, copied, moved or deleted (too much overhead), or you need a way to scan the entire memory and get all the pointers. And at the point where you have a piece of code that knows where every pointer is, you already know which pointers aren't used anywhere anymore so it's a waste to not have it also call free() for you.

Once you have it call free() for you, your piece of code is now a compacting GC, like Java's for example.

Re: Memory Allocation

#114

Post by cconstantine »

bambax wrote: post #112

Excellent, excellent article! I have a question though. > Couldn't we rearrange the memory to get a block of 6 contiguous bytes? Some sort of defragmentation process? > Sadly not. Remember earlier we talked about how the return value of malloc is the address of a byte in memory? Moving allocations won't change the pointers we have already returned from malloc. We would change the value those pointers are pointed at,…

In a language like C that isn't really possible because the language can't keep track of all of the places that memory address is stored.

If malloc were to return something like an address that holds the address of memory allocated there is nothing preventing the program from reading that address, doing math on it, and storing it somewhere else.

Re: Memory Allocation

#115

Post by kaba0 »

bambax wrote: post #112

Excellent, excellent article! I have a question though. > Couldn't we rearrange the memory to get a block of 6 contiguous bytes? Some sort of defragmentation process? > Sadly not. Remember earlier we talked about how the return value of malloc is the address of a byte in memory? Moving allocations won't change the pointers we have already returned from malloc. We would change the value those pointers are pointed at,…

Well, that's what some GCs do, and they do indeed defragment the heap.

Re: Memory Allocation

#116

Post by School-Cotton »

bambax wrote: post #112

Excellent, excellent article! I have a question though. > Couldn't we rearrange the memory to get a block of 6 contiguous bytes? Some sort of defragmentation process? > Sadly not. Remember earlier we talked about how the return value of malloc is the address of a byte in memory? Moving allocations won't change the pointers we have already returned from malloc. We would change the value those pointers are pointed at,…

> Some kind of virtual memory driver that would map old pointers to new adresses transparently for the programs

You would need hardware support for this, since the hardware is what decides what gets returned when a program attempts to read from a memory location.

Hardware already does support virtual memory but the granularity is the page (which are a minimum of 4KiB in most OSs).

Re: Memory Allocation

#117

Post by jandrese »

davidgrenier wrote: post #21
samwho wrote: post #17

Earlier quoted context omitted.

Well shit. I think you're right.

Oh another thing, I'm not a fan of the premise: "As a general-purpose memory allocator, though, we can't get away with having no free implementation." I have a belief that the future of software are short-lived programs that never free memory. Programs allocate and terminate. Short-lived program communicate with each other via blocking CSP-style channels (see Reppy's Concurrent Programming in ML). If you could also e…

So basically garbage collection via just terminating and letting the OS handle it?

Re: Memory Allocation

#118

Post by AshamedCaptain »

derefr wrote: post #106
ilyt wrote: post #94

Earlier quoted context omitted.

I think the far weirder part of this was the kernel-side handling of scrollbars

Until Windows 95, Windows was essentially just a DOS application that grabbed the framebuffer and ran an event loop where it drew "controls" (which includes windows, buttons, text views, and yes, scrollbars.) That was the whole point of it. It wasn't an "OS" per se; DOS was the OS. Windows was what a Linux-head would think of as a combination of an X server and window manager. And Windows loaded your "application" as…

> Until Windows 95, everything was just happening in one shared address space, in real mode. In fact, it was only in Windows 3.1 where user applications stopped running in ring 0!

Windows 3.0 and predecessors runs in processor which had no concept of "ring 0", so that should not be surprising at all...

> Your application wasn't even a "process" per se

I think this is a bit of a "modernistic", "win32y" view of the definition of a process. Surely there are processes/tasks in 3.x -- you can launch multiple instances of a module, each of them can allocate memory separately, each of them have different resources/handles, and the OS will cleanup for you once each instance terminates (cleanly). Obviously, without any type of virtual address space or memory protection, any such process can write to and destroy the memory of any other process, but they are still processes. The existence of Yield/DirectedYield , which do not take/need a window, is also a hint of that. (Note there are no threads in 3.x).

Many platforms (that either predate VM or decide not to use VM for e.g. power usage concerns) worked like this. MacOS, Windows CE, PalmOS, etc.

Re: Memory Allocation

#119

Post by zeusk »

alex7734 wrote: post #104
meekaaku wrote: post #86

Earlier quoted context omitted.

You mean a bug this deep took one day to debug?

Yes and no, it took me from 8am to 3am once we decided it needed to get fixed but really it sat on the app for years, it only happened on a background process that sent print jobs on a timer, since it used Windows GDI to compose the image we sent to the printer it was affected (our "frontend" should've been affected too but never was, I guess because it had a different memory usage pattern). We just had it restart it…

How do you attach kd to VM?

While I was at MS, it was such a big PITA - we just had a bunch of IT managed machines with KVM console access and KDNET for debugging.

Re: Memory Allocation

#120

Post by thangalin »

When writing C, I tend to avoid calling malloc and free directly.

* https://github.com/DaveJarvis/mandelbrot/blob/master/memory....

I then apply this same principle of "opening" and "closing" structures throughout the application. Generally, I can quickly verify that the calls are balanced:

* https://github.com/DaveJarvis/mandelbrot/blob/master/threads...

What's nice about this pattern is that the underlying implementation of exactly how the memory is maintained for a particular data structure (e.g., whether malloc or gdImageCreateTrueColor is called) becomes an implementation detail:

* https://github.com/DaveJarvis/mandelbrot/blob/master/image.c

The main application opens then closes structures in the reverse order:

* https://github.com/DaveJarvis/mandelbrot/blob/master/main.c

This means there's only one call to malloc and one call to free throughout the entire application (third-party libraries notwithstanding), allowing them to be swapped out with ease.

Aside, logging can follow this same idea by restricting where any text is written to the console to a single location in the code base:

* https://github.com/DaveJarvis/mandelbrot/blob/master/logging...

Post reply on HN