Anatomy of a Program in Memory
duartes.org
Anatomy of a Program in Memory
1–10 of 24 posts
Re: Anatomy of a Program in Memory
#2Totally worth the price of admission. Every linux dev, especially c/c++ programmers, should be required to read this post. Once a year.
Re: Anatomy of a Program in Memory
#3According to the comments on that article, the people still don't understand the basics of virtual memory.
"Great article! / Quite shocked to know that windows takes double the kernel memory compared to Linux."
"I’m wondering, though, why does the kernel space consume 1gb? That seems like a lot.."
Re: Anatomy of a Program in Memory
#4According to the comments on that article, the people still don't understand the basics of virtual memory. "Great article! / Quite shocked to know that windows takes double the kernel memory compared to Linux." "I’m wondering, though, why does the kernel space consume 1gb? That seems like a lot.."
Address space is not much bigger than physical memory now for 32 bits. I've read that my laptop could in fact have 4GB instead of the nominal 3GB. So even if the memory is not "consumed", the default must be changed at run time, unless the kernel space literally consumes 2/3 of available memory.
Re: Anatomy of a Program in Memory
#5According to the comments on that article, the people still don't understand the basics of virtual memory. "Great article! / Quite shocked to know that windows takes double the kernel memory compared to Linux." "I’m wondering, though, why does the kernel space consume 1gb? That seems like a lot.."
As narag says, it's only recently that this has become an issue. When physical memory size was orders of magnitude smaller than than the address space, it didn't matter that the kernel carved out 1GB of address space for itself. User applications still had the other 3GB, which was far more than any system had physical memory.
It's only during this limbo period when 4GB of RAM is relatively cheap, yet there are still 32-bit systems kicking around, that it becomes an issue. On a 64-bit system with a 16 exabyte address space, 1GB is a drop in the bucket. Once again, it won't matter.
(I suspect Linux allocates more than this for itself in 64-bit systems, but I don't know the details. At any rate, out of 16EB, even 100GB would hardly be noticed.)
Re: Anatomy of a Program in Memory
#6According to the comments on that article, the people still don't understand the basics of virtual memory. "Great article! / Quite shocked to know that windows takes double the kernel memory compared to Linux." "I’m wondering, though, why does the kernel space consume 1gb? That seems like a lot.."
As narag says, it's only recently that this has become an issue. When physical memory size was orders of magnitude smaller than than the address space, it didn't matter that the kernel carved out 1GB of address space for itself. User applications still had the other 3GB, which was far more than any system had physical memory. It's only during this limbo period when 4GB of RAM is relatively cheap, yet there are still…
From the way I read those comments, I gathered that the comments implied that Windows "consumed" two gigs of ram, where as Linux took 1 gig. I thought there was a dissonance between how people understood "virtual memory," which is essentially a mapping function, to "memory," which people assume to be the physical ram.
Re: Anatomy of a Program in Memory
#7According to the comments on that article, the people still don't understand the basics of virtual memory. "Great article! / Quite shocked to know that windows takes double the kernel memory compared to Linux." "I’m wondering, though, why does the kernel space consume 1gb? That seems like a lot.."
Address space is not much bigger than physical memory now for 32 bits. I've read that my laptop could in fact have 4GB instead of the nominal 3GB. So even if the memory is not "consumed", the default must be changed at run time, unless the kernel space literally consumes 2/3 of available memory.
Given you don't reference what operating system is in use, you're probably using Microsoft Windows here. The implementation of virtual memory has common traits, but details vary by OS and even OS version.
Physical address space and virtual address space are two different aspects of system design, and it is completely feasible to have these values be the same, or to have either of the two be larger than the other. Yes, you can have a 32-bit virtual memory system and 34 or more bits of physical memory, and there can be good reasons to do that, too.
It's also feasible to have address "holes" in virtual address space, and "holes" in physical address space. Address ranges or gaps that are simply not implemented by the particular processor, or that are not instantiated by the memory controller. But I digress.
Save for low-end boxes, all current gear is 64-bit virtual addressing (variously with "holes"), and most current systems can have 48 bits (x86-64) or 50 bits (IA-64) or other implementation-specific ranges for physical addressing. Details vary by processor and by implementation. Available memory varies by budget - 50 bits of memory is expensive - and target market for the box.
And consuming address space is different than consuming memory. On various systems, address space is an inexpensive resource. And on a virtual memory system, memory is limited by physical memory and by backing storage and by your willingness to wait for the transitions between these two resources; for memory paging to occur.
And what your application thinks is in memory might be on disk. When referenced, the data is valid or is paged in and made valid.
Parts of an OS might be memory-resident, and parts can be paged. Applications tend to be paged.
If you're programming in an area that has performance constraints, knowing details of the particular virtual memory and physical memory implementation can be a key to success. Issues around data alignment, multiprocessor memory caching and data structure layout can all be relevant; having a large and improperly-designed data structure can lead to memory paging for each data structure reference, for instance.
Re: Anatomy of a Program in Memory
#8That blog is a gold mine. Subscribed.
Re: Anatomy of a Program in Memory
#9According to the comments on that article, the people still don't understand the basics of virtual memory. "Great article! / Quite shocked to know that windows takes double the kernel memory compared to Linux." "I’m wondering, though, why does the kernel space consume 1gb? That seems like a lot.."
New people are being born all the time!
Re: Anatomy of a Program in Memory
#10I'd like to understand what's being described in this article, but unfortunately I lack the necessary educational background. Does anyone have any links to other resources that will allow me to better understand what is being discussed in this post?