51–60 of 145 posts
This is awesome! To anyone interested in learning more about this, I wrote https://cpu.land/ a couple years ago. It doesn't go as in-depth into e.g. memory layout as OP does but does cover multitasking and how the code is loaded in the first place.
As someone who teaches this stuff at university, I see students getting confused every single year by how textbooks draw memory. The problem is mostly visual, not conceptual. Most diagrams in books and slides use an old hardware-centric convention: they draw higher addresses at the top of the page and lower addresses at the bottom. People sometimes justify this with an analogy like “floors in a building go up,” so ad…
I think I got stuck in the same rut that I learned address space in whilst writing that diagram. I would tend to agree with you that your model makes much more sense to the student. Related: In notation, one thing that I used to struggle with is how addresses (e.g. 0xAB_CD) actually have the bit representation of [0xCD, 0xAB]. Wonder if there's a common way to address that?
About the address notation you're describing, I'm not sure I fully get the problem. Can you spell out the question with a concrete example?
This is what the address space of a real bash process looks like on my machine:
__
$ cat /proc/$(pidof bash)/maps
5e6e8fd0f000-5e6e8fd3f000 r--p 00000000 fc:00 3539412 /usr/bin/bash
5e6e8fd3f000-5e6e8fe2e000 r-xp 00030000 fc:00 3539412 /usr/bin/bash
5e6e8fe2e000-5e6e8fe63000 r--p 0011f000 fc:00 3539412 /usr/bin/bash
5e6e8fe63000-5e6e8fe67000 r--p 00154000 fc:00 3539412 /usr/bin/bash
5e6e8fe67000-5e6e8fe70000 rw-p 00158000 fc:00 3539412 /usr/bin/bash
5e6e8fe70000-5e6e8fe7b000 rw-p 00000000 00:00 0
5e6e94891000-5e6e94a1e000 rw-p 00000000 00:00 0 [heap]
7ec3d1400000-7ec3d16eb000 r--p 00000000 fc:00 3550901 /usr/lib/locale/locale-archive
7ec3d1800000-7ec3d1828000 r--p 00000000 fc:00 3548995 /usr/lib/x86_64-linux-gnu/libc.so.6
7ec3d1828000-7ec3d19b0000 r-xp 00028000 fc:00 3548995 /usr/lib/x86_64-linux-gnu/libc.so.6
7ec3d19b0000-7ec3d19ff000 r--p 001b0000 fc:00 3548995 /usr/lib/x86_64-linux-gnu/libc.so.6
7ec3d19ff000-7ec3d1a03000 r--p 001fe000 fc:00 3548995 /usr/lib/x86_64-linux-gnu/libc.so.6
7ec3d1a03000-7ec3d1a05000 rw-p 00202000 fc:00 3548995 /usr/lib/x86_64-linux-gnu/libc.so.6
7ec3d1a05000-7ec3d1a12000 rw-p 00000000 00:00 0
7ec3d1a2b000-7ec3d1a84000 r--p 00000000 fc:00 3549063 /usr/lib/locale/C.utf8/LC_CTYPE
7ec3d1a84000-7ec3d1a85000 r--p 00000000 fc:00 3549069 /usr/lib/locale/C.utf8/LC_NUMERIC
7ec3d1a85000-7ec3d1a86000 r--p 00000000 fc:00 3549072 /usr/lib/locale/C.utf8/LC_TIME
7ec3d1a86000-7ec3d1a87000 r--p 00000000 fc:00 3549062 /usr/lib/locale/C.utf8/LC_COLLATE
7ec3d1a87000-7ec3d1a88000 r--p 00000000 fc:00 3549067 /usr/lib/locale/C.utf8/LC_MONETARY
7ec3d1a88000-7ec3d1a89000 r--p 00000000 fc:00 3549066 /usr/lib/locale/C.utf8/LC_MESSAGES/SYS_LC_MESSAGES
7ec3d1a89000-7ec3d1a8a000 r--p 00000000 fc:00 3549070 /usr/lib/locale/C.utf8/LC_PAPER
7ec3d1a8a000-7ec3d1a8b000 r--p 00000000 fc:00 3549068 /usr/lib/locale/C.utf8/LC_NAME
7ec3d1a8b000-7ec3d1a8c000 r--p 00000000 fc:00 3549061 /usr/lib/locale/C.utf8/LC_ADDRESS
7ec3d1a8c000-7ec3d1a8d000 r--p 00000000 fc:00 3549071 /usr/lib/locale/C.utf8/LC_TELEPHONE
7ec3d1a8d000-7ec3d1a90000 rw-p 00000000 00:00 0
7ec3d1a90000-7ec3d1a9e000 r--p 00000000 fc:00 3551411 /usr/lib/x86_64-linux-gnu/libtinfo.so.6.4
7ec3d1a9e000-7ec3d1ab1000 r-xp 0000e000 fc:00 3551411 /usr/lib/x86_64-linux-gnu/libtinfo.so.6.4
7ec3d1ab1000-7ec3d1abf000 r--p 00021000 fc:00 3551411 /usr/lib/x86_64-linux-gnu/libtinfo.so.6.4
7ec3d1abf000-7ec3d1ac3000 r--p 0002e000 fc:00 3551411 /usr/lib/x86_64-linux-gnu/libtinfo.so.6.4
7ec3d1ac3000-7ec3d1ac4000 rw-p 00032000 fc:00 3551411 /usr/lib/x86_64-linux-gnu/libtinfo.so.6.4
7ec3d1ac4000-7ec3d1ac5000 r--p 00000000 fc:00 3549065 /usr/lib/locale/C.utf8/LC_MEASUREMENT
7ec3d1ac5000-7ec3d1ac6000 r--p 00000000 fc:00 3549064 /usr/lib/locale/C.utf8/LC_IDENTIFICATION
7ec3d1ac6000-7ec3d1acd000 r--s 00000000 fc:00 3548984 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
7ec3d1acd000-7ec3d1acf000 rw-p 00000000 00:00 0
7ec3d1acf000-7ec3d1ad0000 r--p 00000000 fc:00 3548992 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7ec3d1ad0000-7ec3d1afb000 r-xp 00001000 fc:00 3548992 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7ec3d1afb000-7ec3d1b05000 r--p 0002c000 fc:00 3548992 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7ec3d1b05000-7ec3d1b07000 r--p 00036000 fc:00 3548992 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7ec3d1b07000-7ec3d1b09000 rw-p 00038000 fc:00 3548992 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7ffd266f8000-7ffd26719000 rw-p 00000000 00:00 0 [stack]
7ffd2678a000-7ffd2678e000 r--p 00000000 00:00 0 [vvar]
7ffd2678e000-7ffd26790000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0 [vsyscall]
___
Each line is a memory mapping. The first field is the start address. The second field is the end address. So an entry like
7ffd266f8000-7ffd26719000
means "this mapping covers virtual addresses from 0x7ffd266f8000 up to 0x7ffd26719000."
The addresses are always increasing:
- left to right: within a single line you go from lower address to higher address
- top to bottom: as you go down the list you also go to higher and higher addresses
Exactly like reading a book: left to right and then top to bottom.
> The ELF file contains a dynamic section which tells the kernel which shared libraries to load, and another section which tells the kernel to dynamically “relocate” pointers to those functions, so everything checks out. This is not how dynamic linking works on GNU/Linux. The kernel processes the program headers for the main program (mapping the PT_LOAD segments, without relocating them) and notices the PT_INTERP pro…
I've always wondered why there weren't more popular loaders to choose from given that on Linux loaders are user-space
Loading ELFs and processing relocations is actually not too bad. It’s fun after the initial learning curve.
Then one has to worry about handling of “dlopen” and the loader creating the data structures it cares about. Yuck!!!
It’s kinda a shame because the glibc loader is a bit bloated with all the audit and preload handling. Great for flexibility, not for security.
Earlier quoted context omitted.
> split Glibc into 3 parts: syscalls, dynamic loader and the actual C library. What is left of the C standard library, if you remove syscall wrappers? > ABI hell Is that really the case? From my understanding the problem is more, that Linux isn't an OS, so you can't rely on any *.so being there.
> > split Glibc into 3 parts: syscalls, dynamic loader and the actual C library. > What is left of the C standard library, if you remove syscall wrappers? Still quite a bit actually. Stuff like malloc, realloc, free, fopen, FILE, getaddrinfo, getlogin, math functions like cos, sin tan, stdatomic implementations, some string functions are all defined in C library. They are not direct system calls unlike: open, read, w…
Wrapper around sbrk, mmap, etc. whatever the modern variant is.
> fopen, FILE
Wrapper around open, write, read, close.
> stdatomic implementations
You can argue, these are wrappers around thread syscalls.
> math functions like cos, sin tan, some string functions are all defined in C library
True for these, but they are so small, they could just be inlined directly, on their own they wouldn't necessarily deserve a library.
> That's why I used more specific term GNU/Linux at the start.
While GNU/Linux does describe a complete OS, it doesn't describe any specific OS. Every Distro does it's own thing, so I think these is what you actually need to call an OS. But everything is built so that the user can take the control over the architecture and which components the OS consists of, so every installation can be a snowflake, and then it is technically its own OS.
I personally consider libc and the compiler (which both make a C implementation) to be part of the OS. I think this is both grounded in theory and in practice. Only in some weird middle ground between theory and practice you can consider them to not be.
Earlier quoted context omitted.
I think I got stuck in the same rut that I learned address space in whilst writing that diagram. I would tend to agree with you that your model makes much more sense to the student. Related: In notation, one thing that I used to struggle with is how addresses (e.g. 0xAB_CD) actually have the bit representation of [0xCD, 0xAB]. Wonder if there's a common way to address that?
Yes, I reached the same conclusions the hard way while exploiting memory corruption bugs. Once I understood how misleading these representations can be, everything finally became clear. About the address notation you're describing, I'm not sure I fully get the problem. Can you spell out the question with a concrete example? This is what the address space of a real bash process looks like on my machine: __ $ cat /proc…
As someone who teaches this stuff at university, I see students getting confused every single year by how textbooks draw memory. The problem is mostly visual, not conceptual. Most diagrams in books and slides use an old hardware-centric convention: they draw higher addresses at the top of the page and lower addresses at the bottom. People sometimes justify this with an analogy like “floors in a building go up,” so ad…
I think I got stuck in the same rut that I learned address space in whilst writing that diagram. I would tend to agree with you that your model makes much more sense to the student. Related: In notation, one thing that I used to struggle with is how addresses (e.g. 0xAB_CD) actually have the bit representation of [0xCD, 0xAB]. Wonder if there's a common way to address that?
This convention started on early Intel chips and was kept for backward compatibility. It also has a practical benefit: it makes basic arithmetic and type widening cheaper in hardware. The "low" part of the value is always at the base address, so the CPU can load 8 bits, then 16 bits, then 32 bits, etc. starting from the same address without extra offset math.
So when you say an address like 0xABCD shows up in memory as [0xCD, 0xAB] byte-by-byte, that's not the address being "reversed". That's just the little-endian in-memory layout of that numeric value.
There are also big-endian architectures, where the most significant byte is stored at the lowest address. That matches how humans usually write numbers (0xABCD in memory as [0xAB, 0xCD]). But most mainstream desktop/server CPUs today are little-endian, so you mostly see the little-endian view.
Earlier quoted context omitted.
I think I got stuck in the same rut that I learned address space in whilst writing that diagram. I would tend to agree with you that your model makes much more sense to the student. Related: In notation, one thing that I used to struggle with is how addresses (e.g. 0xAB_CD) actually have the bit representation of [0xCD, 0xAB]. Wonder if there's a common way to address that?
If you're referring to little-endianness, it means the CPU stores multi-byte values in memory with the least significant byte first (at the lowest address). This convention started on early Intel chips and was kept for backward compatibility. It also has a practical benefit: it makes basic arithmetic and type widening cheaper in hardware. The "low" part of the value is always at the base address, so the CPU can load…
> The ELF file contains a dynamic section which tells the kernel which shared libraries to load, and another section which tells the kernel to dynamically “relocate” pointers to those functions, so everything checks out. This is not how dynamic linking works on GNU/Linux. The kernel processes the program headers for the main program (mapping the PT_LOAD segments, without relocating them) and notices the PT_INTERP pro…
I've always wondered why there weren't more popular loaders to choose from given that on Linux loaders are user-space
https://www.kernel.org/doc/html/latest/admin-guide/binfmt-mi...
Earlier quoted context omitted.
> some of it will only ever run on linux servers. I’ve spent quite a lot of time dealing with code that will ever run on Linux which did not in fact only ever run on Linux! Obviously for hobby projects anyone can do what they want. But adult projects should support Windows imho and consider Windows support from the start. Cross-platform is super easy unless you choose to make it hard.
I don't think we are talking about the same type of software? The type I was talking about will only ever run on Linux because it's a (HTTP-ish) server that will only ever run on Linux. Probably a server that is only ever run by a single company on a single CPU type. That company will have complete control of the OS stack, so if it says no Windows, then no Windows has to be supported.
Earlier quoted context omitted.
File descriptors are part of the linux syscall API, not libc. Are you thinking of FILE?
The "syscall API" is part of libc too. The read syscall is a trap, you put arguments in the right registers and issue the correct instruction[1] to enter the kernel. That's not something that can be expressed in C. The read() function that your C code actually uses is a C function provided by the C library. [1] "svc 0" on ARM, "int 0x80" on i386, etc...
For instance, Go directly calls Linux system calls without going through libc (which has lead to lots of workarounds to emulate some glibc-specific behaviour -- swings and roundabouts I guess...).
Other operating systems do not provide this kind of compatibility guarantee and instead require you to always go through libc as the syscall ABI is not stable (though ultimately, you can still use __asm__ if you so choose).
In any case, file descriptors are definitely not a libc construct on Linux.