Awesome read, thanks.
Kernel 101 – Let’s write a Kernel
71–80 of 104 posts
Re: Kernel 101 – Let’s write a Kernel
#72Great! except that to be called a kernel it's missing just a process manager, memory manager, filesystem, process separation and hardware abstraction. Yeah I'm that guy, down vote me as you wish, the article is still wrong. It's a way to load a ring-0 application into grub. Pretty cool, but not a kernel.
It's a statically linked program, real memory, diskless system, kernel-mode-only kernel without too much fluff in the video i/o abstraction.
Is Super Mario Bros. a kernel? Every NES, Game Boy, Sega Genesis, etc... game ran on the "bare metal" without anything resembling an OS (or even a BIOS, really).
Re: Kernel 101 – Let’s write a Kernel
#73I'm not exactly Linus Torvalds but I'm pretty sure a program that prints one line of text is not "a kernel". :)
It is already a kernel, technically, right?
A kernel is a program that provides services to userland programs. I reckon printing "my first kernel" is hardly a service to any userland program.
Re: Kernel 101 – Let’s write a Kernel
#74I kinda wish some of the FS utilities were better. Despite FUSE, mounting a block device (like, disk image) as a non-root user is tough, so writing to an image with actual partitions/FS on it is difficult, especially to script, especially if you don't want to sudo in a build script. losetup on a disk image doesn't (to my knowledge) detect partitions… for reasons unknown to me. Bootsectors are similar. You can't just…
> losetup on a disk image doesn't (to my knowledge) detect partitions… for reasons unknown to me.
You can use the kpartx command for this. This site has a good overview: http://nfolamp.wordpress.com/2010/08/16/mounting-raw-image-f...
Re: Kernel 101 – Let’s write a Kernel
#75This was fantastic. My only question is, how does one gain knowledge of the required x86 hardware specifics he mentions? I don't know where to begin looking to uncover these sorts of things: - The x86 CPU begins execution at the physical address [0xFFFFFFF0] - The bootloader loads the kernel at the physical address [0x100000] - The BIOS copys the contents of the first sector to physical address [0x7c00] Is there an x…
> Is there an x86 instruction manual or is this sort of thing passed down through generations of engineers?
Yes, there is an x86 instruction manuals, colloquially known as "intel manuals".
That's about a dozen volumes of documentation, at a few thousand pages each. It is quite a good example of well written and informative technical documentation.
http://www.intel.com/content/www/us/en/processors/architectu...
Re: Kernel 101 – Let’s write a Kernel
#76There's a slight problem in this tutorial in that it assumes ESP (the stack pointer) will be defined by the boot loader to point to an appropriate location for the stack. However, the Multiboot standard states that ESP is undefined [1], and that the OS should set up its own stack as soon as it needs one (here the CALL instruction uses the stack, and the compiled C code may well too).
An easy way to solve this is to reserve some bytes in the .bss section of the executable for the stack by adding a new section in the assembly file:
[section .bss align=16]
resb 8192
stack_end:
mov esp, stack_end
Re: Kernel 101 – Let’s write a Kernel
#77This seems like a good start but it's worth noting that it isn't guaranteed to work correctly.
The problem is that the control is passed to C code with no stack space set up. It works out of pure luck because the compiler has decided to keep all variables in registers and changing your C compiler flags might make this fail in interesting ways.
Another thing that is missing is clearing the .BSS section before passing control to the C code. It's not used at the moment, though.
I am pointing these things out because in my own Ring—0 programming projects I spent a lot of time debugging some failures related to stack space and an un-initialized BSS section.
Re: Kernel 101 – Let’s write a Kernel
#78This seems like a good start but it's worth noting that it isn't guaranteed to work correctly. The problem is that the control is passed to C code with no stack space set up. It works out of pure luck because the compiler has decided to keep all variables in registers and changing your C compiler flags might make this fail in interesting ways. Another thing that is missing is clearing the .BSS section before passing…
> Another thing that is missing is clearing the .BSS section before passing control to the C code. It's not used at the moment, though.
The Multiboot standard says that the boot loader will clear the .bss section for you - in section 3.1.3: "bss_end_addr Contains the physical address of the end of the bss segment. The boot loader initializes this area to zero"
I don't know what GRUB does if you rely on that fact it can parse ELF files and don't specify the fields like bss_end_addr though. I'm fairly sure it clears it in this case too, but I'm using Multiboot 2 for my OS so the behaviour could be different.
Re: Kernel 101 – Let’s write a Kernel
#79Can anyone give me an idea how much different this would be for 64bit? Do I just change the nasm directive to `bits 64`?
> Can anyone give me an idea how much different this would be for 64bit? Do I just change the nasm directive to `bits 64`?
A lot different. You can see the boot code of my x86_64 hobby kernel project [1].
The reason is that GRUB/Multiboot protocol is actually doing most of the machine initialization, but it can only set up 32 bit mode. 64 bit mode is missing partly because standardizing the Multiboot protocol is dragging behind, partly because there's no one correct way to do this as you can't have identity mapped memory in 64 bit mode (unlike 32 bit mode).
If you read the OSDev wiki, you can find examples of doing machine initialization "from scratch", ie. after the PC BIOS (or UEFI). This involves arcane details about the x86 machine like setting up something called the A20 line (which was a hack that allows to have 1 megabyte of memory - utilizing a spare pin from the keyboard controller!), etc. Dealing with this shit is not time well spent. (UEFI is a lot easier in some ways, harder in others)
This means that the boot code of your kernel must set up long mode, create an initial page table for virtual memory, etc. Here's my limited 64 bit boot code that sets up one 2 megabyte page [2].
Going to 64 bit mode is not that much more code, but it will make kernel development more painful. In particular, switching CPU modes messes up the GDB debugger which must be patched to work at all. And there's a bit more work involved in all the little things that come with it.
So for educational purposes it would make more sense to stick to 32 bit mode than deal with the nitty gritty details of 64 bit long mode.
[1] https://github.com/rikusalminen/danjeros [2] https://github.com/rikusalminen/danjeros/blob/master/src/arc...
Re: Kernel 101 – Let’s write a Kernel
#80Looks great, will keep an eye on this! I also really enjoyed James Molloy's OS kernel development tutorial at http://www.jamesmolloy.co.uk/tutorial_html/ , which takes you from "Hello World" to some real toy OS kernel implementation.
It's worth pointing out there are a few bugs in James Molloy's tutorial [1], and some of the things he does in them aren't exactly best practices - for example, a few I remember are:
- Disabling interrupts and paging (which also has the side effect of flushing the TLB) to copy memory around by physical address. This could be done without disabling them by mapping all of physical memory into virtual memory instead (possible in 64-bit mode, but in 32-bit there isn't enough room when your PC has a similar amount of RAM to virtual memory space, in which case you could map smaller parts of it as needed).
- Moving the stack around to get around the fact that GRUB doesn't set ESP to some well-defined value (instead of defining the stack yourself, which would be much more robust) and then attempting to rewrite the base pointers to fix it. For example, his code can't tell the difference between integers that just happen to have a value in the range of the pointers and a pointer, and will happily rewrite both. Also as ESP isn't defined by the Multiboot standard you could be using any location at all as the stack (such as some memory address that doesn't exist, or your kernel's code itself, or some memory-mapped area for a piece of hardware, etc.) All of which will mean things go wrong. It's better to just set ESP yourself before you enter C - see another of my comments on this submission here [3].
There's actually a newer and much better version of JamesM's tutorials on GitHub, but I believe they aren't quite finished [2].
[1]: http://wiki.osdev.org/James_Molloy%27s_Known_Bugs [2]: https://github.com/jmolloy/JMTK [3]: https://news.ycombinator.com/item?id=7590753