Live data from Hacker News

How to Make a Computer Operating System in C/C++

github.com

41–50 of 88 posts

Re: How to Make a Computer Operating System in C/C++

#41
post #40

Maybe a little irationally, the idea of programming an os in c++ strikes me as very opaque. I think the vipri[1] approach of layering dsls, or the smalltalk idea of a relatively simple vm to seem more understandable than an os that embeds a c++ runtime... As a side note, when looking up [1] I also ran across [2]. [1] http://piumarta.com/software/cola/ [2] http://www.acm.uiuc.edu/sigops/roll_your_own/1.helloworld.ht..…

BeOS was primarily written in C++, although the kernel was dominantly C. For actual parts that require user interaction, C++ and the object model makes a whole lot of sense, although there are probably more mature languages out that would be better candidates now.

Re: How to Make a Computer Operating System in C/C++

#42
post #25

Earlier quoted context omitted.

> write a very simple UNIX-based operating system in C++ (does anyone else smell a contradiction?) I'm going to use this as an opportunity to champion C++ for systems programming despite the fact that it wasn't really your argument. I feel obligated to do this because I agreed with you for a long time on this but have changed my mind over the past year or two. The argument against C++ as a systems programming languag…

You can have generalized containers in C, using the intrusive pattern. For example, Linux's list.h has an intrusive list that is far better than the abysmal std::list from the STL.

A more complete set can be pulled in with libcontainer: http://agottem.com/libcontainer

Re: How to Make a Computer Operating System in C/C++

#43
post #14

I am a graduate student, currently working on building a x86_64 unix like preemptive kernel from scratch, as part of a course. Most of the OS dev guides and books focus on 32 bit arch and I haven't found a single guide so far that is based on 64 bit arch. Since this this guide seems to be in its inception, I hope someone (hopefully me) will send a pull request for a 64 bit tutorial. Building an OS has been in my buck…

There's absolutely a void when it comes to a full amd64 assembly (and associated low level) code. It's a shame, because amd64 is actually rather plesant (as opposed to x86) -- and now rather ubiqutous.

I haven't really looked at the code, but I guess the following at least contains a bit of example code:

http://www.returninfinity.com/baremetal.html

Menuetos also looks rather interesting, unfortunately, as I recall the license for the 64bit version puts the code in a bit of a limbo as far how it's actually useful. There's a fork, but not for amd64 afaik..

http://menuetos.net/

http://kolibrios.org/en/

Re: How to Make a Computer Operating System in C/C++

#44
post #37

Earlier quoted context omitted.

I found a stack overflow question that seems to go over it. Basically it looks like there's almost always a better choice for whatever you're doing. http://stackoverflow.com/questions/18449038/is-there-ever-a-...

That answer is attributing the badness to linked lists, whereas it is fully std::list's. Linked lists are very useful, but it's hard to see that when their canonical (and almost only) implementation is std::list, which is indeed almost entirely useless.

I still don't understand why std::list is bad. Could you explain please?

Re: How to Make a Computer Operating System in C/C++

#45
post #37

Earlier quoted context omitted.

That answer is attributing the badness to linked lists, whereas it is fully std::list's. Linked lists are very useful, but it's hard to see that when their canonical (and almost only) implementation is std::list, which is indeed almost entirely useless.

I still don't understand why std::list is bad. Could you explain please?

std::list can be used in 2 ways:

* With an std::list::iterator in each of your data nodes that represents its own position in the list (this is called the "intrusive style")

* Without an std::list::iterator in each of your data nodes

If you use the (more common) latter form: whenever you have a reference to your own object, you cannot do any of the linked list operations without an O(N) penalty to go and re-find your element in the list!

i.e: Say you have a list of requests, and a timeout callback pops up with a pointer to your request, you cannot use a request pointer to do an O(1) deletion from the list. This kind of operation is what lists are for, and std::list canonically cannot do it.

Any other operation you might want to do relating to the list, given such a pointer is impossible (e.g: create a new request that is immediately between the old request and its next).

All this assumes your object is within just one list. If it is within 2 lists, you have to use an extra indirection: std::list , which makes the problem worse. Even if you do find your request via one of the lists, there is no way to get an iterator for the other list. That means you cannot do any of the list operations on the other list. Again: This is the canonical thing lists were designed for, and std::list cannot do it.

Say that to solve this, you use the intrusive style with std::list. i.e: for every list this object is a member of, you hold an iterator inside your object.

Now the onus is on your to maintain these iterators, in addition to the common list operations. i.e: If you add an element, you need to both call std::list::add, and update the iterator.

Additionally, instead of paying with 2 pointers for each node, as an ordinary doubly-linked list should cost, you have to pay with an extra pointer or two (depending on how the iterator is implemented)!

If you use multiple lists with std::list, you pay with an extra pointer yet!

So if your data structure is within 2 doubly-linked lists, instead of paying the ideal 4 pointers per item, you pay those 4 + 1(indirection) + (2 for two iterators). 75% memory overhead, ruining your cache lines.

The code will be a mess too, due to the duplicate maintenance.

The alternative is much simpler: http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/i...

Use the intrusive approach exclusively. So that you don't need to use a list::add in addition to maintaining the two iterators. You only maintain the "iterators" (now called "list heads").

So your request would look like:

  struct request { struct list_head list1, list2; };
To add it to a list:

  list_add(&request->list1, &some_list_head);
Given a request, you can delete it in O(1) from both lists:

  list_del(&request->list1);
  list_del(&request->list2);
Easy to use and optimal memory use (exactly 2 pointers per list head).

Now, given some pointer from list2, to get back a request, you use:

  struct list_head *some_ptr = ...;
  struct request *req =
    containerof(some_ptr, struct request, list2);
This is slightly-boilerplatey, so I tend to write a little wrapper:

  struct request *request_of_list2(struct list_head *ptr) {
    return containerof(ptr, struct request, list2);
  }
EDIT: almost completely forgot that std::list also uses new to dynamically allocate elements. This effectively means the cost of adding to lists is many times greater than the simple list_add function. Even if you supply your own allocator, this is unnecessarily expensive and by default means that list::add, like remove, are both not O(1) like they ought to be.

Re: How to Make a Computer Operating System in C/C++

#46
post #34

Earlier quoted context omitted.

How about: Any C or C++ programmer knows not to use linked lists anyway.

If you really think that, you've missed CS 101. Or maybe std::list is the only linked list implementation you've seen. In that case, I agree, one should never use std::list. Linux's list.h is extremely useful, and for a wide variety of circumstances, is the most efficient way to manage your data.

Ok, I'll elaborate, especially since my view is at odds with your statement "for a wide variety of circumstances".

As I see it, the only use case where linked lists are superior to other types of lists, like, perhaps, ArrayList in java or vector/deque in C++, is if the following conditions are met:

1: you care about ordering - often you don't care about ordering and in that case, there is no need for a linked list because you can achieve O(1) insertion & removal then too: insertion can always be at the end, removal can be a "swap with end element, remove end element" operation.

2: inserting and/or removing from the middle of the list is a common operation, if it is not a common operation, then the added cost of doing so with a vector may still be outweighed by a vectors other advantages

3: you do not require random access - lists do not provide random access and lookup is O(n). At the expense of additional complexity in implementation and more memory overhead, you could reduce lookup to, OTOH, O(log n) by using a skip-list

4: you do not iterate through the list often - if you do, you are likely going to blow the cache and mess up prefetching due to poor cache locality. Iterating through an array-based data structure can be much faster in this case.

I would say that for a list to make sense, you MUST have 1 and 2 and probably should have 3. 4 is optional, but if true, should make you consider if there might not be a more suitable data structure. In my own personal experience, this is rare. In fact, in my own personal experience, usually, code either does not require 1 or requires 1 but not 2 - either way, lists are not the appropriate data structure in those cases.

Basically, the short version is that they have very poor cache locality, an (IMHO) narrow use case where other data structures don't have superior performance and they take up more memory per node than a lot of other types of lists.

You linked to a stackoverflow question in another comment saying that they attribute std::list's flaws to linked lists as a whole. The biggest issue they seemed to mention, though, was cache locality - I fail to see how intrusive linked lists solve this. The only solution would be to preallocate nodes in consecutive memory locations, but you still take a hit as the links take up memory (whereas in array-based lists you do not need to store links for each node) and if you need to insert/delete in the middle (why are you using linked lists if this isn't the case?) then you end up jumping around the preallocated nodes anyway and after a while will lose any cache-friendliness you may have had.

Maybe you can elaborate what you meant?

To end with an appeal to authority ;-) I'll quote tptacek[1]:

C programmers are trained to use linked lists. They are the first variable-length containers most programmers come into contact with and so C programmers tend to be imprinted on them like ducklings. Linked lists are a poor general purpose container.

EDIT: I guess I missed an anti-condition: if you don't care about performance, then use whatever models your problem best, which may well be a linked list (though the same is true for std::list).

[1] https://news.ycombinator.com/item?id=4455676

Re: How to Make a Computer Operating System in C/C++

#47
post #14

I am a graduate student, currently working on building a x86_64 unix like preemptive kernel from scratch, as part of a course. Most of the OS dev guides and books focus on 32 bit arch and I haven't found a single guide so far that is based on 64 bit arch. Since this this guide seems to be in its inception, I hope someone (hopefully me) will send a pull request for a 64 bit tutorial. Building an OS has been in my buck…

There's some stuff about 64 bit over at http://xomb.org

Re: How to Make a Computer Operating System in C/C++

#48
post #40

Maybe a little irationally, the idea of programming an os in c++ strikes me as very opaque. I think the vipri[1] approach of layering dsls, or the smalltalk idea of a relatively simple vm to seem more understandable than an os that embeds a c++ runtime... As a side note, when looking up [1] I also ran across [2]. [1] http://piumarta.com/software/cola/ [2] http://www.acm.uiuc.edu/sigops/roll_your_own/1.helloworld.ht..…

BeOS was primarily written in C++, although the kernel was dominantly C. For actual parts that require user interaction, C++ and the object model makes a whole lot of sense, although there are probably more mature languages out that would be better candidates now.

I loved BeOS and the C++ application framework... except for the linking model.

You had to provide re-linked executables for every new version of BeOS that came out. It was exceedingly lame to go to an application download page and have to select the version of the application that corresponded to the version of BeOS you installed.

Then when you upgraded your OS, you had to upgrade ALL of your applications. Sure, there were some framework upgrades that didn't necessitate a re-linked application, but as a user you never really knew which those were so had to do a full upgrade of everything to be sure.

Re: How to Make a Computer Operating System in C/C++

#49
post #48

Earlier quoted context omitted.

BeOS was primarily written in C++, although the kernel was dominantly C. For actual parts that require user interaction, C++ and the object model makes a whole lot of sense, although there are probably more mature languages out that would be better candidates now.

I loved BeOS and the C++ application framework... except for the linking model. You had to provide re-linked executables for every new version of BeOS that came out. It was exceedingly lame to go to an application download page and have to select the version of the application that corresponded to the version of BeOS you installed. Then when you upgraded your OS, you had to upgrade ALL of your applications. Sure, the…

Agreed. Although, I don't understand why they didn't provide "shim" .so files. I think this is what linux does, though in practice these days I just use ubuntu packages... And at the time to do the same thing in linux, or FreeBSD (which is what I migrated to when I left BeOS) you had to - or were encouraged to - recompile everything from source.

Re: How to Make a Computer Operating System in C/C++

#50
post #22
post #14

I am a graduate student, currently working on building a x86_64 unix like preemptive kernel from scratch, as part of a course. Most of the OS dev guides and books focus on 32 bit arch and I haven't found a single guide so far that is based on 64 bit arch. Since this this guide seems to be in its inception, I hope someone (hopefully me) will send a pull request for a 64 bit tutorial. Building an OS has been in my buck…

Building OS' is fun. I wish I could spend all day doing it but I've been pushed so far up the stack that I spend most of the time arguing with analysts and preparing documentation... I've built two so far which were (and I think still were until recently) used in production equipment. One Forth system (all 8k of it) that ran a PLC system and a tiny (16k) kernel for an M68k system that was a router for a modbus-like p…

I've been a web developer my entire professional career, and for some ungodly reason I have this need to go work in the embedded sector instead. Am I nuts? ;)
Post reply on HN