Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

IncludeOS roadmap: tiny Node.js-style web services in highly efficient C++

github.com

Post reply on HN
Print view View original

11–20 of 45 posts

Re: IncludeOS roadmap: tiny Node.js-style web services in highly efficient C++

#11

Post by hanez »

semisight wrote: post #9
hanez wrote: post #6

Sounds a little bit crazy to me at first but when thinking about it, it is a nice idea. May you should take a look at Rump Kernels and build your stuff on top. Then you do not need to implement the OS stuff - It's done already. May I am wrong but it seems to be a similiar idea but the following project is currently at the OS level only but some applications like ngnix are working already. I was very confused when I f…

This seems more like MirageOS/Ling than a rumpkernel. From what I remember, rumpkernels are more general, at the cost of being less "unikernel-y" (i.e. lean and fast). I agree that it would be easier and faster to use a rumpkernel if the intention is use in production soon . I can't tell if this is a research project, or intended to be production quality at some point.

I believe to get this production ready will take some years.

From what I understand in the projects FAQ page they want to implement the concept of a Unikernel and this is what the Rump Kernel at http://rumpkernel.org/ is intending too. May I am wrong because I am not to deep into it.

If it is a research project then they should go on... It looks like it is because it is beeing developed at a university in Oslo, Norway. http://www.hioa.no/eng/

Re: IncludeOS roadmap: tiny Node.js-style web services in highly efficient C++

#12

Post by timClicks »

hanez wrote: post #6

Sounds a little bit crazy to me at first but when thinking about it, it is a nice idea. May you should take a look at Rump Kernels and build your stuff on top. Then you do not need to implement the OS stuff - It's done already. May I am wrong but it seems to be a similiar idea but the following project is currently at the OS level only but some applications like ngnix are working already. I was very confused when I f…

There is actually an active unikernel project called IncludeOS (http://www.includeos.org/), which could be a good place to start.

Re: IncludeOS roadmap: tiny Node.js-style web services in highly efficient C++

#13

Post by hanez »

timClicks wrote: post #12
hanez wrote: post #6

Sounds a little bit crazy to me at first but when thinking about it, it is a nice idea. May you should take a look at Rump Kernels and build your stuff on top. Then you do not need to implement the OS stuff - It's done already. May I am wrong but it seems to be a similiar idea but the following project is currently at the OS level only but some applications like ngnix are working already. I was very confused when I f…

There is actually an active unikernel project called IncludeOS ( http://www.includeos.org/ ), which could be a good place to start.

Exactly this is what the original post is about... ;)

Re: IncludeOS roadmap: tiny Node.js-style web services in highly efficient C++

#16

Post by kator »

I love watching the pendulum swing back and forth in tech.

From mainframes to pc's ... to thin clients ... to server rendered HTML ... to client side frameworks ... to microservices

I'm sure I could come up with more examples. I was just thinking of re-architecting a heavy apache web head app into unikernel super light micro services. It just makes me chuckle as we swing back and forth and try to find the right balance. Each generation sort of saying "Hey wait there's a CPU over here on this side of the diagram and it could do X/Y/Z".

Recently I've been working on Raspberry Pi's so much that when I logged into a nice medium sized EC2 instance I was "shocked at how responsive these are". LOL. I started on machines with 4k of memory and built systems that consume TB of ram across many nodes and data centers. It's fun to go back to my roots trying to cram something into an ESP8266 nodemcu device and thinking about how long I can run it on a LiPo battery. I think this back and forth is good for us, but I encourage everyone to spend time on micro devices to remember just how simple you can solve a problem when confronted with the reality of 20k of ram and 4M of flash storage.

Re: IncludeOS roadmap: tiny Node.js-style web services in highly efficient C++

#17

Post by anttiok »

hanez wrote: post #11
semisight wrote: post #9

Earlier quoted context omitted.

This seems more like MirageOS/Ling than a rumpkernel. From what I remember, rumpkernels are more general, at the cost of being less "unikernel-y" (i.e. lean and fast). I agree that it would be easier and faster to use a rumpkernel if the intention is use in production soon . I can't tell if this is a research project, or intended to be production quality at some point.

I believe to get this production ready will take some years. From what I understand in the projects FAQ page they want to implement the concept of a Unikernel and this is what the Rump Kernel at http://rumpkernel.org/ is intending too. May I am wrong because I am not to deep into it. If it is a research project then they should go on... It looks like it is because it is beeing developed at a university in Oslo, Norwa…

No, a rump kernel is not an unikernel, check the FAQ linked from http://rumpkernel.org/

However, you can use rump kernels as a major component of a unikernel implementation. A rump kernel provides environment-agnostic drivers, meaning you can integrate them pretty much anywhere.

Now, what is a unikernel? From my perspective it's essentially: 1) application 2) config/orchestration 3) drivers 4) nibbly "OS" bits

So from the bottom, the nibbly bits include things such as bootstrap, interrupts, thread scheduler, etc. It's quite straightforward code, and a lot simpler that the counterpart you'd find e.g. in Linux. But you can't do much anything useful with the OS when that part is written.

Drivers are difficult because you need so many of them for the OS to be able to do much anything useful, and some drivers require incredible amounts of effort to make them real-world bug compatible. Just consider a TCP/IP stack -- you can write one from scratch in a weekend, but the result won't work on the internet for years. Then you may need to pile on a firewall, IPv6, IPsec, .... A rump kernel will provide componentized drivers for free. The policy of if you use those drivers in a unikernel or microkernel or whateverkernel is up to you, but I guess here we can assume unikernels.

The config/orchestration bits are actually quite an interesting topic currently, IMHO, at lot of opportunities to make great discoveries. Also, a lot of opportunities to use the rope in the wrong way.

The applications depend on what sort of interfaces your unikernel offers. If it offers a POSIX'y interface, you can run existing applications, otherwise you need to develop them for the unikernel.

Now putting rump kernels and unikernels together: the nibbly bits are straightforward, the drivers come for free via rump kernels, and those drivers provide POSIX syscall handlers, so POSIX'y applications just work. That leaves the config/orchestration stuff on the table. There's a rumpkernel-based unikernel called Rumprun available from repo.rumpkernel.org. It's essentially about solving the config/orchestration problems. Due to the rump kernel route, the other problems were already solved in a way which can be considered "good enough" for our purposes.

Hope that clarified the difference between rump kernels and unikernels.

(edit: minor formatting fix)

Re: IncludeOS roadmap: tiny Node.js-style web services in highly efficient C++

#18

Post by kentonv »

std_throwaway wrote: post #2

At first it seemed strange that a Node.js-replacement needs a full IP stack including DHCP. As far as I understood this, they are building a whole operating system to be run in a VM like QEMU which then acts like a Node.js-instance - except that it's programmed in C++ and linked as one whole exeutable which also is a complete OS. I wonder how this compares to Docker or Sandstorm where the existing kernel API is used…

> I wonder how this compares to Docker or Sandstorm where the existing kernel API is used in a virtualized container instead of emulating a whole machine.

Short answer is probably: Better at security, but less efficient.

Xen bugs notwithstanding, the VM boundary has a better security record than the kernel API boundary. (Though Sandstorm -- my project -- has been pretty successful at dodging kernel bugs through aggressive attack surface reduction.)

The problem with VMs is that they're heavy-weight. The hardware interfaces were never intended to be a clean abstraction boundary between software. For example, it's tricky to reclaim unused RAM back from a VM, because within the VM the guest would by default assume it has a fixed amount of RAM that is exclusive to it. Communication probably has to happen in the form of network packets, requiring setting up and traversing a whole IP network stack in the guest, which is a lot more expensive (in both CPU time and memory use) than pipes, unix sockets, or shared memory would be.

However, if you're designing a kernel specifically for use in a VM, you can probably do better on these things, e.g. you can define some virtual hardware interface by which the kernel marks RAM pages unused so that the host can take them back, or you can define a cleaner communication interface. But as you define these interfaces, your "virtual hardware API" ends up looking more and more like a kernel API -- and has greater risk of security bugs.

So one way (containers) you start out with a wide interface and high efficiency but a large attack surface, and you try to narrow the interface to improve security. The other way (VMs) you start with a narrow but inefficient interface which you try to widen to improve performance.

Eventually they will probably converge.

Re: IncludeOS roadmap: tiny Node.js-style web services in highly efficient C++

#19

Post by nickpsecurity »

Gladdyu wrote: post #3

I wrote something similar a while ago, combining the easy-to-use asynchronous programming model of Node.js with the speed of C++. http://gladdy.github.io/c++/2015/07/26/NodeDemystified-pt1.h... source code: https://github.com/Gladdy/Mininode

The example looks really clean. Amazing how far modern C++ has gotten in that area.

Re: IncludeOS roadmap: tiny Node.js-style web services in highly efficient C++

#20

Post by nickpsecurity »

hanez wrote: post #13
timClicks wrote: post #12

Earlier quoted context omitted.

There is actually an active unikernel project called IncludeOS ( http://www.includeos.org/ ), which could be a good place to start.

Exactly this is what the original post is about... ;)

Maybe it's a bot designed to promote IncludeOS. Gives a stock response to anything on the Internet that has IncludeOS in its name. That would make more sense.
Post reply on HN