Linux compatibility in IncludeOS • Hacker News Board

Re: Linux compatibility in IncludeOS

#21

Post by perbu » Wed, Nov 07, 2018, 12:31 PM UTC

hugelgupf wrote: post #13

perbu wrote: post #11

Earlier quoted context omitted.

(I work for Includeos) I don't think we need to run everything. I think we'll be happy if we can take some common libraries and compile them and they work. That is currently our goal. Down the road, depending on interest, we can perhaps expand a bit. >> Moreover, since implementing a system call is typically pretty simple [...] > You're in for a fun ride, folks. Thanks for the heads up. It was meant as opposed to imp…

That sounds much more attainable than what the blog post suggested - sorry if I took that the wrong way!

No worries. I never meant for the blogpost to reach an audience outside of our pretty narrow knitting circle so I didn't pay much attention to detail. I very much appreciate your insights here.

Re: Linux compatibility in IncludeOS

#22

Post by perbu » Wed, Nov 07, 2018, 12:40 PM UTC

hugelgupf wrote: post #9

hugelgupf wrote: post #8

Earlier quoted context omitted.

The number of system calls says close to nothing of the actual API surface present here. It's a poor measure. What actually matters is: what can I run? That said, 42 is not enough to run a lot of applications. It'll be interesting to get to the point where they can run a language's runtime or unit tests. Take it from someone who worked on gVisor: this is gonna take a long time, and eventually you'll end up with a sys…

Given what I just read about IncludeOS (I'm not really familiar), it feels like implementing as much of musl as possible is probably the best you can do compatibility wise. There are going to be a lot of Linux features simply incompatible with includeOS, so claiming that 400 syscalls will be implemented is unlikely. That said - why do you have to get musl compatibility by implementing the Linux syscalls? Is contribut…

Your assumption is correct. IncludeOS will never be fully compatible with Linux. Our goal is to provide "just enough POSIX" so that the applications that are relevant for the OS will compile and run. Mostly we're concerned with various "gateway-like" applications - firewalls, load balancers and down the line also IoT-gateways.

Wrt. to why we have to implement Linux-like syscalls; Musl is written specifically for Linux so in order to "unlock" the relevant POSIX-bits that Musl can provide we have to provide the underpinnings that Musl requires.

I'm hoping we can upstream some of our non-intrusive changes to Musl. However since Musl has a very specific purpose (a Linux-specific C and POSIX library) I'm not certain that our changes are within the scope of the project. To date we've only concerned ourselves with a small subset of Musl and only on amd64.

Re: Linux compatibility in IncludeOS

#23

Post by perbu » Wed, Nov 07, 2018, 12:48 PM UTC

hardwaresofton wrote: post #4

If you're into unikernels, rumpkernels[0] are the least-modification-required approach I've seen. Basically, if you compile your program with a minimally (as minimal as is possible) modified compiler, you get "rumprunnable" output. there's a list of popular packages built to run on rump kernels[1] (Rust is supported[2]). I haven't tried to use the toolchain yet, but it's pretty appealing to me. The biggest issue is t…

I'm hoping we're be able to port libuv to IncludeOS (which should be doable, much like libuv IncludeOS is eventdriven). Then we should be able to get Node up and running. Running Node.js through IncludeOS on Nabla could be interesting in a "FaaS" context.

Last I heard the author of Rumprun has picked up beer brewing full time and isn't interested in maintaining the project.

It would be great if someone where to pick it up.

OSv is also very relevant. They provide a much richer and more compatible runtime environment compared to IncludeOS (I think they even have zfs). And OSv is pretty active these days, so if you looking for a way to run some high performance stuff OSv is worth looking into.

Re: Linux compatibility in IncludeOS

#24

Post by AnIdiotOnTheNet » Wed, Nov 07, 2018, 1:06 PM UTC

jetzzz wrote: post #14

Looks like Linux syscalls are becoming some kind of de-facto standard. They are already implemented by Linux, FreeBSD, Windows and now IncludeOS.

The mind reels.

Not because Linux is a bad kernel, mind you, but because UNIX itself reeks of ideas that were ok when it was originally developed and clunky at best today. Even Microsoft seems to have given up. The future of computing looks pretty grim to me.

Re: Linux compatibility in IncludeOS

#25

Post by chris_wot » Wed, Nov 07, 2018, 1:11 PM UTC

Can someone point me to a more detailed overview of IncludeOS than the brief summary they have on their home page?

Re: Linux compatibility in IncludeOS

#26

Post by chris_wot » Wed, Nov 07, 2018, 1:14 PM UTC

__bjoernd wrote: post #6

freedomben wrote: post #2

I'm pretty ignorant on Unikernels so maybe there's another solution, but not being able to `fork()` is a significant problem for many applications.

My understanding is that unikernels are single-application by design. I agree that a mechanism to spawn other apps is useful, but in this case it won't be a simple fork but the creation of a complete new os instance.

There are plenty of applications that used multiple processes.

Re: Linux compatibility in IncludeOS

#27

Post by solarkraft » Wed, Nov 07, 2018, 1:39 PM UTC

AnIdiotOnTheNet wrote: post #24

jetzzz wrote: post #14

Looks like Linux syscalls are becoming some kind of de-facto standard. They are already implemented by Linux, FreeBSD, Windows and now IncludeOS.

The mind reels. Not because Linux is a bad kernel, mind you, but because UNIX itself reeks of ideas that were ok when it was originally developed and clunky at best today. Even Microsoft seems to have given up. The future of computing looks pretty grim to me.

Could you elaborate?

Re: Linux compatibility in IncludeOS

#28

Post by perbu » Wed, Nov 07, 2018, 1:53 PM UTC

leni536 wrote: post #16

Would it be possible to run Linux as a unikernel and port your application as a kernel module? How a leaned down Linux kernel would compare to includeOS?

There is a project that tries to turn Linux into a library which could lead to a unikernel approach with the Linux. If you look at https://github.com/lkl you can see where they are.

A leaned down Linux-kernel with the application linked into kernel space would be pretty comparable. But it would be pretty enormous beast.

Re: Linux compatibility in IncludeOS

#29

Post by perbu » Wed, Nov 07, 2018, 1:54 PM UTC

chris_wot wrote: post #26

__bjoernd wrote: post #6

Earlier quoted context omitted.

My understanding is that unikernels are single-application by design. I agree that a mechanism to spawn other apps is useful, but in this case it won't be a simple fork but the creation of a complete new os instance.

There are plenty of applications that used multiple processes.

Absolutely. And most of those are deemed out of scope for Unikernels.

Re: Linux compatibility in IncludeOS

#30

Post by perbu » Wed, Nov 07, 2018, 2:26 PM UTC

chris_wot wrote: post #25

Can someone point me to a more detailed overview of IncludeOS than the brief summary they have on their home page?

For a technical, in-depth overview of IncludeOS and how it was made I find the main authors presentation at Cppcon 2017 is a pretty good introduction:

https://www.youtube.com/watch?v=h7D88U-5pKc (one hour, quite entertaining).

There is also the original paper which was presented at the 2015 IEEE 7th International Conference on Cloud Computing Technology and Science (CloudCom).

https://ieeexplore.ieee.org/abstract/document/7396164