Live data from Hacker News

IncludeOS – A minimal, resource efficient unikernel for cloud services

includeos.org

31–40 of 85 posts

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#31

It seems that hypervisors have developed a whole new level of OS API/ABI, sort of like the POSIX specification.

Except there is no common specification, and there's not going to be one either because "standards are for loosers". No meaningful standard has been developed in this decade. It's almost as if the situation with two or more super-stable, compatible POSIX O/Ss, compiler suites, databases, etc. we had was too good to be true.

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#32

I have never really got unikernels, maybe someone could educate me? - There is the argument of reduced attack surface/typechecked safety. I get that, but I don't see how this is different from compiling custom Linux/NetBSD kernels per VM? - There is the argument for increased performance. I don't get that at all -- I've done experiments on unikernel system for MirageOS vs IncludeOS vs Kernel on a web and memcache ser…

I am a kernel developer working on IncludeOS. Take everything I say with a grain of salt as this is just my thoughts at 9 in the morning in written form.

1. I can build a bootable binary without the full TCP and UDP, shaving off 60% of the whole IP stack. LTO removes a large amount of code as well. It makes all the difference.

2. Performance comes from building a fixed binary (all memory addresses are constant), as well as using LTO and C++. IncludeOS is not at its most performant because we haven't been working on performance. Project is still young. We should be faster overall compared to just about anything that isn't compiling to a fixed binary layout.

3. Isolation can come in many forms. On traditional hypervisors you have the same old same old. While on a newer type of hypervisor you can do just about anything from avoiding VMEXITs altogether to removing the ability to even talk to the hypervisor after configuration stage. If the hypervisor is just a thin configuration program...

4. Yes doubly so if your OS uses threads as well, and the schedulers are almost always beating differently on the hypervisor and the guest. This causes even more trashing than normal. In any case, IncludeOS is not using threads at all. It is fully async and matches the hardware really well. The bottleneck is the multiplexing ability of the cloud platform itself. We can live with that.

5. I don't know much about containers other than that I avoid them. Containers share kernel. Sorry, I can't answer this.

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#33

Can I use gdb to debug my applications?

Yes, build for userspace linux and run it as a normal program. Then use all your favorite tools. You can also attach GDB to a qemu process that is waiting for a connection (abit harder). In a production environment on a public cloud it gets even harder. For that I can only recommend UBsan (undefined-sanitizer), logging and memory dumps.

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#35
post #3

finally the end of lx* containers on the way. not the actual end of course, who knows docker guys could buy this one as they did to some other unikernel companies.

I don't understand all the hatred towards containers. It's simply an OS abstraction to facilitate isolation. What's wrong with that? In fact I'd love to see more containers, especially on the Linux desktop: for example a container aware compositor that could render windows from container processes and color code their borders, like Qubes does. I have nothing against VMs either. But even when they're tiny (as in the c…

Docker-style containers only shield you from mixed-library situations eg. where one "service" needs version A and another needs version B. It's painful because this situation is entirely accidental and could be trivially solved by linking everything statically. Then once you have two or more containers up and running, these libraries still need security updates and other fixes which is the entire point of shared libs in the first place. Basically, Docker is not a solution but part of the problem. But with Docker comes a ton of constraints; for example, basic file access and anything related to permissions is a PITA. And you need overreaching orchestration software such as mesos/marathon, k8s, or openshift. It's not that these tools are bad, but rather that they're nuclear weapons for relative little benefit, requiring expensive devops experts. All so you can pack multiple services on a single physical host (which would be much easier using classical service runtimes), and so your HTTP services (because everything has to be HTTP because of the firewalls) can all listen on virtual port 80/8080.

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#36

Cool! Can this be used with languages other than C++?

Lua and Micropython have been ported - mostly as PoC. I expect us to do some proper ports next years, likely Node and Python. C support is OK, but it should be fairly simple to expand it as MUSL provides IncludeOS with the relevant bits of POSIX as long as IncludeOS can provide the underlying syscalls.

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#37
FWIW the most interesting tidbit I've learned after working with IncludeOS for couple of years is this; The operating system cannot reconfigure itself.

All Unix and Windows-derived systems are meant to have the ability to reconfigure themselves. For a lot of systems there isn't really a good reason why the system itself should have this capability.

My Wifi access point is a good example. The fact that it runs Linux means that a security flaw gives the attacker the tools to fundamentally modify the devices' behavior. The attacker can install sniffers, blockchain miners or spambots, without vendor approval. This makes any RCE bug critical and in general means the device is dangerous to use if the vendor stops patching it.

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#38

It seems that hypervisors have developed a whole new level of OS API/ABI, sort of like the POSIX specification.

Except there is no common specification, and there's not going to be one either because "standards are for loosers". No meaningful standard has been developed in this decade. It's almost as if the situation with two or more super-stable, compatible POSIX O/Ss, compiler suites, databases, etc. we had was too good to be true.

IncludeOS can support most of POSIX. It uses Musl, which provides a full POSIX implementation for the system calls that are implemented.

Obviously things like fork() won't work.

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#39
post #30

This is potentially a big step forward for the cloud, this is also the missing component to push Serverless. Currently if you want to use containers and Serverless you'll need to spin up 200+MB of linux bistro + tons of dependancies that have nothing to do with what you originally planned, that's to say running a container with a virtual machine on it. This kernel would allow to replace Unix for a much more lightweig…

You'd need a small and efficient hypervisor. Ukvm and Solo5 provides such a system and if I remember correctly IncludeOS can boot and shutdown in less than 10ms on those systems.

So yeah, it would be interesting to see this on a serverless platform. Hit me up if you have a serverless platform. :-)

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#40
post #38

Earlier quoted context omitted.

Except there is no common specification, and there's not going to be one either because "standards are for loosers". No meaningful standard has been developed in this decade. It's almost as if the situation with two or more super-stable, compatible POSIX O/Ss, compiler suites, databases, etc. we had was too good to be true.

IncludeOS can support most of POSIX. It uses Musl, which provides a full POSIX implementation for the system calls that are implemented. Obviously things like fork() won't work.

> Obviously things like fork() won't work.

And that's a pretty tough constraint in non-GCd environments, as now you can't rely on a host O/Ss process resource management. Manual memory management in C and C++ apps and long-running services make all the difference in terms of development complexity because of memory fragmentation, async (where memory can be required/released at any time), etc.

Post reply on HN