Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

IncludeOS – A minimal, resource efficient unikernel for cloud services

includeos.org

Post reply on HN
Print view View original

41–50 of 85 posts

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#41

Post by bizzleDawg »

fwsgonzo wrote: post #32
tinktank wrote: post #27

I have never really got unikernels, maybe someone could educate me? - There is the argument of reduced attack surface/typechecked safety. I get that, but I don't see how this is different from compiling custom Linux/NetBSD kernels per VM? - There is the argument for increased performance. I don't get that at all -- I've done experiments on unikernel system for MirageOS vs IncludeOS vs Kernel on a web and memcache ser…

I am a kernel developer working on IncludeOS. Take everything I say with a grain of salt as this is just my thoughts at 9 in the morning in written form. 1. I can build a bootable binary without the full TCP and UDP, shaving off 60% of the whole IP stack. LTO removes a large amount of code as well. It makes all the difference. 2. Performance comes from building a fixed binary (all memory addresses are constant), as w…

I was reading this thinking that security would be mentioned at some point in your response! Presumably a big benefit of smaller codebases is a more realistic possibility of auditing the security?

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#42

Post by jacobush »

tannhaeuser wrote: post #40
perbu wrote: post #38

Earlier quoted context omitted.

IncludeOS can support most of POSIX. It uses Musl, which provides a full POSIX implementation for the system calls that are implemented. Obviously things like fork() won't work.

> Obviously things like fork() won't work. And that's a pretty tough constraint in non-GCd environments, as now you can't rely on a host O/Ss process resource management. Manual memory management in C and C++ apps and long-running services make all the difference in terms of development complexity because of memory fragmentation, async (where memory can be required/released at any time), etc.

Good riddance, I say. The cloud and programming in general could stand to take some pages from the embedded programmers cookbooks, security in general will improve. Plenty of programs are written such that they allocate all the memory ever needed up front, and that's it.

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#43

Post by presscast »

A few silly questions from someone who comes from containers (Docker, specifically).

1. Can I "run" and "manage" unikernels on my development machine like I would a Docker image? (Use case: keeping build and deploy environments identical)

2. What's the workflow for inspecting running containers?

3. Is it possible to work in languages other than C++?

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#44

Post by fwsgonzo »

bizzleDawg wrote: post #41
fwsgonzo wrote: post #32

Earlier quoted context omitted.

I am a kernel developer working on IncludeOS. Take everything I say with a grain of salt as this is just my thoughts at 9 in the morning in written form. 1. I can build a bootable binary without the full TCP and UDP, shaving off 60% of the whole IP stack. LTO removes a large amount of code as well. It makes all the difference. 2. Performance comes from building a fixed binary (all memory addresses are constant), as w…

I was reading this thinking that security would be mentioned at some point in your response! Presumably a big benefit of smaller codebases is a more realistic possibility of auditing the security?

Less code means less attack surface and easier auditability. VMs are traditionally very isolated, and guests running in special hypervisors like uKVM (and others) are even more so. The simpler hypercall API simplifies things. No shared kernel, like with containers. Immutability.

I could write about this in much more detail, but then I would have to have help writing it. So I won't.

I should also add that unikernels traditionally run with kernel privileges, but that is not a requirement. They can run just fine with some kind of modern hypervisor that sets everything up beforehand. In that case, the unikernel runs like a normal userspace program, just inside a VM. All communication would be with MMIO and MSRs that don't exit.

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#45

Post by chrisper »

fwsgonzo wrote: post #32
tinktank wrote: post #27

I have never really got unikernels, maybe someone could educate me? - There is the argument of reduced attack surface/typechecked safety. I get that, but I don't see how this is different from compiling custom Linux/NetBSD kernels per VM? - There is the argument for increased performance. I don't get that at all -- I've done experiments on unikernel system for MirageOS vs IncludeOS vs Kernel on a web and memcache ser…

I am a kernel developer working on IncludeOS. Take everything I say with a grain of salt as this is just my thoughts at 9 in the morning in written form. 1. I can build a bootable binary without the full TCP and UDP, shaving off 60% of the whole IP stack. LTO removes a large amount of code as well. It makes all the difference. 2. Performance comes from building a fixed binary (all memory addresses are constant), as w…

Any reasons you are avoiding containers?

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#46

Post by Polyisoprene »

tannhaeuser wrote: post #35
ckocagil wrote: post #12

Earlier quoted context omitted.

I don't understand all the hatred towards containers. It's simply an OS abstraction to facilitate isolation. What's wrong with that? In fact I'd love to see more containers, especially on the Linux desktop: for example a container aware compositor that could render windows from container processes and color code their borders, like Qubes does. I have nothing against VMs either. But even when they're tiny (as in the c…

Docker-style containers only shield you from mixed-library situations eg. where one "service" needs version A and another needs version B. It's painful because this situation is entirely accidental and could be trivially solved by linking everything statically. Then once you have two or more containers up and running, these libraries still need security updates and other fixes which is the entire point of shared libs…

Containers have more than just filesystem namespacing.

You also get a network namespace, process namespace, user namespace and ability to set cpu/memory limits.

Since you would have a container for each application, you have an easier time setting and testing restrictive apparmor/selinux capabilities and even get some hardening out of the box.

Sure you could get them without containers, but the whole benefit is doing it in a standard, easy way.

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#47

Post by fwsgonzo »

chrisper wrote: post #45
fwsgonzo wrote: post #32

Earlier quoted context omitted.

I am a kernel developer working on IncludeOS. Take everything I say with a grain of salt as this is just my thoughts at 9 in the morning in written form. 1. I can build a bootable binary without the full TCP and UDP, shaving off 60% of the whole IP stack. LTO removes a large amount of code as well. It makes all the difference. 2. Performance comes from building a fixed binary (all memory addresses are constant), as w…

Any reasons you are avoiding containers?

I am simply unfamiliar with them. Created one container once, and that's it. If I knew them better I would probably use them more.

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#48

Post by perbu »

tannhaeuser wrote: post #40
perbu wrote: post #38

Earlier quoted context omitted.

IncludeOS can support most of POSIX. It uses Musl, which provides a full POSIX implementation for the system calls that are implemented. Obviously things like fork() won't work.

> Obviously things like fork() won't work. And that's a pretty tough constraint in non-GCd environments, as now you can't rely on a host O/Ss process resource management. Manual memory management in C and C++ apps and long-running services make all the difference in terms of development complexity because of memory fragmentation, async (where memory can be required/released at any time), etc.

fork() requires support for multiple processes. IncludeOS only has one process, so a fork() doesn't make sense there.

Also, I should point out that relying on fork/exec to clean up long running processes sound like something of a hack. Would you really like to have a system that is required to restart itself ever to often as a strategy to manage memory?

We've just spent quite a bit of time implementing a buddy allocator to combat memory fragmentation on long running systems. It seems to do a good job and I believe it is more or less a solved problem.

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#49

Post by hestefisk »

Thaxll wrote: post #13
jmspring wrote: post #10

Earlier quoted context omitted.

Jails and Zones did their jobs before LXC was a thing. The better technology doesn’t always win out — see Windows history.

Jails and Zones are inferior to Docker overall, the advantage of Docker is not the isolation technology but the whole workflow of using those containers.

You can get a similar workflow with Jails. Docker became popular because of the CLI but there are many similar options now on FreeBSD.

Re: IncludeOS – A minimal, resource efficient unikernel for cloud services

#50

Post by tannhaeuser »

perbu wrote: post #48
tannhaeuser wrote: post #40

Earlier quoted context omitted.

> Obviously things like fork() won't work. And that's a pretty tough constraint in non-GCd environments, as now you can't rely on a host O/Ss process resource management. Manual memory management in C and C++ apps and long-running services make all the difference in terms of development complexity because of memory fragmentation, async (where memory can be required/released at any time), etc.

fork() requires support for multiple processes. IncludeOS only has one process, so a fork() doesn't make sense there. Also, I should point out that relying on fork/exec to clean up long running processes sound like something of a hack. Would you really like to have a system that is required to restart itself ever to often as a strategy to manage memory? We've just spent quite a bit of time implementing a buddy alloca…

I don't agree multiple processes are hackish at all. CPUs/MMUs have dedicated hardware for managing memory, O/Ss have tools to limit and monitor memory usage, and separate memory regions/segments can be very effective and simple means for security isolation (save for meltdown/rowhammer-style attacks). I know this could be used as a general argument against any innovation, but single-process-per-requests have worked well since inetd (4.3BSD in 1977). Considering hardware has improved orders-of-magnitudes since, what is the advantage of doing the same with very restricted programming models, especially when developer costs are dominating new service developments?
Post reply on HN