Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My First Unikernel

roscidus.com

Post reply on HN
Print view View original

11–20 of 53 posts

Re: My First Unikernel

#11

Post by wmf »

sp332 wrote: post #4
dkarapetyan wrote: post #3

I'm a little confused about the direction things are going in. I like high level languages and I like that the OS manages certain resources so I don't have to. This guy is writing directly to block devices from OCaml. Don't get me wrong it's all pretty cool but there is some kind of dissonance there I can't reconcile. Is Xen the new OS now?

Yes, basically. You let the host manage memory, disk, network etc and your program is a guest OS that doesn't need all the extraneous processes that usually go with a whole OS. No background services or cron jobs. Edit: as enduser pointed out, in addition to saving resources, this means there is no context-switching from kernelspace to userspace in your guest VM!

You let the host manage memory, disk, network etc and your program is a [process] that doesn't need all the extraneous processes that usually go with a whole OS. No background services or cron jobs.

So it's like Docker.

Re: My First Unikernel

#12

Post by paulasmuth »

wmf wrote: post #11
sp332 wrote: post #4

Earlier quoted context omitted.

Yes, basically. You let the host manage memory, disk, network etc and your program is a guest OS that doesn't need all the extraneous processes that usually go with a whole OS. No background services or cron jobs. Edit: as enduser pointed out, in addition to saving resources, this means there is no context-switching from kernelspace to userspace in your guest VM!

You let the host manage memory, disk, network etc and your program is a [process] that doesn't need all the extraneous processes that usually go with a whole OS. No background services or cron jobs. So it's like Docker.

No, this is fundamentally different from docker.

Docker/cgroups/namespaces allow you to isolate multiple applications running in userspace on the same kernel to a very high degree.

This gets rid of isolation and multiprocessing altogether and runs a single application without any kernel at all (i.e. with just a very limited amount of support code linked in).

Put simply; the technology behind docker improves isolation between processes and this does the exact opposite.

Re: My First Unikernel

#15

Post by andolanra »

paulasmuth wrote: post #6

This sounds like an awesome and fun project to hack on! However, optimizing around context switches and task preemptions is something you would usually do if your application is actually bound by IO/context switching, is extremely latency sensitive or when you are trying to squeeze the last bits of performance out of a machine. Why did you choose to build such a microoptimized system in a garbage collected language?…

Don't think of it as optimizing around context switches—think of it as just omitting what you don't need, and losing context-switches as a side benefit. A multi-user OS has a lot of stuff which might not be necessary for a single virtualized service (various security mechanisms, lots of file system niceties, running other services, &c), so writing a unikernel like this allows you to select exactly as much as you need for your particular service.

OCaml isn't all that much slower than C++. To use the Programming Language Shootout as a rough esimation[^1], it can even come close to matching C++ in certain programs, and is rarely more than three times as slow. And of course—your type system will catch more errors and your resulting code will be much shorter (and in my opinion, at least, easier to understand.)

Finally: the article didn't say "the Linux kernel"—it said "Ubuntu." The kernel itself might be secure and reliable, but a running Linux system is much, much more than just the kernel. And while C can be security-audited, many of the properties that are important to verify in a C program come entirely for free from something like OCaml—e.g., an arbitrary piece of C code might not segfault given certain input, but a given piece of OCaml code definitely won't. So maybe a running Linux system is "secure enough", but a unikernel like this will have a much smaller attack surface and stronger inherent security properties with basically no extra work.

Disclaimer: I'm not the original author, I'm just speaking generally.

[^1]: http://benchmarksgame.alioth.debian.org/u64/benchmark.php?te...

Re: My First Unikernel

#16

Post by amirmc »

enduser wrote: post #5

If it helps to add clarity to what this is about, Xen still requires a host (dom0) operating system, usually Linux but also NetBSD and OpenSolaris. Mirage OS (the basis for building your own unikernel) allows building applications as domU kernels. This is advantageous because it minimizes context-switching and allows all of the code in a Xen VM (domU) to be written in a safer language than C. So the dom0 OS (Linux) s…

Even if you have your own hardware there are scenarios where you might want stronger isolation, increased density and/or heterogeneous deployments. All of which is achievable with Unikernels running on Xen.

Re: My First Unikernel

#17

Post by andolanra »

paulasmuth wrote: post #12
wmf wrote: post #11

Earlier quoted context omitted.

You let the host manage memory, disk, network etc and your program is a [process] that doesn't need all the extraneous processes that usually go with a whole OS. No background services or cron jobs. So it's like Docker.

No, this is fundamentally different from docker. Docker/cgroups/namespaces allow you to isolate multiple applications running in userspace on the same kernel to a very high degree. This gets rid of isolation and multiprocessing altogether and runs a single application without any kernel at all (i.e. with just a very limited amount of support code linked in). Put simply; the technology behind docker improves isolation…

I don't see how this, "...does the exact opposite," of what Docker does—both are very different approaches to the similar goals of having individual processes run in isolated ways. Docker does it by isolating individual running Linux processes under a single kernel, whereas unikernel-based systems do it by building those processes as lightweight programs that get compiled to their own kernel images and then running those on a hypervisor to achieve isolation.

Re: My First Unikernel

#18

Post by paulasmuth »

andolanra wrote: post #15
paulasmuth wrote: post #6

This sounds like an awesome and fun project to hack on! However, optimizing around context switches and task preemptions is something you would usually do if your application is actually bound by IO/context switching, is extremely latency sensitive or when you are trying to squeeze the last bits of performance out of a machine. Why did you choose to build such a microoptimized system in a garbage collected language?…

Don't think of it as optimizing around context switches—think of it as just omitting what you don't need, and losing context-switches as a side benefit. A multi-user OS has a lot of stuff which might not be necessary for a single virtualized service (various security mechanisms, lots of file system niceties, running other services, &c), so writing a unikernel like this allows you to select exactly as much as you need…

> so writing a unikernel like this allows you to select exactly as much as you need for your particular service.

I agree this is a big upside of the authors approach. Less dependencies lead to fewer problems caused by external/upstream changes.

> OCaml isn't all that much slower than C++. To use the Programming Language Shootout as a rough esimation[^1], it can even come close to matching C++ in certain programs, and is rarely more than three times as slow.

I wasn't only referring to raw execution performance but also to GC pauses and GC overhead which I think are the bigger issue. The benchmark you linked tests for compute load so this doesn't really show up. Anecdotal point; in most real world apps I have worked on the GC was a limiting factor.

> Finally: the article didn't say, "the Linux kernel"—it said, "Ubuntu." The kernel itself might be secure and reliable, but a running Linux system is much, much more than just the kernel.

That's the beauty of having a kernel though. If one of those userland processes is broken it won't affect the whole system.

> an arbitrary piece of C code might not segfault given certain input, but a given piece of OCaml code definitely won't.

This assumes that the OCaml compiler/interpreter and the hardware are free of bugs...

Re: My First Unikernel

#19

Post by mwcampbell »

amirmc wrote: post #16
enduser wrote: post #5

If it helps to add clarity to what this is about, Xen still requires a host (dom0) operating system, usually Linux but also NetBSD and OpenSolaris. Mirage OS (the basis for building your own unikernel) allows building applications as domU kernels. This is advantageous because it minimizes context-switching and allows all of the code in a Xen VM (domU) to be written in a safer language than C. So the dom0 OS (Linux) s…

Even if you have your own hardware there are scenarios where you might want stronger isolation, increased density and/or heterogeneous deployments. All of which is achievable with Unikernels running on Xen.

How would unikernels running on Xen give you better density than containers (which are really just namespaced processes) running on a shared Unix kernel on bare metal? Wouldn't the Xen-based approach have more overhead?

Re: My First Unikernel

#20

Post by frik »

enduser wrote: post #5

If it helps to add clarity to what this is about, Xen still requires a host (dom0) operating system, usually Linux but also NetBSD and OpenSolaris. Mirage OS (the basis for building your own unikernel) allows building applications as domU kernels. This is advantageous because it minimizes context-switching and allows all of the code in a Xen VM (domU) to be written in a safer language than C. So the dom0 OS (Linux) s…

Even though Amazon EC2 is Xen-based (internally), you have to run a lightweight Linux distro to use Mirage on AWS EC2 (at least for now), as I learned from the MirageOS posting a few weeks ago.
Post reply on HN