Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unikernels: The Next Stage of Linux’s Dominance (2019)

dl.acm.org

Post reply on HN
Print view View original

121–130 of 187 posts

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#121

Post by bonzini »

rwmj wrote: post #37
pinopinopino wrote: post #36

Earlier quoted context omitted.

Really cool work! Is there really a support layer for Emacs in the kernel? Now the real question. You are statically linking against a modified glibc, so they make calls to your kernel library instead to the system call and you can do some nice optimizations along the way. Which I think make sense. I can see this is cool for things that directly interface to glibc, but do you think this is viable for applications usi…

At the moment you must go through glibc. For example issuing SYSENTER or INT 80h will not work. We have only run C applications so far, so I'm unsure how it will all work for servers written in other programming languages. I think applications which rely on complex multi-process middleware (like Java JBoss, etc) will be hard or even impossible to port.

You can always "unikernelize" the JVM. OSv had a kernel-optimized JVM for example.

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#122

Post by throwaway894345 »

MrBuddyCasino wrote: post #79
microcolonel wrote: post #67

Earlier quoted context omitted.

> So, besides performance: what actual, real-life problems does this solve? Well, don't go jumping to "besides performance". Dennard scaling is dead, and the things that can interrupt your program in a full preemptive kernel system are myriad. Furthermore, even if your application properly manages and produces backpressure, the OS can introduce buffering where you don't want it, often by necessity (to avoid massive c…

So, a niche technology for a handful of HFT firms, betting platforms and hyper-scale cloud vendors. Those already use non-mainstream tech (eg LMAX disruptor) to achieve max performance. This is all nice and good, but the Unikernel guys claim since 20 years its the next big revolutionary thing. In the real world, the evolutionary approach using containers has turned out to solve the problems most people care about. Do…

Unikernels could also be the next phase of cloud orchestration technologies. They would be truly lightweight VMs, and obviate the need for containers (collapse the VM orchestration and container orchestration layers into a single orchestration layer). This would offer better security and better performance without impacting developer/operator friendliness. And a side note about improved performance, in the cloud space, being able to boot in ten ms means you can start a VM in the scope of a single request, which means you can scale up dynamically without keeping a bunch of VMs running idle in the background just in case your traffic spikes. This is real, significant cost savings for cloud providers and customers.

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#123

Post by freedomben »

rwmj wrote: post #39
dathinab wrote: post #38

Earlier quoted context omitted.

But what about the cloud and auto scaling wouldn't that count as distribution (through for internal use only)?

It's an interesting point. I wonder if sending a binary to AWS counts as distribution to a third party? (Of course I work for Red Hat and the cult thinks all code should be free :-)

Yes all code should be free!

(I work for Red Hat also :-)

I was originally just being silly but figured I would add something of substance here. I think the GPL goes a little too far but I like the spirit of it. As a user if I pay for an application, I should get a copy of the code for personal use as well. I don't think that gives me the right to distribute it because it's still somebody's property, but applications should be distributed to customers with their source IMHO.

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#124

Post by bonzini »

rwmj wrote: post #32
perbu wrote: post #22

Earlier quoted context omitted.

Basically you replace the system calls into the kernel with library calls. So you end up with a single binary that contains both the application as well as the operating system functionality. This can lead to some rather spectacular performance benefits. One thing is that you remove the syscall overhead, but more importantly you can do a lot more optimizations at the compiler/linking steps. You can remove complex cod…

Another performance benefit (unsurprising in highsight) comes from removing KPTI and similar Spectre mitigations. These have really made system calls in regular kernels expensive.

On the other hand if you run only one application on your VM you can just disable them, because one you have access to the userspace application there is nothing else on the machine to steal via side channel attacks.

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#125

Post by throwaway894345 »

MichaelMoser123 wrote: post #97
rwmj wrote: post #2

I'm one of the authors on this paper, so ask away if you have questions.

isn't it hard to develop/debug/monitor an application with this approach? I mean you can't just run a debugger as another process and attach to the thing, you can't run tcpdump or strace, nothing of that sort. Also every bad pointer access will require a reboot, wouldn't it? I mean how do you develop an application with this approach?

Probably best to use a memory safe language so you don’t have bad pointer accesses and generally minimize debugging significantly.

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#126

Post by uep »

PaulDavisThe1st wrote: post #112
rectang wrote: post #111

Earlier quoted context omitted.

I've been researching this for a long time, and it's unclear to me whether Linux with PREEMPT_RT patches would meet that sub-ms requirement. I see numbers all over the place from various sources, from sub-millisecond to more than 10 ms. 10 milliseconds is often considered the threshold of human perception with regards to audio latency. (It's possible that it's lower under some circumstances, but 10 ms is a good enoug…

I write realtime audio software for Linux, and have done so for 20 years. While your goal is admirable - it certainly is possible to come up with scenarios where "sub-ms" latency is desirable - it's really not relevant. Your stated goal ("...lots of DSP ... under 10 msec") is already entirely achievable on Linux, assuming you're close enough to your speakers (or wearing headphones). But sub-msec can only make sense h…

I'm curious what you think of PipeWire:

https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Per...

It claims to get near Jack latency, while implementing a more generic audio graph with better security than PulseAudio.

In regards to Linux scheduler performance, do you have any experience with SCHED_DEADLINE?

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#127

Post by rbanffy »

throwaway894345 wrote: post #122
MrBuddyCasino wrote: post #79

Earlier quoted context omitted.

So, a niche technology for a handful of HFT firms, betting platforms and hyper-scale cloud vendors. Those already use non-mainstream tech (eg LMAX disruptor) to achieve max performance. This is all nice and good, but the Unikernel guys claim since 20 years its the next big revolutionary thing. In the real world, the evolutionary approach using containers has turned out to solve the problems most people care about. Do…

Unikernels could also be the next phase of cloud orchestration technologies. They would be truly lightweight VMs, and obviate the need for containers (collapse the VM orchestration and container orchestration layers into a single orchestration layer). This would offer better security and better performance without impacting developer/operator friendliness. And a side note about improved performance, in the cloud spac…

Debugging a single-executable container is quite unpleasant. I assume debugging unikernels or thin VMs (with a kernel and a single program) would be a similar experience.

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#128

Post by PaulDavisThe1st »

uep wrote: post #126
PaulDavisThe1st wrote: post #112

Earlier quoted context omitted.

I write realtime audio software for Linux, and have done so for 20 years. While your goal is admirable - it certainly is possible to come up with scenarios where "sub-ms" latency is desirable - it's really not relevant. Your stated goal ("...lots of DSP ... under 10 msec") is already entirely achievable on Linux, assuming you're close enough to your speakers (or wearing headphones). But sub-msec can only make sense h…

I'm curious what you think of PipeWire: https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Per... It claims to get near Jack latency, while implementing a more generic audio graph with better security than PulseAudio. In regards to Linux scheduler performance, do you have any experience with SCHED_DEADLINE?

We don't use SCHED_DEADLINE with audio, because things are driven by hardware interrupts/requirements, which makes SCHED_FIFO more appropriate ("wakeup! time do the work! right now! until it's done!"). macOS doesn't really have SCHED_FIFO, so we end up forced to use something to deadline scheduling there. It works fine.

I commented a lot on PipeWire when it started. Thankfully Robin Gareus and others managed to get its developer to change course somewhere along the journey to use a better basic model (notably, pull not push). I no longer follow its development. If it works, it will be good for Linux audio. I don't know if it will, or won't.

[for anyone who doesn't know, I'm the original author of JACK]

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#129

Post by PaulDavisThe1st »

rectang wrote: post #120
PaulDavisThe1st wrote: post #112

Earlier quoted context omitted.

I write realtime audio software for Linux, and have done so for 20 years. While your goal is admirable - it certainly is possible to come up with scenarios where "sub-ms" latency is desirable - it's really not relevant. Your stated goal ("...lots of DSP ... under 10 msec") is already entirely achievable on Linux, assuming you're close enough to your speakers (or wearing headphones). But sub-msec can only make sense h…

As for contributing to existing projects, I hear you — I've been highly active in Open Source for well over a decade. Conceptually, the closest thing to what I want to do is Pure Data — and I have in fact contributed to it. But I'm also extremely motivated and willing to go all the way down and write the entire thing from scratch if I have to. Or to learn enough about every last step in the chain that I can actually…

The word "latency" covers many things, many subtly different from each other. One of its meanings, unrelated to the one you're talking about, is the delay in signal flow caused by algorithms (many digital filters, for example). Often called "plugin delay compensation", or more generically "latency compensation".

Let me just point out that it has taken 20 years and a guy whose PhD thesis was about latency compensation in a DAW to finally "fix" this sort of "latency" in Ardour. This is a massively harder problem from a design perspective than the scheduling latency issues you've referred to.

[ EDIT: to be fair, the actual correct solution to latency compensation didn't take 20 years to implement, more like a year or so when taking place within the context of a large existing code base. ]

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#130

Post by zarkov99 »

rbanffy wrote: post #127
throwaway894345 wrote: post #122

Earlier quoted context omitted.

Unikernels could also be the next phase of cloud orchestration technologies. They would be truly lightweight VMs, and obviate the need for containers (collapse the VM orchestration and container orchestration layers into a single orchestration layer). This would offer better security and better performance without impacting developer/operator friendliness. And a side note about improved performance, in the cloud spac…

Debugging a single-executable container is quite unpleasant. I assume debugging unikernels or thin VMs (with a kernel and a single program) would be a similar experience.

Could you not do all your debugging and development on a normal host and only deploy on a unikernel when you are stable? There is already a divide between development and production with debug and release builds the later of which are a massive pain to debug.
Post reply on HN