Unikernels: The Next Stage of Linux’s Dominance (2019) • Hacker News Board

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#111

Post by rectang » Sat, May 16, 2020, 2:38 PM UTC

EnigmaCurry wrote: post #102

rectang wrote: post #95

Can a unikernel be used to create a sound/music application which is not afflicted by the latency problems which bedevil such applications when running inside traditional operating systems? For example, could it be used for a softsynth which takes in midi input and emits digital audio with the OS only contributing guaranteed sub-millisecond latency?

is that not what the linux realtime patches do? https://wiki.archlinux.org/index.php/Realtime_kernel_patchse...

I've been researching this for a long time, and it's unclear to me whether Linux with PREEMPT_RT patches would meet that sub-ms requirement. I see numbers all over the place from various sources, from sub-millisecond to more than 10 ms.

10 milliseconds is often considered the threshold of human perception with regards to audio latency. (It's possible that it's lower under some circumstances, but 10 ms is a good enough approximation for my purposes.) My goal is to create a sound/music tool which runs lots of DSP and which has a total system latency of under 10 milliseconds, including OS, application (including intrinsic latency of DSP procedures), hardware, and sound in air (about 1 ms per 3 meters).

When I read about latency, I often see "we're at 6-9 ms, that's good enough because it's not perceptible". Unfortunately, that's not good enough if there are several components which contribute to total system latency and they are all pushing 10 ms. Hence, my sub-ms requirement for the OS.

Committing to a platform will be a costly choice. I don't want to invest in writing for realtime Linux only to find that I really need to run a hardcore RTOS, or run dedicated DSP chips, etc.

That's why I'm interested in whether running a unikernel can offer stronger guarantees.

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#112

Post by PaulDavisThe1st » Sat, May 16, 2020, 2:46 PM UTC

rectang wrote: post #111

EnigmaCurry wrote: post #102

Earlier quoted context omitted.

is that not what the linux realtime patches do? https://wiki.archlinux.org/index.php/Realtime_kernel_patchse...

I've been researching this for a long time, and it's unclear to me whether Linux with PREEMPT_RT patches would meet that sub-ms requirement. I see numbers all over the place from various sources, from sub-millisecond to more than 10 ms. 10 milliseconds is often considered the threshold of human perception with regards to audio latency. (It's possible that it's lower under some circumstances, but 10 ms is a good enoug…

I write realtime audio software for Linux, and have done so for 20 years.

While your goal is admirable - it certainly is possible to come up with scenarios where "sub-ms" latency is desirable - it's really not relevant.

Your stated goal ("...lots of DSP ... under 10 msec") is already entirely achievable on Linux, assuming you're close enough to your speakers (or wearing headphones).

But sub-msec can only make sense here if it describes scheduler latency, since there's no audio hardware that can function in the sub-msec range. The linux scheduler is way, way below that threshold and SCHED_FIFO threads will see that performance barring hardware issues (c.f. https://manual.ardour.org/setting-up-your-system/the-right-c...)

Finally .... writing audio software is a lot of fun. But please don't just jump in without seeing if you can instead contribute to an existing project first. The field is littered with the dead and half-dead corpses representing the discarded work of developers who thought it would be fun, and then moved on.

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#113

Post by gandalfgeek » Sat, May 16, 2020, 3:04 PM UTC

If you're too lazy to read the whole paper I've made a quick explainer video:

https://youtu.be/3NWUgBsEXiU

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#114

Post by eyberg » Sat, May 16, 2020, 3:37 PM UTC

neilwilson wrote: post #56

Seems to me that a unikernel database should be the first application. Databases tend to bypass practically all the facilities of a kernel anyway. It’s often surprised me that they haven’t merged before now.

There is plenty of work in this area. The team behind OSv ending up writing https://www.scylladb.com/ because of their earlier interest in databases.

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#115

Post by eyberg » Sat, May 16, 2020, 3:38 PM UTC

MrBuddyCasino wrote: post #54

> Unikernels have demonstrated enormous advantages over Linux in many important domains By domains, do they mean that as "actually in use in certain sectors in the industry" or "a prototype has shown that"? > causing some to propose that the days of Linux's dominance may be coming to an end Who exactly would make that claim? So, besides performance : what actual, real-life problems does this solve? I think there is s…

Security (general lack of user-land and in-ability to run more than one program), server density (run thousands of vms/box).

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#116

Post by eyberg » Sat, May 16, 2020, 3:40 PM UTC

MichaelMoser123 wrote: post #97

rwmj wrote: post #2

I'm one of the authors on this paper, so ask away if you have questions.

isn't it hard to develop/debug/monitor an application with this approach? I mean you can't just run a debugger as another process and attach to the thing, you can't run tcpdump or strace, nothing of that sort. Also every bad pointer access will require a reboot, wouldn't it? I mean how do you develop an application with this approach?

One of our engineers spent yesterday debugging a networking issue on Google Cloud for Nanos.

Not saying it is easy but at the end of the day it's just engineering work.

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#117

Post by eyberg » Sat, May 16, 2020, 3:42 PM UTC

MaxBarraclough wrote: post #57

rwmj wrote: post #2

I'm one of the authors on this paper, so ask away if you have questions.

What do you make of the highly critical Assessing Unikernel Security paper [0]? Have things changed since its publication? [0] https://www.nccgroup.trust/globalassets/our-research/us/whit... , from https://news.ycombinator.com/item?id=19738905

You might enjoy reading my response to that:

https://nanovms.com/dev/tutorials/assessing-unikernel-securi...

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#118

Post by eyberg » Sat, May 16, 2020, 3:45 PM UTC

axegon_ wrote: post #27

Full disclosure: I have only briefly fiddled with unikernels. And in all honesty, I think there could be a good real life application for them. But the biggest problems I see is that even though they have been around for a while, building them is still incredibly complex and time consuming and there isn't a core community behind them and looking at it, it seems like they are starting to suffer from the javascript syn…

Well one of the problems here is that it's kernel work and not everyone has the skillset to do that work, worst no one wants to fund it.

However, the tooling is changing.

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#119

Post by eyberg » Sat, May 16, 2020, 3:51 PM UTC

wltprgm wrote: post #70

rwmj wrote: post #2

I'm one of the authors on this paper, so ask away if you have questions.

Correct me if I am wrong, Unikernels are basically allowing binary to run at Ring 0? What do you think about WASM in the kernel with the same high level concept with Unikernel? link in the below https://destroyallsoftware.com/talks/the-birth-and-death-of-...

I'll admit I don't know much about WASM but access to raw sockets and 32bit only sounds extremely limiting.

Until those are dealt with wasm will never move past it's current status quo - malware in the browser.

Re: Unikernels: The Next Stage of Linux’s Dominance (2019)

#120

Post by rectang » Sat, May 16, 2020, 4:00 PM UTC

PaulDavisThe1st wrote: post #112

rectang wrote: post #111

Earlier quoted context omitted.

I've been researching this for a long time, and it's unclear to me whether Linux with PREEMPT_RT patches would meet that sub-ms requirement. I see numbers all over the place from various sources, from sub-millisecond to more than 10 ms. 10 milliseconds is often considered the threshold of human perception with regards to audio latency. (It's possible that it's lower under some circumstances, but 10 ms is a good enoug…

I write realtime audio software for Linux, and have done so for 20 years. While your goal is admirable - it certainly is possible to come up with scenarios where "sub-ms" latency is desirable - it's really not relevant. Your stated goal ("...lots of DSP ... under 10 msec") is already entirely achievable on Linux, assuming you're close enough to your speakers (or wearing headphones). But sub-msec can only make sense h…

As for contributing to existing projects, I hear you — I've been highly active in Open Source for well over a decade. Conceptually, the closest thing to what I want to do is Pure Data — and I have in fact contributed to it.

But I'm also extremely motivated and willing to go all the way down and write the entire thing from scratch if I have to. Or to learn enough about every last step in the chain that I can actually control for latency — and it may be that it takes about the same amount of work. Finding numbers I can trust on latency seems hopeless. Everybody fudges rather than fails.