Earlier quoted context omitted.
Unikernels could also be the next phase of cloud orchestration technologies. They would be truly lightweight VMs, and obviate the need for containers (collapse the VM orchestration and container orchestration layers into a single orchestration layer). This would offer better security and better performance without impacting developer/operator friendliness. And a side note about improved performance, in the cloud spac…
Debugging a single-executable container is quite unpleasant. I assume debugging unikernels or thin VMs (with a kernel and a single program) would be a similar experience.
Unikernels: The Next Stage of Linux’s Dominance (2019)
141–150 of 187 posts
Re: Unikernels: The Next Stage of Linux’s Dominance (2019)
#142> Unikernels have demonstrated enormous advantages over Linux in many important domains By domains, do they mean that as "actually in use in certain sectors in the industry" or "a prototype has shown that"? > causing some to propose that the days of Linux's dominance may be coming to an end Who exactly would make that claim? So, besides performance : what actual, real-life problems does this solve? I think there is s…
> So, besides performance: what actual, real-life problems does this solve? Well, don't go jumping to "besides performance". Dennard scaling is dead, and the things that can interrupt your program in a full preemptive kernel system are myriad. Furthermore, even if your application properly manages and produces backpressure, the OS can introduce buffering where you don't want it, often by necessity (to avoid massive c…
Re: Unikernels: The Next Stage of Linux’s Dominance (2019)
#143Earlier quoted context omitted.
Debugging a single-executable container is quite unpleasant. I assume debugging unikernels or thin VMs (with a kernel and a single program) would be a similar experience.
Could you not do all your debugging and development on a normal host and only deploy on a unikernel when you are stable? There is already a divide between development and production with debug and release builds the later of which are a massive pain to debug.
Sure! And, still, when you deploy, it fails. At least one of your assumptions that led to your conclusion that your development/testing environment is equivalent to the production one are flawed. Find which ones.
Re: Unikernels: The Next Stage of Linux’s Dominance (2019)
#144Earlier quoted context omitted.
> So, besides performance: what actual, real-life problems does this solve? Well, don't go jumping to "besides performance". Dennard scaling is dead, and the things that can interrupt your program in a full preemptive kernel system are myriad. Furthermore, even if your application properly manages and produces backpressure, the OS can introduce buffering where you don't want it, often by necessity (to avoid massive c…
It’s much easier to tweak a normal kernel (or just use whatever is already there, eg real-time scheduling to get rid of preemption) than move to a unikernel.
The other thing that people will look at is doing something like alpine && a heavy-handed seccomp/apparmor but then you're not really doing any cuts at all.
Re: Unikernels: The Next Stage of Linux’s Dominance (2019)
#145When I make a system where a unikernel could do the job, I usually want a full OS for program setup and initialization, and then an isolated core for the main loop, sharing (single-writer) memory pages with other, less performance-critical processes for logging, stats reporting, and any needed file system activities. The makers of top-performing NICs have been quite good at providing direct user-space access to their…
I don't think the tuning you refer to is sufficient to bring the platform noise to levels required for some workloads. You will still see a lot of system call interrupts, TLB shootdowns, timer events, etc. Funnily enough yesterday I published an article tangential to this problem field. http://bitcharmer.blogspot.com/2020/05/t_84.html I'm not an expert on unikernels but my assumption is that you will see none of that…
Re: Unikernels: The Next Stage of Linux’s Dominance (2019)
#146Earlier quoted context omitted.
I don't think the tuning you refer to is sufficient to bring the platform noise to levels required for some workloads. You will still see a lot of system call interrupts, TLB shootdowns, timer events, etc. Funnily enough yesterday I published an article tangential to this problem field. http://bitcharmer.blogspot.com/2020/05/t_84.html I'm not an expert on unikernels but my assumption is that you will see none of that…
Good writeup, but is there any mitigation for TLS shootdowns?
You still have cores to isolate, busybody kernel threads to suppress, and hardware interrupts to direct elsewhere, but TLB shootdown paranoia is largely a product of the current fashion favoring multi-threading over running separate processes with carefully chosen sharing.
Re: Unikernels: The Next Stage of Linux’s Dominance (2019)
#147Full article text: http://sci-hub.tw/10.1145/3317550.3321445 Maybe link the top-level post there?
There is a link to the PDF on the page. No ACM membership required. https://dl.acm.org/doi/pdf/10.1145/3317550.3321445
Re: Unikernels: The Next Stage of Linux’s Dominance (2019)
#148Earlier quoted context omitted.
As for contributing to existing projects, I hear you — I've been highly active in Open Source for well over a decade. Conceptually, the closest thing to what I want to do is Pure Data — and I have in fact contributed to it. But I'm also extremely motivated and willing to go all the way down and write the entire thing from scratch if I have to. Or to learn enough about every last step in the chain that I can actually…
The word "latency" covers many things, many subtly different from each other. One of its meanings, unrelated to the one you're talking about, is the delay in signal flow caused by algorithms (many digital filters, for example). Often called "plugin delay compensation", or more generically "latency compensation". Let me just point out that it has taken 20 years and a guy whose PhD thesis was about latency compensation…
My perception is that I am not capable of reliably tuning a general purpose OS for a total system latency of under 10 msec. I can't give the total system a hard number and believe that it will obey; instead, I need to perform a lot of esoteric tweakery of subsystems I probably don't understand. The system won't alert me reliably when it fails, but will instead either drop out or just give me more latency than I asked for — and there are innumerable factors outside my control that could cause it to fail.
However, the composition tool I want to create has a pretty small set of requirements, if I accept that it only need serve my particular use case. So how about I ensure that my app is the only thing running on the hardware, via unikernel, or RTOS, or even bare metal?
Implementing all of my compositional requirements is probably easier and certainly more rewarding than tweaking latency parameters without having confidence that my results will be enduring or predictable.
If it turns out that the knowledge I gain during that exercise allows me to control latency well enough and I can return to mainline operating systems and contribute to existing projects, all the better. I don't really want to go down this path, but I'm not willing to accept a tool that maybe-kinda-sorta-sometimes meets my absolute requirements.
Re: Unikernels: The Next Stage of Linux’s Dominance (2019)
#149Earlier quoted context omitted.
I write realtime audio software for Linux, and have done so for 20 years. While your goal is admirable - it certainly is possible to come up with scenarios where "sub-ms" latency is desirable - it's really not relevant. Your stated goal ("...lots of DSP ... under 10 msec") is already entirely achievable on Linux, assuming you're close enough to your speakers (or wearing headphones). But sub-msec can only make sense h…
As for contributing to existing projects, I hear you — I've been highly active in Open Source for well over a decade. Conceptually, the closest thing to what I want to do is Pure Data — and I have in fact contributed to it. But I'm also extremely motivated and willing to go all the way down and write the entire thing from scratch if I have to. Or to learn enough about every last step in the chain that I can actually…
https://tel.archives-ouvertes.fr/tel-01947309/document ;
also see the intro of section 10.4 with a couple relevant references. Of course that only benchmarks the part that you as a developer can have a meaningful impact on, not the time spent in JACK / ALSA / ...
To give an anecdote, with the hardware mentioned at the beginning of said chapter I can reliably put JACK in a 16 samples buffer at 44100 (~0.7 ms) and do a few things without crackles. not a lot though :p
Re: Unikernels: The Next Stage of Linux’s Dominance (2019)
#150Earlier quoted context omitted.
It’s much easier to tweak a normal kernel (or just use whatever is already there, eg real-time scheduling to get rid of preemption) than move to a unikernel.
I kinda disagree. We looked at doing this and just couldn't fathom how it could be done. You might be surprised at the amount of code and entanglement there is to support multiple users, multiple processes - that touches everything from IPC/shared memory to address space to permissions. Very large deep cuts would have to be made and then you'd need to ensure your patchset stays congruent with upstream. The other thin…