Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Case for a High-Level Kernel-Bypass I/O Abstraction

irenezhang.net

Post reply on HN
Print view View original

11–20 of 27 posts

Re: The Case for a High-Level Kernel-Bypass I/O Abstraction

#11

Post by iyzhang »

yourbandsucks wrote: post #10
iyzhang wrote: post #6

Earlier quoted context omitted.

High-level means not exposing hardware limitations to the application. The primary target applications are datacenter services, which spend much of their time processing network I/O. As network latencies lower to a few microseconds, datacenter applications like Redis will need kernel-bypass because the kernel will become too expensive for them. In our experiments with a 25Gb network, the Linux kernel and POSIX interf…

So what's the API look like? You said you don't want to make users deal with flow control and hardware details.. does that imply a userspace bypass library which does that stuff for us? Does it look posixy?

It looks POSIX-like but uses high-level queues and fixes some issues with epoll. The lack of an atomic data unit and the overhead of the poor epoll interface cost too much to retain for kernel-bypass. Take a look at the paper for more details.

Re: The Case for a High-Level Kernel-Bypass I/O Abstraction

#12

Post by gigatexal »

pjmlp wrote: post #5
AstralStorm wrote: post #2

What does it even mean by high level? Once the API is high level enough it gets unusable by major users who are high end networking, GPU and graphics libraries and low latency sound. Nobody else truly needs to bypass the kernel. Even low latency can work with good real time task handling, making the users exactly two cases, who have special DMA handling in hardware already. If it means introducing special case kernel…

Well, that is the whole point of unikernels. Memory safe languages with rich runtimes, only need a mini kernel to run bare metal. Windows has been pushing for user space drivers for a while now, including GPUs. Android is following the same path with Project Treble, and who knows what will happen with Fuchsia. Likewise on many high integrity OSes for embedded deployment.

Does such a runtime and ecosystem exist for go, and if not, (even if so) what other languages?

Re: The Case for a High-Level Kernel-Bypass I/O Abstraction

#13

Post by amluto »

iyzhang wrote: post #11
yourbandsucks wrote: post #10

Earlier quoted context omitted.

So what's the API look like? You said you don't want to make users deal with flow control and hardware details.. does that imply a userspace bypass library which does that stuff for us? Does it look posixy?

It looks POSIX-like but uses high-level queues and fixes some issues with epoll. The lack of an atomic data unit and the overhead of the poor epoll interface cost too much to retain for kernel-bypass. Take a look at the paper for more details.

Where’s the paper? After looking at your site, it’s not obvious to me what paper to look at.

Re: The Case for a High-Level Kernel-Bypass I/O Abstraction

#14

Post by SEJeff »

AstralStorm wrote: post #2

What does it even mean by high level? Once the API is high level enough it gets unusable by major users who are high end networking, GPU and graphics libraries and low latency sound. Nobody else truly needs to bypass the kernel. Even low latency can work with good real time task handling, making the users exactly two cases, who have special DMA handling in hardware already. If it means introducing special case kernel…

You would likely be mistaken. Real time kernels (like Redhat's MRG product, or the -rt patchset from Thomas Gleixner) have less jitter aka more predictable timing. However, in virtually every average case, they have slightly higher latency than the stock linux kernel.

The reality of 100G+ networking or low latency networking is the kernel can't keep up with the interrupts from the hardware, so you turn interrupts per packet off (adaptive coalescing / ethtool -C for ethernet), so userspace tcp/ip stacks such as Intel / Linux Foundation's DPDK[1], Solarflare's OpenOnload[2], Mellanox's VMA[3] and Chelsio's Wire Direct[4] exist to fill this need. Heck, even the BBC wrote their own kernel bypass[5] networking layer! Note that Solarflare, Mellanox, and Chelsio are all heavily used in High Performance Computing supercomputers along with finance such as electronic trading. If there was no need, there wouldn't be so many options due to the market wanting them.

[1] https://www.dpdk.org/

[2] https://www.openonload.org/

[3] http://www.mellanox.com/page/software_vma

[4] https://www.chelsio.com/nic/wire-direct/

[5] https://www.bbc.co.uk/rd/blog/2018-04-high-speed-networking-...

Source: have worked in electronic trading as a Linux engineer for 11-12ish years.

Re: The Case for a High-Level Kernel-Bypass I/O Abstraction

#15

Post by SEJeff »

yourbandsucks wrote: post #10
iyzhang wrote: post #6

Earlier quoted context omitted.

High-level means not exposing hardware limitations to the application. The primary target applications are datacenter services, which spend much of their time processing network I/O. As network latencies lower to a few microseconds, datacenter applications like Redis will need kernel-bypass because the kernel will become too expensive for them. In our experiments with a 25Gb network, the Linux kernel and POSIX interf…

So what's the API look like? You said you don't want to make users deal with flow control and hardware details.. does that imply a userspace bypass library which does that stuff for us? Does it look posixy?

Solarflare's OpenOnload or Mellanox's VMA both show up as LD_PRELOADS that overload any traditional socket programming unless you want to code your apps to their API directly.

Re: The Case for a High-Level Kernel-Bypass I/O Abstraction

#16

Post by pjmlp »

gigatexal wrote: post #12
pjmlp wrote: post #5

Earlier quoted context omitted.

Well, that is the whole point of unikernels. Memory safe languages with rich runtimes, only need a mini kernel to run bare metal. Windows has been pushing for user space drivers for a while now, including GPUs. Android is following the same path with Project Treble, and who knows what will happen with Fuchsia. Likewise on many high integrity OSes for embedded deployment.

Does such a runtime and ecosystem exist for go, and if not, (even if so) what other languages?

Java has baremetal deployments on the embedded space, PTC and Aicas are the two most well known ones.

.NET has netduino and eventually meadow, although it is a subset.

Erlang has GRiSP.

OCaml has MirageOS.

To come back to your question, Go has tinyGo, gVisor, emgo, Biscuit.

And you can have a look at this as well, https://nanovms.com/dev/tutorials/running-go-unikernels

And here http://unikernel.org/projects/

Re: The Case for a High-Level Kernel-Bypass I/O Abstraction

#17

Post by iyzhang »

amluto wrote: post #13
iyzhang wrote: post #11

Earlier quoted context omitted.

It looks POSIX-like but uses high-level queues and fixes some issues with epoll. The lack of an atomic data unit and the overhead of the poor epoll interface cost too much to retain for kernel-bypass. Take a look at the paper for more details.

Where’s the paper? After looking at your site, it’s not obvious to me what paper to look at.

Paper can be found here: http://irenezhang.net/papers/demikernel-hotos19.pdf

Re: The Case for a High-Level Kernel-Bypass I/O Abstraction

#18

Post by SEJeff »

SEJeff wrote: post #14
AstralStorm wrote: post #2

What does it even mean by high level? Once the API is high level enough it gets unusable by major users who are high end networking, GPU and graphics libraries and low latency sound. Nobody else truly needs to bypass the kernel. Even low latency can work with good real time task handling, making the users exactly two cases, who have special DMA handling in hardware already. If it means introducing special case kernel…

You would likely be mistaken. Real time kernels (like Redhat's MRG product, or the -rt patchset from Thomas Gleixner) have less jitter aka more predictable timing. However, in virtually every average case, they have slightly higher latency than the stock linux kernel. The reality of 100G+ networking or low latency networking is the kernel can't keep up with the interrupts from the hardware, so you turn interrupts per…

Here is another good overview of this: https://blog.cloudflare.com/kernel-bypass/

Re: The Case for a High-Level Kernel-Bypass I/O Abstraction

#19

Post by justicezyx »

I am surprised and disappointed that the original paper and the blog post has zero reference to unikernel research, despite the fact that unikernel pretty much is the whole encompassing idea.

I am wondering whether or not this is a missing or a different understanding the concept.

Edit: Sorry I did not really get the difference between library OS and unikernels.

It's still a lack of reference considering their connections.

Re: The Case for a High-Level Kernel-Bypass I/O Abstraction

#20

Post by justicezyx »

iyzhang wrote: post #6
AstralStorm wrote: post #2

What does it even mean by high level? Once the API is high level enough it gets unusable by major users who are high end networking, GPU and graphics libraries and low latency sound. Nobody else truly needs to bypass the kernel. Even low latency can work with good real time task handling, making the users exactly two cases, who have special DMA handling in hardware already. If it means introducing special case kernel…

High-level means not exposing hardware limitations to the application. The primary target applications are datacenter services, which spend much of their time processing network I/O. As network latencies lower to a few microseconds, datacenter applications like Redis will need kernel-bypass because the kernel will become too expensive for them. In our experiments with a 25Gb network, the Linux kernel and POSIX interf…

> High-level means not exposing hardware limitations to the application.

This seems counter-intuitive.

Hardware limitations mean different abstraction than OS-level APIs, as them to applications.

Even POSIX does not expose hardware limitations.

Rather, high-level in the paper is more like some suitable interface to a wide range of applications. I.e., high-level as it's targeted to be used directly by applications as a portable interface.

Post reply on HN