Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

BPF: A New Type of Software

brendangregg.com

Post reply on HN
Print view View original

31–40 of 192 posts

Re: BPF: A New Type of Software

#31

Post by kbumsik »

andbberger wrote: post #30
kbumsik wrote: post #22

Earlier quoted context omitted.

> "if this exceeds some limits or bogs down system for more than X milliseconds, disable and give error". AFAIK indefinite loops in a BPF are not allowed: The kernel eBPF verifier will reject to load such programs. So the execution time of BPF programs will not be variable time and will be predictive. I'm not sure if users need to care about the cases of long execution times when no loops are allowed.

How is that possible?? If true, that would imply BPF programs are not Turing complete.

You are right, eBPF is non-Turing complete. To be precise I heard that the kernel eBPF verifier ensures the flow of a BPF program is a kind of a DAG (no cycles = no loops). So eBPF VM is definitely not a general-purpose machine.

Re: BPF: A New Type of Software

#32

Post by detaro »

andbberger wrote: post #30
kbumsik wrote: post #22

Earlier quoted context omitted.

> "if this exceeds some limits or bogs down system for more than X milliseconds, disable and give error". AFAIK indefinite loops in a BPF are not allowed: The kernel eBPF verifier will reject to load such programs. So the execution time of BPF programs will not be variable time and will be predictive. I'm not sure if users need to care about the cases of long execution times when no loops are allowed.

How is that possible?? If true, that would imply BPF programs are not Turing complete.

That was specifically a design goal to make sure their runtime is bounded, and most tasks, especially in that problem space, can be done without being turing-complete.

Re: BPF: A New Type of Software

#33

Post by garblegarble »

andbberger wrote: post #30
kbumsik wrote: post #22

Earlier quoted context omitted.

> "if this exceeds some limits or bogs down system for more than X milliseconds, disable and give error". AFAIK indefinite loops in a BPF are not allowed: The kernel eBPF verifier will reject to load such programs. So the execution time of BPF programs will not be variable time and will be predictive. I'm not sure if users need to care about the cases of long execution times when no loops are allowed.

How is that possible?? If true, that would imply BPF programs are not Turing complete.

>If true, that would imply BPF programs are not Turing complete.

It's not intended to be turing complete, just to be useful enough while provably not harming the system (e.g. by never terminating). There's quite a good write-up at https://lwn.net/Articles/773605/

Re: BPF: A New Type of Software

#34

Post by maxdamantus »

joosters wrote: post #26
maxdamantus wrote: post #13

Earlier quoted context omitted.

I think it's more to do with avoiding overheads typically associated with system calls (presumably involving some interrupt and disabling/enabling/changing paging behaviour). Here's an example of a syscall-heavy command on my system: $ time dd if=/dev/zero bs=1 count=10M of=/dev/null 10485760+0 records in 10485760+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 7.09089 s, 1.5 MB/s real 0m7.092s user 0m2.123s sys…

How much of that time is really spent in the system call interface? You've got 4.968s of system time there (i.e. broadly the time spent in kernel code) and 2.123s of user time. Given that the user-space program is effectively a tight loop around read() and write() calls, we can assume that almost all of those 2 seconds are spent going through the syscall plumbing. Now, there's going to also be some of the kernel-side…

There shouldn't really be any significant filesystem code involved, since once `dd` has opened the files, it should have handlers for those devices more-or-less directly in its descriptor table. Once you have a descriptor to a pipe or device, there shouldn't be any filesystem-level checking in the middle of your reads/writes; all you're doing is filling/emptying buffers.

And given that I can write a program that makes 132 million calls per second to the glibc `putchar` function (which also buffers), I'm pretty sure there's a lot of time that can be shaved off as we start to replace the system call mechanism with plain function calls.

Re: BPF: A New Type of Software

#35

Post by phkamp »

The Lost Generation discovers IBM Mainframe Channel Programs?

Want to bet if they are going to make all the same mistakes themselves, or if they are willing to learn from history?

Re: BPF: A New Type of Software

#36

Post by barrkel »

maxdamantus wrote: post #13
Iv wrote: post #6

So from my understanding, that's a kind of "secure" (I'd like to know more about the security model tbh) module that runs with kernel privilege with no scheduling (so it runs until completion). These are supposed to be short and I am assuming, can't call libs and can't allocate memory (outside a predefined stack I would guess?) Aren't they very similar to interrupts? What is the difference there? The kernel API?

I think it's more to do with avoiding overheads typically associated with system calls (presumably involving some interrupt and disabling/enabling/changing paging behaviour). Here's an example of a syscall-heavy command on my system: $ time dd if=/dev/zero bs=1 count=10M of=/dev/null 10485760+0 records in 10485760+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 7.09089 s, 1.5 MB/s real 0m7.092s user 0m2.123s sys…

Have you forgotten about Meltdown, Spectre, and all the other cache attacks?

Re: BPF: A New Type of Software

#37

Post by tsukurimashou »

ComSubVie wrote: post #18

Haven't used it till now (except maybe via nft?). What I'm not sure is: who is preventing BPF to be used as rootkits? Since they are run inside the kernel and cannot be inspected (?) can they be used to hide malicious activity?

I was wondering about that as well

Re: BPF: A New Type of Software

#38

Post by gjulianm »

z3t4 wrote: post #20
rusk wrote: post #19

Earlier quoted context omitted.

Real-time, low latency, network-based applications. At the pace of network events, CPU is still very fast by perhaps at least order of magnitude. However, latency introduced by system calls is significant. This allows you to run certain classes of application in kernel space with these overheads largely mitigated. Principally it's monitoring and "observability" applications, but apparently it's much more flexible now…

Wouldn't virtualization kill the perf benefit, or is this supposed to run "on the metal"?

I don't know about other devices, but network cards have pretty good support for virtualization. A lot of them have virtual devices that the card generates and are added to the system as if they were different PCI devices. Then, each VM gets one of those virtual devices and they access it directly as a regular PCI device, with no virtualization layer whatsoever.

Re: BPF: A New Type of Software

#39

Post by de_watcher »

barrkel wrote: post #36
maxdamantus wrote: post #13

Earlier quoted context omitted.

I think it's more to do with avoiding overheads typically associated with system calls (presumably involving some interrupt and disabling/enabling/changing paging behaviour). Here's an example of a syscall-heavy command on my system: $ time dd if=/dev/zero bs=1 count=10M of=/dev/null 10485760+0 records in 10485760+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 7.09089 s, 1.5 MB/s real 0m7.092s user 0m2.123s sys…

Have you forgotten about Meltdown, Spectre, and all the other cache attacks?

It's one of the two phases. We're back to the other one, wait for a couple of months.

Re: BPF: A New Type of Software

#40

Post by maxdamantus »

barrkel wrote: post #36
maxdamantus wrote: post #13

Earlier quoted context omitted.

I think it's more to do with avoiding overheads typically associated with system calls (presumably involving some interrupt and disabling/enabling/changing paging behaviour). Here's an example of a syscall-heavy command on my system: $ time dd if=/dev/zero bs=1 count=10M of=/dev/null 10485760+0 records in 10485760+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 7.09089 s, 1.5 MB/s real 0m7.092s user 0m2.123s sys…

Have you forgotten about Meltdown, Spectre, and all the other cache attacks?

These are things that kernel developers are surely mindful of when coming up with and implementing eBPF functionality.

Regardless, I'm sure I've run this same test years ago and seen the system call count still in the same order (that is, a couple of million per second). I really doubt Spectre mitigations are what are causing what should be a few dereferences and function calls to take around a thousand clock cycles.

Post reply on HN