Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

BPF: A New Type of Software

brendangregg.com

Post reply on HN
Print view View original

91–100 of 192 posts

Re: BPF: A New Type of Software

#91

Post by throwaway2048 »

badfrog wrote: post #54
yehia2amer wrote: post #50

This is one of the moments that i read an article and say to myself: “Those stuff needs a smarter programmer than i am” I will try to watch the video, hopefully it is simpler to understand !

I've thought that many times and always been wrong. At the end of the day, code is just code. After you take the time to understand the environment it's running in, it's not that much different than anything else you could write. Creating BPF in the first place took a lot of cleverness. Figuring out how to fit it into a massive infrastructure at a place like Facebook took a lot of cleverness. But most of the people a…

There is lots of code that is difficult to understand, for anyone.

What about code that proves a complex, many page mathematical theorem?

What about code that does complex and very math heavy things like GCM encryption modes, or statistical compression via prediction by partial matching and arithmetic coding?

Just reading the code isn't always enough, plenty of complicated code requires understanding of concepts far outside what you could hope to fit in a comment.

Re: BPF: A New Type of Software

#92

Post by zymhan »

z3t4 wrote: post #20
rusk wrote: post #19

Earlier quoted context omitted.

Real-time, low latency, network-based applications. At the pace of network events, CPU is still very fast by perhaps at least order of magnitude. However, latency introduced by system calls is significant. This allows you to run certain classes of application in kernel space with these overheads largely mitigated. Principally it's monitoring and "observability" applications, but apparently it's much more flexible now…

Wouldn't virtualization kill the perf benefit, or is this supposed to run "on the metal"?

Virtualized OSes can still access network hardware directly via SR-IOV

Re: BPF: A New Type of Software

#93

Post by [deleted] »

manyworlds wrote: post #73
StreamBright wrote: post #68

Earlier quoted context omitted.

I guess you are missing the point by 100000 miles. The whole point of BPF is the avoid writing a C kernel module and pull in the entire problem domain of C programs running in Ring0. this was called out explicitly in the video.

He’s not off the mark completely, you can maintain production safety by using a BPF->kernel module compiler. Unnecessary to have an entire JIT infrastructure in the kernel just to get the safety benefits of BPF.

[deleted]

Re: BPF: A New Type of Software

#94

Post by monocasa »

manyworlds wrote: post #67
rhinoceraptor wrote: post #55

Earlier quoted context omitted.

BPF is completely production safe. So there is no way for a BPF program to crash the kernel, introduce significant performance latency, or have any side effects on the kernel/user space. Obviously, kernel modules have none of those properties. Also, BPF has been around for almost 30 years, and you're likely using it. tcpdump is basically just a BPF bytecode frontend, for example.

You can maintain production safety by using a BPF->kernel module compiler. This additionally removes the need to have the bpf compiler in the kernel, reducing both core size and vulnerability surface area. No reason BPF must imply JIT

The end goal with bpf is to allow arbitrary untrusted programs to load bpf programs. If you were just loading kernel modules you wouldn't be able to maintain kernel integrity and let arbitrary programs load code.

Re: BPF: A New Type of Software

#95

Post by ixtli »

meithecatte wrote: post #90
nikanj wrote: post #86

Earlier quoted context omitted.

Bytecode + compiler was also how both Flash and Java Applets worked. Have we forgotten how secure those were?

It's also how wasm works, and in some sense -- JavaScript. Somehow, these introduce much less security problems.

The 'somehow' seems rather clear to me when you look at how the organizations who governed these languages and how they're run.

Re: BPF: A New Type of Software

#96

Post by cpr »

hackworks wrote: post #79

eBPF can be viewed as a mechanism to safely run user code in kernel since it uses a DSL and a compiler before the byte code is executed in kernel. This opens up doors for running performance critical functionality in kernel without having to bundle it with the kernel or very tightly coupled with the kernel version. Optimizing FUSE is an example: https://extfuse.github.io/ I expect custom security auditing software, r…

> firewalls with rule engines implemented in eBPF in the coming future

Wasn't this the original purpose of BPF?

Re: BPF: A New Type of Software

#97

Post by monocasa »

nikanj wrote: post #86
hackworks wrote: post #79

eBPF can be viewed as a mechanism to safely run user code in kernel since it uses a DSL and a compiler before the byte code is executed in kernel. This opens up doors for running performance critical functionality in kernel without having to bundle it with the kernel or very tightly coupled with the kernel version. Optimizing FUSE is an example: https://extfuse.github.io/ I expect custom security auditing software, r…

Bytecode + compiler was also how both Flash and Java Applets worked. Have we forgotten how secure those were?

Java implemented the sandbox as java code in the same VM as what it was sandboxing, then later attached reflection so you had a billion different ways to get references to those classes blocking your access and do brain surgery on them. Oof.

Flash was a lovecraftian horror novel of a code base that even Adobe didn't feel totally comfortable making changes in.

Re: BPF: A New Type of Software

#98

Post by jancsika »

rusk wrote: post #19
jokoon wrote: post #16

I have a hard time understanding what you would use it for. I could understand a use-case, but I fail to understand why it would be that much useful. I have a sense it allows much better performance for horizontal scaling, but I'm not sure...

Real-time, low latency, network-based applications. At the pace of network events, CPU is still very fast by perhaps at least order of magnitude. However, latency introduced by system calls is significant. This allows you to run certain classes of application in kernel space with these overheads largely mitigated. Principally it's monitoring and "observability" applications, but apparently it's much more flexible now…

To get the realtime benefits I'd have to run everything that requires realtime scheduling as an eBPF program, right?

I mean, suppose I want to make an eBPF program to pipe audio input into my very fancy DSP algo. Awesome. But now I've limited the remainder of my signal chain to remain in eBPF land, haven't I? The moment I throw the signal to Supercollider, Pd, some Jack client, etc., I'm back to being a mere userland mortal and taking the performance hit that entails. (Unless I sample that data coming into userland at a lower rate, but that severely limits the use cases for such a design.)

Judging by some of the screenshots I've seen Linux audio users seem to want to mix their samples through every single audio program they can get their hands on. So at least in this domain I have a hard time seeing how eBPF could practically help realtime scheduling.

Edit: clarification

Re: BPF: A New Type of Software

#99

Post by rusk »

jancsika wrote: post #98
rusk wrote: post #19

Earlier quoted context omitted.

Real-time, low latency, network-based applications. At the pace of network events, CPU is still very fast by perhaps at least order of magnitude. However, latency introduced by system calls is significant. This allows you to run certain classes of application in kernel space with these overheads largely mitigated. Principally it's monitoring and "observability" applications, but apparently it's much more flexible now…

To get the realtime benefits I'd have to run everything that requires realtime scheduling as an eBPF program, right? I mean, suppose I want to make an eBPF program to pipe audio input into my very fancy DSP algo. Awesome. But now I've limited the remainder of my signal chain to remain in eBPF land, haven't I? The moment I throw the signal to Supercollider, Pd, some Jack client, etc., I'm back to being a mere userland…

Hm, good point. But couldn't you say the same thing about CUDA? There is an overhead involved in shunting data to and from the cores ... but the nature of the work being done, and the massive parallelism, mean the overall end-to-end volume of throughput increases dramatically.

I don't know enough about audio processing to answer your specific question ... but processing network traffic for instance, you have the opportunity to "do things" with traffic. For instance partially decode PDUs to see if they interest you, and then pass them on to user-land for more in-depth processing if so ...

EDIT I think you're thinking in terms exclusively of `mapping` operations, but this kind of thing can be great for `reducing` or filtering.

Re: BPF: A New Type of Software

#100

Post by dbancajas »

rusk wrote: post #19
jokoon wrote: post #16

I have a hard time understanding what you would use it for. I could understand a use-case, but I fail to understand why it would be that much useful. I have a sense it allows much better performance for horizontal scaling, but I'm not sure...

Real-time, low latency, network-based applications. At the pace of network events, CPU is still very fast by perhaps at least order of magnitude. However, latency introduced by system calls is significant. This allows you to run certain classes of application in kernel space with these overheads largely mitigated. Principally it's monitoring and "observability" applications, but apparently it's much more flexible now…

Could you allocate a memory dynamically inside your BPF program?
Post reply on HN