Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

BPF: A New Type of Software

brendangregg.com

Post reply on HN
Print view View original

141–150 of 192 posts

Re: BPF: A New Type of Software

#141

Post by eyberg »

SahAssar wrote: post #126
kizer wrote: post #112

I've wondered why operating systems, aside from hypervisors, are overwhelmingly the first abstraction - I know the obvious benefits, or rather necessities (processor sharing, security, file system, etc etc) - but in ultra-specialized perf-critical applications I'd have thought economic pressures would have materialized a greater variety of ad-hoc bare-metal software. I guess we're headed that way w/ the twilight of M…

Isn't that sorta what a unikernel is?

yep - and some of us are working on that

Re: BPF: A New Type of Software

#142

Post by mkup »

boomlinde wrote: post #88
nicoburns wrote: post #78

Earlier quoted context omitted.

> Running untrusted unbounded code in the kernel is not smart. Well currently we run code (e.g. drivers) as trusted full-permission code. Surely, web-assembly would be better than this from a security perspective.

What use case do you envision?

Out-of-tree drivers, support for closed-source drivers in Linux (assuming kernel API for wasm drivers is stable).

Re: BPF: A New Type of Software

#143

Post by PaulDavisThe1st »

smorrow wrote: post #139
willisbueller wrote: post #135

Sounds a lot like SPIN OS https://en.wikipedia.org/wiki/SPIN_(operating_system) They have to make do without type safety (in SPIN's case provided by modula-3), but it's really cool to see it tried. For those interested: SPIN and other hybrid kernels (like Exo) were created in the fall-out of "microkernels are bad" by attempting to allow a hybrid approach. Linux was created around the same time staying straight in the…

Out of the loop, why is Linux now "more hybrid"?

kernel modules and BPF ....

Re: BPF: A New Type of Software

#144

Post by boomboomsubban »

smorrow wrote: post #139
willisbueller wrote: post #135

Sounds a lot like SPIN OS https://en.wikipedia.org/wiki/SPIN_(operating_system) They have to make do without type safety (in SPIN's case provided by modula-3), but it's really cool to see it tried. For those interested: SPIN and other hybrid kernels (like Exo) were created in the fall-out of "microkernels are bad" by attempting to allow a hybrid approach. Linux was created around the same time staying straight in the…

Out of the loop, why is Linux now "more hybrid"?

Many of the benefits of microkernels have been implemented in Linux over the past 25 years. Modules, FUSE, and live patching for example.

Re: BPF: A New Type of Software

#145

Post by PaulDavisThe1st »

MisterTea wrote: post #119
jancsika wrote: post #98

Earlier quoted context omitted.

To get the realtime benefits I'd have to run everything that requires realtime scheduling as an eBPF program, right? I mean, suppose I want to make an eBPF program to pipe audio input into my very fancy DSP algo. Awesome. But now I've limited the remainder of my signal chain to remain in eBPF land, haven't I? The moment I throw the signal to Supercollider, Pd, some Jack client, etc., I'm back to being a mere userland…

> So at least in this domain I have a hard time seeing how eBPF could practically help realtime scheduling. Right, because jack is a user space collection of programs. It's obvious that any real-time or low latency DSP audio processing pipeline would have to be moved to BPF in its entirety. So yes, BPF would work just fine if a new DAW pipeline was designed to work with in it. User space programs would only function…

You're never going to fit Ardour's engine into 1M instructions, nor is it going to work without unbounded loops. So that's not going to happen.

More interesting would be using BPF for the driver and being able to directly invoke JACK (or some other user-space endpoint) in a way that essentially bypasses the scheduler.

Put differently: rather than rely on the normal scheduler pathway to waking up JACK (or some other user-space endpoint), have BPF do it as soon as the device driver is ready.

But I don't think BPF has any way to do this at present - it seems to rely on accumulation of data in kernel space and a user space poll-driven process to pick it up and push it at the user.

Re: BPF: A New Type of Software

#146

Post by pm90 »

smorrow wrote: post #139
willisbueller wrote: post #135

Sounds a lot like SPIN OS https://en.wikipedia.org/wiki/SPIN_(operating_system) They have to make do without type safety (in SPIN's case provided by modula-3), but it's really cool to see it tried. For those interested: SPIN and other hybrid kernels (like Exo) were created in the fall-out of "microkernels are bad" by attempting to allow a hybrid approach. Linux was created around the same time staying straight in the…

Out of the loop, why is Linux now "more hybrid"?

Because it has a crap ton of drivers and shit in the kernel, which is the opposite of unikernel/microkernel approach where most of this functionality is implemented in userspace. See eg Minix

Re: BPF: A New Type of Software

#147

Post by PaulDavisThe1st »

rusk wrote: post #99
jancsika wrote: post #98

Earlier quoted context omitted.

To get the realtime benefits I'd have to run everything that requires realtime scheduling as an eBPF program, right? I mean, suppose I want to make an eBPF program to pipe audio input into my very fancy DSP algo. Awesome. But now I've limited the remainder of my signal chain to remain in eBPF land, haven't I? The moment I throw the signal to Supercollider, Pd, some Jack client, etc., I'm back to being a mere userland…

Hm, good point. But couldn't you say the same thing about CUDA? There is an overhead involved in shunting data to and from the cores ... but the nature of the work being done, and the massive parallelism, mean the overall end-to-end volume of throughput increases dramatically. I don't know enough about audio processing to answer your specific question ... but processing network traffic for instance, you have the oppo…

There's bandwidth and there's latency. CUDA has the bandwidth, but not the latency, for generalizable low latency audio processing.

Re: BPF: A New Type of Software

#148

Post by eloff »

EsssM7QVMehFPAs wrote: post #134
leetrout wrote: post #62

Earlier quoted context omitted.

All snark aside, is there potential benefit to Varnish Cache with this becoming widely adopted? Things that could only be accomplished at lower layers like this implies access to?

Since there is already highly efficient event driven I/O and a context switch is bound to happen for applications that require major business logic in user space, I doubt there will be huge benefits for large server applications. Maybe simplified versions work with this paradigm with significant performance gains, but I doubt that you could simply plug Varnish or nginx in and see much of an improvement. There is stil…

I think you could use this to generate a reverse proxy for the current config in nginx, maybe if only for some subset of configurations. Like a fast path for common flows. But it wouldn't be easy.

Re: BPF: A New Type of Software

#149

Post by monocasa »

sigjuice wrote: post #121
monocasa wrote: post #94

Earlier quoted context omitted.

The end goal with bpf is to allow arbitrary untrusted programs to load bpf programs. If you were just loading kernel modules you wouldn't be able to maintain kernel integrity and let arbitrary programs load code.

I can't find the LKML thread, but there is some fundamental problem with unloading modules safely. BPF programs might not have such limitations.

Oh, neat! Yeah, I didn't think of this but that totally makes sense.

BPF programs declare their resources to the kernel (maps, etc.), but modules are reliant on the __exit function cleaning everything up properly. The correctness of __exit is unverified, difficult, and practically one of the least tested pathways which makes it traditionally fraught with bugs.

Re: BPF: A New Type of Software

#150

Post by boomlinde »

mkup wrote: post #142
boomlinde wrote: post #88

Earlier quoted context omitted.

What use case do you envision?

Out-of-tree drivers, support for closed-source drivers in Linux (assuming kernel API for wasm drivers is stable).

While I see how out-of-tree drivers using a stable API might be a desirable goal, I don't understand what WASM adds to the table. Maybe some degree of cross-arch portability, but that's already afforded by C and the kernel.
Post reply on HN