Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

BPF: A New Type of Software

brendangregg.com

Post reply on HN
Print view View original

171–180 of 192 posts

Re: BPF: A New Type of Software

#171

Post by monocasa »

manyworlds wrote: post #167
monocasa wrote: post #158

Earlier quoted context omitted.

Sort of, but misses some of the larger picture. The main reason that fs code is faster in the kernel is the direct access to kernel data structures. File system, virtual memory, and buffer cache are all three sides of the same coin. Once you divorce yourself from direct (even if sandboxed) read and writes of the underlying data structures, you impose a massive overhead.

Hmm I don’t think this is the case, at least not when comparing fuse to in-kernel file systems. Having access to native VM structures only helps to the extent that you can avoid copies, yet in fuse, only one extra copy takes place. I think having to switch tasks (and associated work: swapping mm, flushing tlb, ireting/syscalling, synchronization) is really what kills perf

Torvalds disagrees with you, at least wrt to the fundamental limitation here (there may be other issues layered on top of it of course).

> No, you need not just the blocks, you need the actual cache chain data structures themselves. For doing things like good read-ahead, you need to be able to (efficiently) look up trivial things like "is that block already in the cache".

> So you need not only the data, you need the _tags_ too.

> In other words, your filesystem needs to have access to the whole disk cache layer, not just the contents. Or it will not perform well.

https://yarchive.net/comp/microkernels.html

Edit: and the context of this discussion was fairly ancient systems with tagged TLBs and simple in order cores with syscalls nearly as cheap as regular user space call instructions. They were still ungodly slow with microkernels, and his explanation is the meat of his view as to why. It's all about having the data in the right place with as little synchronization required.

Re: BPF: A New Type of Software

#172

Post by rhinoceraptor »

manyworlds wrote: post #67
rhinoceraptor wrote: post #55

Earlier quoted context omitted.

BPF is completely production safe. So there is no way for a BPF program to crash the kernel, introduce significant performance latency, or have any side effects on the kernel/user space. Obviously, kernel modules have none of those properties. Also, BPF has been around for almost 30 years, and you're likely using it. tcpdump is basically just a BPF bytecode frontend, for example.

You can maintain production safety by using a BPF->kernel module compiler. This additionally removes the need to have the bpf compiler in the kernel, reducing both core size and vulnerability surface area. No reason BPF must imply JIT

eBPF and kernel modules solve completely different tasks. If you need a kernel module to accomplish your task, then by definition you cannot accomplish it with eBPF.

Re: BPF: A New Type of Software

#173

Post by monocasa »

manyworlds wrote: post #169
monocasa wrote: post #165

Earlier quoted context omitted.

That's where it is today, but the end goal is removing that restriction. It probably would have happened quicker if Spectre/Meltdown hadn't come out of nowhere. Like it used to be that KVM required CAP_SYS_ADMIN as well, but now that's been opened up to whoever has permissions to the device file. Start requiring "own the box anyway" privileges while the feature bakes, but open it up as it becomes more mature and atta…

Sure but I don’t see any compelling use cases to motivate opening it up. Do you have an example of one? Even if the kernel opens it up without a compelling use case, it seems likely that distribution policy will keep it default locked to root. Which is my point here. I don’t see a compelling reason to have a JIT in the kernel when AOT BPF seems to cover 90% of all existing use cases. In fact I may even write a bpf to…

* Syscall tracing, sandboxing, and monitoring

* KVM device MMIO emulation

* OS personality emulation (like WSL but doesn't require root)

* New synchronization primitives to user space (like XOK's wake predicates)

* A lot of others..

Modern BPF is the exploring the same cornerstones as exokernels and really opens up a whole bunch of concepts that haven't been seen in mainstream kernels, particularly if non privileged users are allowed to invoke it.

Re: BPF: A New Type of Software

#174

Post by waste_monk »

rusk wrote: post #19
jokoon wrote: post #16

I have a hard time understanding what you would use it for. I could understand a use-case, but I fail to understand why it would be that much useful. I have a sense it allows much better performance for horizontal scaling, but I'm not sure...

Real-time, low latency, network-based applications. At the pace of network events, CPU is still very fast by perhaps at least order of magnitude. However, latency introduced by system calls is significant. This allows you to run certain classes of application in kernel space with these overheads largely mitigated. Principally it's monitoring and "observability" applications, but apparently it's much more flexible now…

> Real-time, low latency, network-based applications.

I thought these kinds of applications generally (try to) avoid the kernel entirely e.g. high frequency trading algorithms running on FPGAs.

Maybe using it to get data on/off of the FPGA for charting / updates / etc.

Re: BPF: A New Type of Software

#175

Post by pests »

anderspitman wrote: post #154
irq-1 wrote: post #109

Earlier quoted context omitted.

I was thinking about something similar to extfuse, but for a remote filesystem. Here's some note scraps: > Server abilities can be changed by uploading JS functions/libs.. > Want to search a file formats meta-data on the server? Upload JS to read the meta-data and index it, and provide the search options.. > Servers should also use the best options for the tasks they provide: Grep to seach text -- and not Grep like f…

This is an interesting idea. Sort of the inverse of a web app. Part of the problem with the cloud for large datasets (ie genomics) is getting the computation close enough to the data (the UI being the third leg of the stool). If you could upload small processing scripts (or ebpf/wasm) to the exact node where the data lives in real time, it might open up some novel techniques. Kind of like current serverless tech but…

I believe this was a huge strength of MapReduce back in the day. The mapping and reducing of initial data would happen on the node the data was stores in. The only thing being transferred was the code and the results.

Re: BPF: A New Type of Software

#176

Post by manyworlds »

monocasa wrote: post #173
manyworlds wrote: post #169

Earlier quoted context omitted.

Sure but I don’t see any compelling use cases to motivate opening it up. Do you have an example of one? Even if the kernel opens it up without a compelling use case, it seems likely that distribution policy will keep it default locked to root. Which is my point here. I don’t see a compelling reason to have a JIT in the kernel when AOT BPF seems to cover 90% of all existing use cases. In fact I may even write a bpf to…

* Syscall tracing, sandboxing, and monitoring * KVM device MMIO emulation * OS personality emulation (like WSL but doesn't require root) * New synchronization primitives to user space (like XOK's wake predicates) * A lot of others.. Modern BPF is the exploring the same cornerstones as exokernels and really opens up a whole bunch of concepts that haven't been seen in mainstream kernels, particularly if non privileged…

Thanks for the examples but those all still seem like things vast majority of Linux users can do today, since vast majority of Linux users have root access. Both desktop and server.

Mobile users like android don’t have root but I don’t see why an untrusted mobile app would need bpf.

Only benefit of allowing non-root that I can see is enabling untrusted containers in cloud environments to do the same. All large cloud providers use KVM/zen (not containers) for untrusted users in which case they already have root.

Can you give an example of a scenario where the user doesn’t have root yet still would want to do those things?

Re: BPF: A New Type of Software

#177

Post by manyworlds »

monocasa wrote: post #171
manyworlds wrote: post #167

Earlier quoted context omitted.

Hmm I don’t think this is the case, at least not when comparing fuse to in-kernel file systems. Having access to native VM structures only helps to the extent that you can avoid copies, yet in fuse, only one extra copy takes place. I think having to switch tasks (and associated work: swapping mm, flushing tlb, ireting/syscalling, synchronization) is really what kills perf

Torvalds disagrees with you, at least wrt to the fundamental limitation here (there may be other issues layered on top of it of course). > No, you need not just the blocks, you need the actual cache chain data structures themselves. For doing things like good read-ahead, you need to be able to (efficiently) look up trivial things like "is that block already in the cache". > So you need not only the data, you need the…

Ah okay, fine grained control of cache to minimize IO waiting is a good counterpoint.

I actually have a lot of experience in this area, and I can say that effective readahead is a bit of a crapshoot. Only really works in trivial cases. Ultimately if IO latency sucks, nothing can save you.

His particular point doesn’t fully make sense either. It’s easy to kick off readahead when you only have access to block data, the kernel won’t issue redundant IO requests for blocks already in the cache. Also mlock/madvise give a lot of control in terms of dictating eviction strategies for special blocks.

All thins equal (costless syscall/mm swapping, IO), I still think inter-task synchronization is the largest overhead, but I have no numbers to back it up. Something tells me Marshalling all IO syscalls to a kernel thread would be about as slow as a user-space FUSE task.

Re: BPF: A New Type of Software

#178

Post by Iv »

wisty wrote: post #10
Iv wrote: post #6

So from my understanding, that's a kind of "secure" (I'd like to know more about the security model tbh) module that runs with kernel privilege with no scheduling (so it runs until completion). These are supposed to be short and I am assuming, can't call libs and can't allocate memory (outside a predefined stack I would guess?) Aren't they very similar to interrupts? What is the difference there? The kernel API?

IIR I think it's a VM(?) that has certain limits, and because of those limits it's OK to run it in kernel-space. IIRC it's not Turning complete, and has a fixed run time. So Linux can just say "OK run this now" and not worry about scheduling. It's like putting a green thread in the kernel but to do this safely you need very strict restrictions (finite memory it can access, finite number of steps, etc). You can thus g…

Got it, thanks! I did not realize it runs in a VM, ok, that indeed makes sense to call it a different kind of application.

Re: BPF: A New Type of Software

#179

Post by sansnomme »

throwaway894345 wrote: post #116
hackworks wrote: post #79

eBPF can be viewed as a mechanism to safely run user code in kernel since it uses a DSL and a compiler before the byte code is executed in kernel. This opens up doors for running performance critical functionality in kernel without having to bundle it with the kernel or very tightly coupled with the kernel version. Optimizing FUSE is an example: https://extfuse.github.io/ I expect custom security auditing software, r…

According to https://news.ycombinator.com/item?id=18496054 , these programs have to halt? How does this system guarantee that the programs halt? Does this mean eBPF is not Turing complete?

Commonly known as total functional programming.

https://en.m.wikipedia.org/wiki/Total_functional_programming

Post reply on HN