Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

eBPF Is Awesome

filipnikolovski.com

Post reply on HN
Print view View original

21–30 of 46 posts

Re: eBPF Is Awesome

#21

Post by londons_explore »

tptacek wrote: post #17
chubot wrote: post #7

Earlier quoted context omitted.

I think dtrace has the same problem, i.e. it's pretty tightly coupled to the exact functions / trace points in the kernel. A different kernel can break a dtrace script, although I think their code changes a lot less than Linux does. It seems somewhat unavoidable, if the goal is to introspect the kernel at a very intimate level ...

BTF is designed to avoid the problem, by having the live kernel export symbols; supposedly --- I haven't used it --- the toolchain even converts it to a header file, so BPF programs just include a "vmlinux.h" instead of includes pointing into kernel source (which is a nightmare). It's ambitious and I'm surprised it does as much as it does but apparently they're solving this problem.

I can imagine it leading to ossification of kernel internals.

Imagine when someone comes up with a revolutionary new paging method, but it causes everyone's eBPF scripts to fail to load and a bunch of tools to break...

Re: eBPF Is Awesome

#22

Post by ithkuil »

0b01 wrote: post #12
ncmncm wrote: post #8

It seems worth mentioning that the code actually executing in the kernel, when it is running your eBPF, is native machine code, ahead-of-time compiled from the bytecode program you gave to the kernel.

Technically it's JITted not AOT compiled.

The line is blurry.

JIT compilation means compiling code on the fly right when you're about to execute it. When the code is part of a larger program, jitting allows to compile the parts of it you actually need and avoid wasting time to compile other parts. It also allows to compile the relevant code paths based on dynamic flow analysis, which often involves interpreting your program the first time you run it and emitting instructions for the next time around (tracing)

If the code unit is small and you know you're going to run it all, you can compile it in one swoop. If you compile it when you load it in the kernel, as opposed to compiling it lazily right before you run it the first time, i think it's fair to call this Ahead-of-Time compilation, even if the compilation happens right next the use site and not as part of the developer tool chain

Re: eBPF Is Awesome

#23

Post by bcantrill »

chubot wrote: post #7
tptacek wrote: post #3

Most examples of BPF code are written in a mix of Python and C using BCC, the "BPF Compiler Collection", which essentially treats all of LLVM and clang as a library callable from Python code. I can't get my head around using it that way, and have found it pretty straightforward to just write C programs, compiled with clang `-target bpf`. Until very recently, writing anything interesting this way required you to decla…

I think dtrace has the same problem, i.e. it's pretty tightly coupled to the exact functions / trace points in the kernel. A different kernel can break a dtrace script, although I think their code changes a lot less than Linux does. It seems somewhat unavoidable, if the goal is to introspect the kernel at a very intimate level ...

No, DTrace does not have this problem, though our solution to it is one of the least well known aspects of DTrace: we have a notion of explicit stability that allows for stable scripts to be built on top of very low level implementation details that themselves might change. See the chapter on "Stability" in the Dynamic Tracing Guide[1] for details.

[1] http://dtrace.org/guide/chp-stab.html

Re: eBPF Is Awesome

#24

Post by laserbeam »

0b01 wrote: post #9
ChrisMarshallNY wrote: post #5

That sounds extremely cool. Sadly, I don't program in Linux, so I can't use it. :'(

If you program on Windows you should check out Event Tracing for Windows(ETW). Similar to eBPF, ETW is a logging framework inside Windows kernal. Microsoft.Diagnostics.Tracing.TraceEvent[0] is a nice nuget package for logging and analyzing ETL files. [0] https://github.com/microsoft/perfview/blob/master/documentat...

But only after reading this glorious and funny article about using ETW for logging thread context switches. https://caseymuratori.com/blog_0025

Re: eBPF Is Awesome

#25

Post by perlgeek »

Is it possible to write device drivers in eBPF?

(I've asked this before, but haven't gotten any response, and no clear answer from Google/DDG either).

Re: eBPF Is Awesome

#26

Post by 0b01 »

laserbeam wrote: post #24
0b01 wrote: post #9

Earlier quoted context omitted.

If you program on Windows you should check out Event Tracing for Windows(ETW). Similar to eBPF, ETW is a logging framework inside Windows kernal. Microsoft.Diagnostics.Tracing.TraceEvent[0] is a nice nuget package for logging and analyzing ETL files. [0] https://github.com/microsoft/perfview/blob/master/documentat...

But only after reading this glorious and funny article about using ETW for logging thread context switches. https://caseymuratori.com/blog_0025

Thankfully all of this native API is abstracted away in C#.

Re: eBPF Is Awesome

#28

Post by bboreham »

tptacek wrote: post #3

Most examples of BPF code are written in a mix of Python and C using BCC, the "BPF Compiler Collection", which essentially treats all of LLVM and clang as a library callable from Python code. I can't get my head around using it that way, and have found it pretty straightforward to just write C programs, compiled with clang `-target bpf`. Until very recently, writing anything interesting this way required you to decla…

CO-RE is great, but for those who have to run on older kernels an approach is to loop, guessing the offset and running an experiment to see if correct:

https://github.com/weaveworks/tcptracer-bpf/blob/cd53e7c84ba...

This was done (by Kinvolk) for the visualisation tool Weave Scope; also picked up by DataDog https://github.com/DataDog/datadog-process-agent/tree/master...

Re: eBPF Is Awesome

#29

Post by chubot »

bcantrill wrote: post #23
chubot wrote: post #7

Earlier quoted context omitted.

I think dtrace has the same problem, i.e. it's pretty tightly coupled to the exact functions / trace points in the kernel. A different kernel can break a dtrace script, although I think their code changes a lot less than Linux does. It seems somewhat unavoidable, if the goal is to introspect the kernel at a very intimate level ...

No, DTrace does not have this problem, though our solution to it is one of the least well known aspects of DTrace: we have a notion of explicit stability that allows for stable scripts to be built on top of very low level implementation details that themselves might change. See the chapter on "Stability" in the Dynamic Tracing Guide[1] for details. [1] http://dtrace.org/guide/chp-stab.html

Thanks for the reference... although it seems clear upon reading it that the problem is mitigated (by tagging/documentation) but still present. Although the tool support looks nice and is probably something that could be borrowed for eBPF.

Re: eBPF Is Awesome

#30

Post by polskibus »

What are the benefits of using eBPF besides a promise of observability "for free"?

Can eBPF be used for observability using platforms like Java or .net core, or does their platform VMs obfuscate too much and monitoring them using eBPF is not feasible?

How does eBPF work wrt OpenTelemetry etc.? Should OpenTelemetry be seen as standardized interfaces to which eBPF reports data?

Post reply on HN