Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Application Traffic with eBPF

thebsdbox.co.uk

Post reply on HN
Print view View original

1–10 of 20 posts

Re: Application Traffic with eBPF

#3

Post by jcul »

jimmyed wrote: post #2

Isn't this how tcpdump/ngrep/gopacket work? For parsing the HTTP protocol, I find netpeek effective [1] https://github.com/darshanime/netpeek

I'm not too familiar with eBPF (yet, keep meaning to dig into it).

But I believe tcpdump just opens a AF_PACKET socket, and it will add a BPF filter if one is specified.

I don't know enough to say how this relates to the eBPF stuff, but I think internally the kernel may convert the BPF program to eBPF.

Edit: netpeek looks cool, thanks for sharing!

Re: Application Traffic with eBPF

#4

Post by sharangxy »

For those interested, you can also take a look at our open-source project DeepFlow - https://deepflow.io - https://github.com/deepflowio/deepflow

We use eBPF to achieve non-intrusive (we call it `zero-code`) observability without modifying any application code, and have implemented three core features: Universal Map, Distributed Tracing, and Continuous Profiling.

Yes, we have implemented *Distributed* tracing using eBPF. Because of this achievement, we have also published a paper in ACM SIGCOMM 2023.

Re: Application Traffic with eBPF

#5

Post by intelVISA »

sharangxy wrote: post #4

For those interested, you can also take a look at our open-source project DeepFlow - https://deepflow.io - https://github.com/deepflowio/deepflow We use eBPF to achieve non-intrusive (we call it `zero-code`) observability without modifying any application code, and have implemented three core features: Universal Map, Distributed Tracing, and Continuous Profiling. Yes, we have implemented *Distributed* tracing using e…

The copy is way too buzzword dense imo

Re: Application Traffic with eBPF

#6

Post by phrotoma »

sharangxy wrote: post #4

For those interested, you can also take a look at our open-source project DeepFlow - https://deepflow.io - https://github.com/deepflowio/deepflow We use eBPF to achieve non-intrusive (we call it `zero-code`) observability without modifying any application code, and have implemented three core features: Universal Map, Distributed Tracing, and Continuous Profiling. Yes, we have implemented *Distributed* tracing using e…

For distributed tracing, how is deepflow able to correlate an inbound request (eg. client call) with an outbound request (eg. 3rd party API call required to service client call) without being inside the business logic?

Re: Application Traffic with eBPF

#7

Post by vitus »

jimmyed wrote: post #2

Isn't this how tcpdump/ngrep/gopacket work? For parsing the HTTP protocol, I find netpeek effective [1] https://github.com/darshanime/netpeek

tcpdump only uses BPF, not eBPF. BPF is a simpler language that, among other things, is guaranteed to run in finite time because it doesn't have backward jumps, and has limitations on program size (4096 instructions). (The "e" in eBPF stands for "extended", as it extends BPF to remove those limitations, among other changes.)

It compiles your filter expression into a series of instructions, using libpcap. For instance, the output of `tcpdump -d -y EN10MB 'ip and tcp port 80' (which is rather similar to the use case in the OP, but not identical, since it doesn't strip headers), on my machine, is:

    # Load the 2-byte ethernet protocol into A (the accumulator register).
    (000) ldh      [12]
    # If IPv4, continue to line 2. Else, jump to line 12.
    (001) jeq      #0x800           jt 2 jf 12
    # Load the one-byte IP protocol into A.
    (002) ldb      [23]
    # If TCP, continue. Else, jump to line 12.
    (003) jeq      #0x6             jt 4 jf 12
    # Load the 2 bytes corresponding to IP flags / fragment offset into A.
    (004) ldh      [20]
    # If the fragment offset is nonzero, jump to line 12. Else, continue.
    (005) jset     #0x1fff          jt 12 jf 6
    # Load the internet header length into X. (Note that this is the bottom 4
    # bits of the first byte of the IPv4 header, expressed in 4-byte words)
    (006) ldxb     4*([14]&0xf)
    # Load the source port into A.
    (007) ldh      [x + 14]
    # If 80, jump to 11. Else, continue.
    (008) jeq      #0x50            jt 11 jf 9
    # Load the dest port into A.
    (009) ldh      [x + 16]
    # If 80, jump to 11. Else, jump to 12.
    (010) jeq      #0x50            jt 11 jf 12
    # Accept. Return 262144 bytes, the default snaplen.
    (011) ret      #262144
    # Reject. Literally, return 0 bytes.
    (012) ret      #0
If you know how to read assembly, it should be fairly straightforward to follow a typical program (you'll need various protocol header wire formats handy if you haven't memorized the offsets).

Re: Application Traffic with eBPF

#8

Post by thebsdbox »

vitus wrote: post #7
jimmyed wrote: post #2

Isn't this how tcpdump/ngrep/gopacket work? For parsing the HTTP protocol, I find netpeek effective [1] https://github.com/darshanime/netpeek

tcpdump only uses BPF, not eBPF. BPF is a simpler language that, among other things, is guaranteed to run in finite time because it doesn't have backward jumps, and has limitations on program size (4096 instructions). (The "e" in eBPF stands for "extended", as it extends BPF to remove those limitations, among other changes.) It compiles your filter expression into a series of instructions, using libpcap. For instance…

Wow.. that was an epic response :-D

Re: Application Traffic with eBPF

#9

Post by sofixa »

sharangxy wrote: post #4

For those interested, you can also take a look at our open-source project DeepFlow - https://deepflow.io - https://github.com/deepflowio/deepflow We use eBPF to achieve non-intrusive (we call it `zero-code`) observability without modifying any application code, and have implemented three core features: Universal Map, Distributed Tracing, and Continuous Profiling. Yes, we have implemented *Distributed* tracing using e…

I have to say I find projects that talk about generic concepts (observability, tracing, eBPF), but then when you dig in the docs it's 100% Kubernetes-specific, to be highly misleading. Not everyone uses Kubernetes, and distributed tracing is a good thing to have regardless of the underlying platform.

Re: Application Traffic with eBPF

#10

Post by phillipcarter »

sofixa wrote: post #9
sharangxy wrote: post #4

For those interested, you can also take a look at our open-source project DeepFlow - https://deepflow.io - https://github.com/deepflowio/deepflow We use eBPF to achieve non-intrusive (we call it `zero-code`) observability without modifying any application code, and have implemented three core features: Universal Map, Distributed Tracing, and Continuous Profiling. Yes, we have implemented *Distributed* tracing using e…

I have to say I find projects that talk about generic concepts (observability, tracing, eBPF), but then when you dig in the docs it's 100% Kubernetes-specific, to be highly misleading. Not everyone uses Kubernetes, and distributed tracing is a good thing to have regardless of the underlying platform.

Another variant of this is when something mentions code profiling and it only supports Go.
Post reply on HN