Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Packet, where are you? – eBPF-based Linux kernel networking debugger

github.com

Post reply on HN
Print view View original

11–20 of 20 posts

Re: Packet, where are you? – eBPF-based Linux kernel networking debugger

#11

Post by bazzargh »

ralgozino wrote: post #4

Asking as someone with almost no experience with eBPF. How do you read the otuput of the tool? In the README's example GIF, If I didn't know there's an IPTables rule dropping the packages I would not know that the package is being dropped.

if you have a recent enough kernel, this change https://github.com/cilium/pwru/pull/148 means that it will print the reason the packet was dropped in the output, as logged by kfree_skb_reason - see https://lwn.net/Articles/885729/

There's a whole heap of reasons a packet can be dropped: https://github.com/torvalds/linux/blob/76f598ba7d8e2bfb4855b...

In any case, this makes it less painful than going back to read the kernel source. I don't think that the gif in the README is up to date with this change - it's 2 years old and this feature only appeared 2 months ago.

Re: Packet, where are you? – eBPF-based Linux kernel networking debugger

#12

Post by dhaavi »

aeonik wrote: post #8
totetsu wrote: post #2

Is 'tracing network packets in the kernel' also what tcpdump/snoop/nettl dp?

I can speak for tcpdump and the answer is no. It only looks at the network interface. Something I often want to do is catch which process sent a few UDP packets. Netstat and ss won't catch it because it's too short of a time frame, and tcpdump doesn't contain any information about the kernel. I've been using opensnitch which uses eBPF rules to track this information lately, but I'm looking for something more flexible…

I don't know what you mean by "flexible", but you could check if what we are building - Portmaster Privacy Suite - goes in that direction: https://safing.io/

Re: Packet, where are you? – eBPF-based Linux kernel networking debugger

#14

Post by ralgozino »

bazzargh wrote: post #11
ralgozino wrote: post #4

Asking as someone with almost no experience with eBPF. How do you read the otuput of the tool? In the README's example GIF, If I didn't know there's an IPTables rule dropping the packages I would not know that the package is being dropped.

if you have a recent enough kernel, this change https://github.com/cilium/pwru/pull/148 means that it will print the reason the packet was dropped in the output, as logged by kfree_skb_reason - see https://lwn.net/Articles/885729/ There's a whole heap of reasons a packet can be dropped: https://github.com/torvalds/linux/blob/76f598ba7d8e2bfb4855b... In any case, this makes it less painful than going back to read the…

that is pretty cool and a major QoL improvement indeed!

Re: Packet, where are you? – eBPF-based Linux kernel networking debugger

#15

Post by elesiuta »

aeonik wrote: post #8
totetsu wrote: post #2

Is 'tracing network packets in the kernel' also what tcpdump/snoop/nettl dp?

I can speak for tcpdump and the answer is no. It only looks at the network interface. Something I often want to do is catch which process sent a few UDP packets. Netstat and ss won't catch it because it's too short of a time frame, and tcpdump doesn't contain any information about the kernel. I've been using opensnitch which uses eBPF rules to track this information lately, but I'm looking for something more flexible…

> Something I often want to do is catch which process sent a few UDP packets.

Catching short lived processes and packets is one of the things I specialized picosnitch [1] for, which focuses strictly on monitoring.

[1] https://github.com/elesiuta/picosnitch

Re: Packet, where are you? – eBPF-based Linux kernel networking debugger

#17

Post by tptacek »

nicevalue wrote: post #16

What is the difference b.w this tool and Dropwatch [ https://linux.die.net/man/1/dropwatch ?]

dropwatch is much more limited, and effectively obsolete. It's a purpose-designed kernel feature with its own netlink protocol that has one serious consumer (the accursed dropwatch.c client). Leave pwru aside for a minute, and everything dropwatch can do, you can do better just with bpftrace.

Re: Packet, where are you? – eBPF-based Linux kernel networking debugger

#20

Post by suprjami »

nicevalue wrote: post #16

What is the difference b.w this tool and Dropwatch [ https://linux.die.net/man/1/dropwatch ?]

Dropwatch just tells you where packets were freed from memory by the "bad" path of `kfree_skb()`. It relies on the "good" packet free path to use `consume_skb()` which isn't always true, so you get a number of false positives with dropwatch. It's hard to spot small individual packet loss among all the noise.

pwru offers src/dst IP/port filtering, so it's easier to exclude uninteresting traffic, and it tracks an skb's journey through the entire network stack, printing each function the skb touches over its lifetime.

They are quite different tools, they probably have some overlap depending on situation.

Also the actual dropwatch command is obsolete, it's more common and useful to use the SystemTap script, or perf, or bpftrace.

Post reply on HN