Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No More Blue Fridays

brendangregg.com

Post reply on HN
Print view View original

31–40 of 282 posts

Re: No More Blue Fridays

#31

Post by mrpippy »

> Once Microsoft's eBPF support for Windows becomes production-ready, Windows security software can be ported to eBPF as well.

This doesn’t seem grounded in reality. If you follow the link to the “hooks” that Windows eBPF makes available [1], it’s just for incoming packets and socket operations. IOW, MS is expecting you to use the Berkeley Packet Filter for packet filtering. Not for filtering I/O, or object creation/use, or any of the other million places a driver like Crowdstrike’s hooks into the NT kernel.

In addition, they need to be in the kernel in order to monitor all the other 3rd party garbage running in kernel-space. ELAM (early-launch anti-malware) loads anti-malware drivers first so they can monitor everything that other drivers do. I highly doubt this is available to eBPF.

If Microsoft intends eBPF to be used to replace kernel-space anti-malware drivers, they have a long, long way to go.

[1]: https://microsoft.github.io/ebpf-for-windows/ebpf__structs_8...

Re: No More Blue Fridays

#32

Post by worthless-trash »

efee22 wrote: post #14
dijit wrote: post #9

Earlier quoted context omitted.

And assuming there's no bugs in the BPF code... Oh wait: https://news.ycombinator.com/item?id=41031699

RHEL kernel.. right. Imho, I'd trust an upstream stable kernel far more than a RHEL one for production which has dozen of feature backports and an internal kABI to maintain.. granted RH has a QA team, but it is still impossible to test everything beforehand.

On the upside, non root users can't insert ebpf code, so its a priv'ed operation, not like other distros.

Re: No More Blue Fridays

#33

Post by efee22 »

skywhopper wrote: post #21

The implicit assumption of the article is that eBPF code can't crash a kernel, but the article itself eventually admits that it can and has done, including last month. eBPF is a safer way of providing kernel-extension functionality, for sure, but presenting it as the perfect solution is just asking to have your argument dismissed. eBPF is not perfect. And there's plenty of things it can't do. The very sandbox rules t…

It's not a silver bullet, however, it is still better to pushing all the panicable bugs into one community-maintained section (e.g. eBPF verifier). All vendors have an incentive to help get right and this is much better than every vendor shipping their own panicable bugs in their own out of tree kernel modules. Additionally, it's not just the industry looking at eBPF, but also academia in terms of formally verifying these critical sections.

Re: No More Blue Fridays

#34

Post by Smaug123 »

kayo_20211030 wrote: post #15

This isn't right. If I need a system to run with a piece of code, then it shouldn't run at all if that piece of code is broken. Ignoring the failure is perverse. Let's say that the driver code ensures that some medical machine has safety locks (safeguards) in place to make sure that piece of equipment won't fry you to a crisp; I'd prefer that the whole thing not run at all rather than blithely operate with the safegu…

I think the premise is false? It's up to the eBPF implementor what to do in the case of invalid input; the kernel could choose to perform a controlled shutdown in that case. (I have no idea what e.g. Linux actually does here, but one could imagine worlds where the action it takes on invalid input is configurable.)

Also your statement is sometimes not true, although I certainly sympathise in the mainline case. In some contexts you really do need to keep on trucking. The first example to spring to mind is "the guidance computers on an automated Mars lander"; the round-trip to Earth is simply too long to defer responsibility in that case. If you shut down then you will crash, but if you do your best from a corrupted state then you merely probably crash, which is presumably better.

Re: No More Blue Fridays

#35

Post by aksdlf »

shrx wrote: post #18

From the article: > If the verifier finds any unsafe code, the program is rejected and not executed. The verifier is rigorous -- the Linux implementation has over 20,000 lines of code [0] -- with contributions from industry (e.g., Meta, Isovalent, Google) and academia (e.g., Rutgers University, University of Washington). [0] links to https://github.com/torvalds/linux/blob/master/kernel/bpf/ver... which has this inter…

I'm glad to hear that Meta and Google code is "rigorous". I'd prefer INRIA, universities that fund theorem provers, industries where correctness matters like aerospace or semiconductors.

Re: No More Blue Fridays

#37

Post by Archelaos »

blinkingled wrote: post #30

Ok. But the good old push code to staging / canary it before mainstream updates was a simpler way of solving the same problem. Crowdstrike knows the computers they're running on, it is trivial to implement a system where only few designated computers download and install the update and report metrics before the update controller decides to push it to next set.

It would mitigate the problem, but not solve it. You can still imagine a condition that only occurs after the update has been rolled out everywhere. Furthermore, such a bug would still be extremely problematic for the concerned customers, even if not all of them were affected. In addition, it would be necessary to react very quickly in the case of zero-day vulnerabilities.

Re: No More Blue Fridays

#38

Post by shahahqq »

mrpippy wrote: post #31

> Once Microsoft's eBPF support for Windows becomes production-ready, Windows security software can be ported to eBPF as well. This doesn’t seem grounded in reality. If you follow the link to the “hooks” that Windows eBPF makes available [1], it’s just for incoming packets and socket operations. IOW, MS is expecting you to use the Berkeley Packet Filter for packet filtering. Not for filtering I/O, or object creation/…

I hope though that Microsoft will double down on their eBPF support for Windows after this incident.

Re: No More Blue Fridays

#39

Post by hello_moto »

lordnacho wrote: post #11
StevenWaterman wrote: post #6

Earlier quoted context omitted.

If you consider kernel programming to be inherently unsafe, then you would consider this to be inevitable, meaning it's not really the specific company's fault. They were just the unlucky ones.

They could have helped their luck by doing some of the common sense things suggested in the article. For instance, why not find a subset of your customers that are low risk, push it out to them, and see what happens? Or perhaps have your own fleet of example installations to run things on first. None of which depends on any specific technology.

"find a subset of low risk customers" and use them as test subject?

Repeat that a few times to understand the repercussions.

If I were the customers and I found out that I was used as test subject, how would I feel?

Re: No More Blue Fridays

#40

Post by xyzzy123 »

So many problems though! including commercial monocultures, lack of update consent, blast radius issues, etc etc. There's a commons in our pockets but that is very difficult to regulate for. The will keep putting the gun to your head until you keep choosing the monoculture.
Post reply on HN