Live data from Hacker News

No More Blue Fridays

brendangregg.com

71–80 of 282 posts

Re: No More Blue Fridays

#71

Earlier quoted context omitted.

"find a subset of low risk customers" and use them as test subject? Repeat that a few times to understand the repercussions. If I were the customers and I found out that I was used as test subject, how would I feel?

Canary deployments are already an industry accepted practice and it’s shocking Crowdstrike apparently doesn’t do them.

Which industry? Cybersecurity or Cloud software?

Re: No More Blue Fridays

#72
post #2

> In the future, computers will not crash due to bad software updates, even those updates that involve kernel code. In the future, these updates will push eBPF code. Assuming every security critical system will be on a recent enough kernel to support this...

Considering the number of systems running very obsolete OSes these days: WinNT (4x or 3x), Windows, DOS, or various proprietary Unixen, stale Linux flavours, etc., etc., ... yes, quite.

Re: No More Blue Fridays

#73
post #31

> Once Microsoft's eBPF support for Windows becomes production-ready, Windows security software can be ported to eBPF as well. This doesn’t seem grounded in reality. If you follow the link to the “hooks” that Windows eBPF makes available [1], it’s just for incoming packets and socket operations. IOW, MS is expecting you to use the Berkeley Packet Filter for packet filtering. Not for filtering I/O, or object creation/…

Yes, we know eBPF must attach to equivalent events to Linux, but given there are already many event sources and consumers in Windows, the work is to make eBPF another consumer -- not to invent instrumentation frameworks from scratch. Just to use an analogy: Imagine people do their banking on JavaScript websites with Google Chrome, but if they use Microsoft Edge it says "JavaScript isn't supported, please download and…

This assumes eBPF becomes the standard. It's not clear Microsoft wants that. They could create something else which integrates with dot net and push for that instead.

Also this problem of too much software running in the kernel in an unbounded manner has long existed. Why should Microsoft suddenly invest in solving it on Windows?

Re: No More Blue Fridays

#74
post #31

> Once Microsoft's eBPF support for Windows becomes production-ready, Windows security software can be ported to eBPF as well. This doesn’t seem grounded in reality. If you follow the link to the “hooks” that Windows eBPF makes available [1], it’s just for incoming packets and socket operations. IOW, MS is expecting you to use the Berkeley Packet Filter for packet filtering. Not for filtering I/O, or object creation/…

I hope though that Microsoft will double down on their eBPF support for Windows after this incident.

Doubt it. Microsoft is clearly over Windows. They continue to produce it but every release feels like "Ugh, fine, since you are paying me a ton of money."

Internally, Microsoft is running more and more workloads on Linux and externally, I've had .Net team tell me more than once that Linux is preferred environment for .Net. SQL Server team continues to push hard for Linux compatibility with every release.

EDIT: Windows Desktop gets more love because they clearly see that as important market. I'm talking more Windows Server.

Re: No More Blue Fridays

#75

Earlier quoted context omitted.

Well it is useful in practice, there are some pretty useful products based on eBPF on Linux, most notably Cilium (and, shameless plug for the one I’m working on: Parca, an eBPF-based CPU profiler).

Bad wording on my part, and I still don't know how to word it better. I'm sure this thing is useful, I don't think everyone who contributed code was just clueless. However, the claim "in the future, computers will not crash due to bad software updates, even those updates that involve kernel code" must be false. There is no way it is true. Whatever Cilium is, I cannot believe it generally prevents kernel crashes.

The claim isn't that eBPF generally prevents kernel crashes. It's that it prevents crashes in the subset of programs it's designed for, in particular for instrumentation, which Crowdstrike is (in this author's conception) an instance of.

Re: No More Blue Fridays

#76

Earlier quoted context omitted.

They could have helped their luck by doing some of the common sense things suggested in the article. For instance, why not find a subset of your customers that are low risk, push it out to them, and see what happens? Or perhaps have your own fleet of example installations to run things on first. None of which depends on any specific technology.

"find a subset of low risk customers" and use them as test subject? Repeat that a few times to understand the repercussions. If I were the customers and I found out that I was used as test subject, how would I feel?

> If I were the customers and I found out that I was used as test subject, how would I feel?

In reality, every business has relationships that it values more than others. If I wasn't paying a lot for it, and if I was running something that wasn't critical (like my side project) then why not? You can price according to what level of service you want to provide.

Re: No More Blue Fridays

#77

> an unprecedented example of the inherent dangers of kernel programming I take issue with that. Kernel programming was not to blame; looking up addresses from a file and accessing those memory locations without any validation is. The same technique would yield the same result at any Ring.

At Ring 3 it would crash an app, not the entire OS.

Yes, the kernel is fine and is not to blame. But running basically a rootkit controlled by a third party indeed is to blame.

Re: No More Blue Fridays

#78
post #60

Can someone tell me what's the advantage of eBPF over a user mode driver? The article makes it look it eBPF is have your cake and eat it too solution which is too good to be true? Can you run graphics drivers in eBPF for example?

No, you can't run arbitrary general-purpose programs in eBPF, and you cannot run graphics drivers in it. You generally can't run programs with unprovably bounded loops in eBPF, and your program can interact with the kernel only through a small series of explicitly enumerated "helpers" (for any given type of eBPF program, you probably have about 20 of these in total).

Re: No More Blue Fridays

#80
> If your company is paying for commercial software that includes kernel drivers or kernel modules, you can make eBPF a requirement. It's possible for Linux today, and Windows soon. While some vendors have already proactively adopted eBPF (thank you), others might need a little encouragement from their paying customers.

How about Microsoft's large government and commercial customers make it a requirement that MS does not develop a single new feature for the next two fucking years or however long it takes to go through the entirety of the Windows+Office+Exchange code base and to make sure there are no security issues in there?

We don't need ads in the start menu, we don't need telemetry, we don't need desktop Outlook becoming a rotten slow and useless web app, we don't need AI, we certainly don't need Recall. We need an OS environment that doesn't need a Patch Tuesday where we have to check if the update doesn't break half the canary machines.

And while MS is at that they can also take the goddamn time and rework the entire configuration stack. I swear to god, it drives me nuts. There's stuff that's only accessible via the registry (and there is no comprehensive documentation showing exactly what any key in the registry can do - large parts of that are MS-internal!), there's stuff only accessible via GPO, there's stuff hidden in CPLs dating back to Windows 3.11, and there's stuff in Windows' newest UI/settings framework.

Post reply on HN