Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No More Blue Fridays

brendangregg.com

Post reply on HN
Print view View original

91–100 of 282 posts

Re: No More Blue Fridays

#91

Post by blinkingled »

Archelaos wrote: post #37
blinkingled wrote: post #30

Ok. But the good old push code to staging / canary it before mainstream updates was a simpler way of solving the same problem. Crowdstrike knows the computers they're running on, it is trivial to implement a system where only few designated computers download and install the update and report metrics before the update controller decides to push it to next set.

It would mitigate the problem, but not solve it. You can still imagine a condition that only occurs after the update has been rolled out everywhere. Furthermore, such a bug would still be extremely problematic for the concerned customers, even if not all of them were affected. In addition, it would be necessary to react very quickly in the case of zero-day vulnerabilities.

Yes, I am not arguing against having the ability to deal with it quickly - I am saying canary/ staging helps you do exactly that. Because as we see in the case of Intel CPUs and Crowdstrike some problems or scale of some problems is best prevented.

Re: No More Blue Fridays

#92

Post by nequo »

worthless-trash wrote: post #32
efee22 wrote: post #14

Earlier quoted context omitted.

RHEL kernel.. right. Imho, I'd trust an upstream stable kernel far more than a RHEL one for production which has dozen of feature backports and an internal kABI to maintain.. granted RH has a QA team, but it is still impossible to test everything beforehand.

On the upside, non root users can't insert ebpf code, so its a priv'ed operation, not like other distros.

Isn’t it tied to CAP_BPF on every distro since the 5.8 kernel?

https://mdaverde.com/posts/cap-bpf/

Re: No More Blue Fridays

#93

Post by red_admiral »

shrx wrote: post #18

From the article: > If the verifier finds any unsafe code, the program is rejected and not executed. The verifier is rigorous -- the Linux implementation has over 20,000 lines of code [0] -- with contributions from industry (e.g., Meta, Isovalent, Google) and academia (e.g., Rutgers University, University of Washington). [0] links to https://github.com/torvalds/linux/blob/master/kernel/bpf/ver... which has this inter…

eBPF is not Turing-complete, I suppose.

Re: No More Blue Fridays

#94

Post by throwaway2037 »

stackskipton wrote: post #74
shahahqq wrote: post #38

Earlier quoted context omitted.

I hope though that Microsoft will double down on their eBPF support for Windows after this incident.

Doubt it. Microsoft is clearly over Windows. They continue to produce it but every release feels like "Ugh, fine, since you are paying me a ton of money." Internally, Microsoft is running more and more workloads on Linux and externally, I've had .Net team tell me more than once that Linux is preferred environment for .Net. SQL Server team continues to push hard for Linux compatibility with every release. EDIT: Window…

This claim about SQL Server: Is it due to disk access being slower from NT kernel compared to Linux kernel?

Re: No More Blue Fridays

#95

Post by __MatrixMan__ »

Maybe we should start taking Fridays off to commemorate the event, which probably would have been less bad if more people spent less time with their nose to the grindstone and had more time to stop and think about how it all was shaping up and how they could influence that shape.

Re: No More Blue Fridays

#96

Post by [deleted] »

dathinab wrote: post #82
shrx wrote: post #18

From the article: > If the verifier finds any unsafe code, the program is rejected and not executed. The verifier is rigorous -- the Linux implementation has over 20,000 lines of code [0] -- with contributions from industry (e.g., Meta, Isovalent, Google) and academia (e.g., Rutgers University, University of Washington). [0] links to https://github.com/torvalds/linux/blob/master/kernel/bpf/ver... which has this inter…

the halting problem is only true for _arbitrary_ programs but there are always sets of programs for which it is clearly possible to guarantee their termination e.g. the program `return 1+1;` is guaranteed to halt e.g. given program like `while condition(&mut state) { ... }` with where `condition()` is guaranteed to halt but otherwise unknown is not guaranteed to halt, but if you turn it into `for _ in 0..1000 { if !c…

[deleted]

Re: No More Blue Fridays

#97

Post by red_admiral »

lucianbr wrote: post #50
skywhopper wrote: post #21

The implicit assumption of the article is that eBPF code can't crash a kernel, but the article itself eventually admits that it can and has done, including last month. eBPF is a safer way of providing kernel-extension functionality, for sure, but presenting it as the perfect solution is just asking to have your argument dismissed. eBPF is not perfect. And there's plenty of things it can't do. The very sandbox rules t…

It's casually claiming to have solved the halting problem, at least within some limited but useful context. That should be impossible, and it turns out, it is. I expect it can be solved within some limited contexts, but those contexts are not useful, at least not at the level of "generic kernel code".

It solves the halting problem by not being Turing complete. I presume each eBPF runs in a context with bounded memory, requested up front, for one thing; it also disallows jumps unless you can prove the code still halts.

Re: No More Blue Fridays

#98

Post by School-Cotton »

lucianbr wrote: post #89
School-Cotton wrote: post #68

Earlier quoted context omitted.

Correct, you will never be able to write any possible arbitrary code and have it run in eBPF. It necessarily constrains the class of programs you can write. But the constrained set is still quite useful and probably includes the crowdstrike agent. Also, although this isn't the case now, it's possible to imagine that the verifier could be relaxed to allow a Turing-complete subset of C that supports infinite loops whil…

If the verifier allowed a Turing-complete language, it would solve the halting probem, which is impossible.

My point is that the verifier could be relaxed to accept programs that never halt, thus not needing to solve the halting problem. You could then have the kernel just kill it after running over a certain maximum amount of time.

Re: No More Blue Fridays

#99

Post by philistine »

surajrmal wrote: post #73
brendangregg wrote: post #46

Earlier quoted context omitted.

Yes, we know eBPF must attach to equivalent events to Linux, but given there are already many event sources and consumers in Windows, the work is to make eBPF another consumer -- not to invent instrumentation frameworks from scratch. Just to use an analogy: Imagine people do their banking on JavaScript websites with Google Chrome, but if they use Microsoft Edge it says "JavaScript isn't supported, please download and…

This assumes eBPF becomes the standard. It's not clear Microsoft wants that. They could create something else which integrates with dot net and push for that instead. Also this problem of too much software running in the kernel in an unbounded manner has long existed. Why should Microsoft suddenly invest in solving it on Windows?

Apple took the lead on this front. It has closed easy access to the kernel by apps, and made a list of APIs to try and replace the lost functionality. Anyone maintaining a kernel module on macOS is stuck in the past.

Of course, the target area of macOS is much smaller than Windows, but it is absolutely possible to kick all code, malware and parasitic security services alike, from accessing the kernel.

The safest kernel is the one that cannot be touched at runtime.

Re: No More Blue Fridays

#100

Post by Xunjin »

philistine wrote: post #99
surajrmal wrote: post #73

Earlier quoted context omitted.

This assumes eBPF becomes the standard. It's not clear Microsoft wants that. They could create something else which integrates with dot net and push for that instead. Also this problem of too much software running in the kernel in an unbounded manner has long existed. Why should Microsoft suddenly invest in solving it on Windows?

Apple took the lead on this front. It has closed easy access to the kernel by apps, and made a list of APIs to try and replace the lost functionality. Anyone maintaining a kernel module on macOS is stuck in the past. Of course, the target area of macOS is much smaller than Windows, but it is absolutely possible to kick all code, malware and parasitic security services alike, from accessing the kernel. The safest kern…

> The safest kernel is the one that cannot be touched at runtime.

Can you expand what you mean here? Because depending on the application you are running, you will need at least talk with some APIs to get privileged access?

Post reply on HN