Live data from Hacker News

No More Blue Fridays

brendangregg.com

101–110 of 282 posts

Re: No More Blue Fridays

#101
post #35
post #18

From the article: > If the verifier finds any unsafe code, the program is rejected and not executed. The verifier is rigorous -- the Linux implementation has over 20,000 lines of code [0] -- with contributions from industry (e.g., Meta, Isovalent, Google) and academia (e.g., Rutgers University, University of Washington). [0] links to https://github.com/torvalds/linux/blob/master/kernel/bpf/ver... which has this inter…

I'm glad to hear that Meta and Google code is "rigorous". I'd prefer INRIA, universities that fund theorem provers, industries where correctness matters like aerospace or semiconductors.

[flagged]

Re: No More Blue Fridays

#102
post #31

> Once Microsoft's eBPF support for Windows becomes production-ready, Windows security software can be ported to eBPF as well. This doesn’t seem grounded in reality. If you follow the link to the “hooks” that Windows eBPF makes available [1], it’s just for incoming packets and socket operations. IOW, MS is expecting you to use the Berkeley Packet Filter for packet filtering. Not for filtering I/O, or object creation/…

I hope though that Microsoft will double down on their eBPF support for Windows after this incident.

Keep in mind they don't just allow any old code to execute in the kernel.

They do have rigorous tests (WHQL), it's just Crowdstrike decided that was too burdensome for their frequent updates, and decided to inject code from config files (thus bypassing the control).

The fault here is entirely with Crowdstrike.

Re: No More Blue Fridays

#103

Earlier quoted context omitted.

Doubt it. Microsoft is clearly over Windows. They continue to produce it but every release feels like "Ugh, fine, since you are paying me a ton of money." Internally, Microsoft is running more and more workloads on Linux and externally, I've had .Net team tell me more than once that Linux is preferred environment for .Net. SQL Server team continues to push hard for Linux compatibility with every release. EDIT: Window…

They aren't over windows. They continue to be incredibly interested in and actively developing how much money they can suck from their users. Especially via various forms of ads. But yeah, kernel features are few and far between.

See also: https://en.wikipedia.org/wiki/Cash_cow

Re: No More Blue Fridays

#104
post #31

> Once Microsoft's eBPF support for Windows becomes production-ready, Windows security software can be ported to eBPF as well. This doesn’t seem grounded in reality. If you follow the link to the “hooks” that Windows eBPF makes available [1], it’s just for incoming packets and socket operations. IOW, MS is expecting you to use the Berkeley Packet Filter for packet filtering. Not for filtering I/O, or object creation/…

Microsoft already has an extensible file system filter capability in place, which is what current AV uses. Does it make sense to add eBPF on top of that and if so, are there any performance downsides, like we see with file system filters?

Re: No More Blue Fridays

#105

The implicit assumption of the article is that eBPF code can't crash a kernel, but the article itself eventually admits that it can and has done, including last month. eBPF is a safer way of providing kernel-extension functionality, for sure, but presenting it as the perfect solution is just asking to have your argument dismissed. eBPF is not perfect. And there's plenty of things it can't do. The very sandbox rules t…

It's casually claiming to have solved the halting problem, at least within some limited but useful context. That should be impossible, and it turns out, it is. I expect it can be solved within some limited contexts, but those contexts are not useful, at least not at the level of "generic kernel code".

eBPF started out as Berkeley Packet Filters. People wanted to be able to set up complex packet filters. Things like 'udp and src host 192.168.0.3 and udp[4:2]=0x0034 and udp[8:2]=0x0000 and udp[12]=0x01 and udp[18:2]=0x0001 and not src port 3956'

So BPF introduced a very limited bytecode, which is complex enough that it can express long filters with lots of and/or/brackets - but which is limited enough it's easy to check the program terminates and is crash-free. It's still quite limited - prior to ~2019, all loops had to be fully unrolled at compile time as the checker didn't support loops.

It turned out that, although limited, this worked pretty well for filtering packets - so later, when people wanted a way to filter all system calls they realised they could extend the battle-tested BPF system.

Nobody is claiming to have solved the halting problem.

Re: No More Blue Fridays

#106

Earlier quoted context omitted.

This assumes eBPF becomes the standard. It's not clear Microsoft wants that. They could create something else which integrates with dot net and push for that instead. Also this problem of too much software running in the kernel in an unbounded manner has long existed. Why should Microsoft suddenly invest in solving it on Windows?

Apple took the lead on this front. It has closed easy access to the kernel by apps, and made a list of APIs to try and replace the lost functionality. Anyone maintaining a kernel module on macOS is stuck in the past. Of course, the target area of macOS is much smaller than Windows, but it is absolutely possible to kick all code, malware and parasitic security services alike, from accessing the kernel. The safest kern…

I don't think Microsoft has a choice with regards to kernel access. Hell, individuals currently use undocumented NT APIs. I can't imagine what happens to backwards compat if kernel access is closed.

Apple's closed ecosystem is entirely different. They'll change architectures on a whim and users will go with the flow (myself included).

Re: No More Blue Fridays

#107
post #57

Earlier quoted context omitted.

The medical machine software should just refuse to run with an error message if a critical driver was not loaded. The OS bricking is causing way more trouble where an IT technician now needs to fix something where it otherwise would just be updating the faulty driver... Also does your car not start if you are missing water for the wiper?

Water for the wiper is userland feature. 3rd party hooking into kernel is 3rd party responsibility. It is like equipping your car with LPG - THAT hooks into engine (kernel). And When I had a faulty gas pressure sensor then my car actually halted (BSOD if you will) instead of automatically failing over to gasoline as it is by design. You can argue that car had no means to continue execution but kernel has, however inv…

Initially I was inclined to disagree ("these things should always fail safe") however with more and more stuff being pushed into the kernel it's hard to say that you're wrong or exactly where a line needs to be drawn between "minimally functional system" and "dangerously out of control system".

I think until we discover a technology that forces commercial software vendors to employ functioning QA departments none of this will really solve anything.

Re: No More Blue Fridays

#108

Earlier quoted context omitted.

Doubt it. Microsoft is clearly over Windows. They continue to produce it but every release feels like "Ugh, fine, since you are paying me a ton of money." Internally, Microsoft is running more and more workloads on Linux and externally, I've had .Net team tell me more than once that Linux is preferred environment for .Net. SQL Server team continues to push hard for Linux compatibility with every release. EDIT: Window…

This claim about SQL Server: Is it due to disk access being slower from NT kernel compared to Linux kernel?

It's just easier for everyone involved (outside Windows GUI clicker admins) if it runs on Linux. Containerization is easier, configuration is easier and operating system is much more robust.

Re: No More Blue Fridays

#109
post #7
post #2

> In the future, computers will not crash due to bad software updates, even those updates that involve kernel code. In the future, these updates will push eBPF code. Assuming every security critical system will be on a recent enough kernel to support this...

I think with a LTS distribution you should get very far these days when it comes to implementing such sensors.

On rhel8 variants, you can use the Oracle UEK to get eBPF.

https://blogs.oracle.com/linux/post/oracle-linux-and-bpf

  $ cat /etc/redhat-release /etc/oracle-release /proc/version
  Red Hat Enterprise Linux release 8.10 (Ootpa)
  Oracle Linux Server release 8.10
  Linux version 5.15.0-203.146.5.1.el8uek.x86_64 (mockbuild@host-100-100-224-48) (gcc (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9.2.0.1), GNU ld version 2.36.1-4.0.1.el8_6) #2 SMP Thu Feb 8 17:14:39 PST 2024

Re: No More Blue Fridays

#110
post #79

I don't do any kernel stuff so I'm out of my element, but doesn't the fact that Crowdstrike & Linux kernel eBPF already caused kernel crashes[1] sort of downplay the rosiness of the state of things? [1]: https://access.redhat.com/solutions/7068083

This is specifically addressed in the post you are replying to
Post reply on HN