Live data from Hacker News

The real realtime preemption end game

lwn.net

201–210 of 281 posts

Re: The real realtime preemption end game

#201
post #200
post #177

Earlier quoted context omitted.

> If you're not waiting for the remote log server to write the messages to its disk before proceeding, then it seems like that's not guaranteed to me? Depends on your failure model. I'd consider e.g. "received in memory by at least 3/5 remote servers in separate datacenters" to be safer than "committed to local disk".

You're still on one side or another of the CAP triangle. In a network partition, you are either offline or your data is not consistent. If you're writing local to your system, you're losing data if there's a single device failure. https://en.wikipedia.org/wiki/CAP_theorem

For logs, which are immutable time series journals, any copy is entirely sufficient. The first write is a quorum. Also from a systems POV reads are not a feature of logs.

Re: The real realtime preemption end game

#202

Earlier quoted context omitted.

Capabilities are important, but I don’t think that was introduced by seL4. Mach (which underlies macOS) has the same capability-based system.

I didn't say seL4 introduced capabilities. However, to my knowledge, seL4 was the first kernel to show that pervasive usage of capabilities is both feasible and beneficial.

There's quite a history of capabilities-based research OS's that culminated in, but did not start with L4 (of which seL4 is a later variant).

Re: The real realtime preemption end game

#203

What a blast from the past. I compiled a kernel for Debian with RT_PREEMPT about 17-18 years ago to use with scientific equipment that needed tighter timings. I was very impressed at the latencies and jitter. I haven’t really thought about it since then, but I can imagine lots of used cases for something like an embedded application with raspberry pi where you don’t quite want to make the leap into a microcontroller…

Interesting to mention the Raspberry Pi. I saw an article just a day or two ago that claimed that the RpiOS was stated by and ran on top of RTOS. That's particularly interesting because at one time years ago, I saw suggestions that Linux could run as a task on an RTOS. Things that required hard real time deadlines could run on the RTOS and not be subject to the delays that a virtual memory system could entail.

I don't recall if this was just an idea or was actually implemented. I also have seen only the one mention of RpiOS on an RTOS so I'm curious about that.

Re: The real realtime preemption end game

#204

Earlier quoted context omitted.

I didn't say seL4 introduced capabilities. However, to my knowledge, seL4 was the first kernel to show that pervasive usage of capabilities is both feasible and beneficial.

There's quite a history of capabilities-based research OS's that culminated in, but did not start with L4 (of which seL4 is a later variant).

Yes, but I believe seL4 took it to the max. I may be wrong on that count, but I think seL4 is unique in that it leverages capabilities for pretty much everything except the scheduler. (There was work in that area, but it's incomplete.)

Re: The real realtime preemption end game

#205

Earlier quoted context omitted.

If your logging service is down all bets are off. But by buffering logs you're now accepting that problems not related to the logging service will also cause you to drop logs - as I mentioned, your service crashing, or being OOM'd, would be one example.

What's more likely? An intermittent network issue, the logging service being momentarily down, or a local crash that only affects your buffering queue? If an OOM happens, all bets are off anyway, since it has as much likelihood of taking out your application as it does your buffering code. The local buffering code might very well be part of the application in the first place, so the fate of the buffering code is the…

> It seems you're trying very hard to contrive a situation where doing nothing is better than taking reasonable steps to counter occasional network hiccups.

I think you've completely misunderstood me then. I haven't taken a stance at all on what should be done. I'm only trying to agree with the grandparent poster about logging ultimately reflecting CAP Theorem.

Re: The real realtime preemption end game

#206

Earlier quoted context omitted.

I didn't say seL4 introduced capabilities. However, to my knowledge, seL4 was the first kernel to show that pervasive usage of capabilities is both feasible and beneficial.

The other L4s before it showed that caps are useful and can be implemented efficiently.

https://dl.acm.org/doi/pdf/10.1145/2517349.2522720

" We took a substantially different approach with seL4; its model for managing kernel memory is seL4’s main contribution to OS design. Motivated by the desire to reason about resource usage and isolation, we subject all kernel memory to authority conveyed by capabili- ties (except for the fixed amount used by the kernel to boot up, including its strictly bounded stack). "

I guess I should've said seL4 took capabilities to the extreme.

Re: The real realtime preemption end game

#207

QNX had this right decades ago. The microkernel has upper bounds on everything it does. There are only a few tens of thousands of lines of microkernel code. All the microkernel does is allocate memory, dispatch the CPU, and pass messages between processes. Everything else, including drivers and loggers, is in user space and can be preempted by higher priority threads. The QNX kernel doesn't do anything with strings.…

QNX is used in vehicle infotainment systems no? Where else? I'm not bothered by the kernel bloat. There's a lot of dev time being invested in Linux and while the desktop is not as much of a priority as say the server space a performant kernel on handhelds and other such devices and the dev work to get it there will benefit the desktop users like myself.

Railroads/Positive Train Control, emergency call centers, etc. QNX is used all over the place. If you want an even more impressive Microkernel RTOS, then Green Hills INTEGRITY is a great example. It's the RTOS behind the B-2, F-{16,22,35}, Boeing 787, Airbus A380, Sikorsky S-92, etc.

Re: The real realtime preemption end game

#208
post #66

Earlier quoted context omitted.

I don't think realtime anything has much to do with mutex deadlocks, those are pretty much orthogonal concepts. In fact, I would make a stronger claim: if your "realtime" system can deadlock, it's either not really realtime or it has a design flaw and should be sent back to the drawing board. It's not like you can say "oh, we have a realtime kernel now, so deadlocks are the kernel's problem". Actual realtime systems…

I had an introductory course on OS and learned about hard real-time systems. I had impression hard real-time is about memory, deadlocks, livelocks, starvation, and so on. And in general about how to design system that moves forward even in presence of serious bugs and unplanned-for circumstances.

Bugs related to concurrency - which is where you get race conditions and deadlocks - tend to pop up wherever there's an implied sequence of dependencies to complete the computation, and the sequence is determined dynamically by an algorithm.

For example, if I have a video game where there's collision against the walls, I can understand this as potentially colliding against "multiple things simultaneously", since I'm likely to describe the scene as a composite of bounding boxes, polygons, etc.

But to get an answer for what to do in response when I contact a wall, I have to come up with an algorithm that tests all the relevant shapes or volumes.

The concurrency bug that appears when doing this in a naive way is that I test one, give an answer to that, then modify the answer when testing the others. That can lead to losing information and "popping through" a wall. And the direction in which I pop through depends on which one is tested first.

The conventional gamedev solution to that is to define down the solution set so that it no longer matters which order I test the walls in: with axis aligned boxes, I can say "move only the X axis first, then move only the Y axis". Now there is a fixed order, and a built-in bias to favor one or the other axis. But this is enough for the gameplay of your average platforming game.

The generalization on that is to describe it as a constraint optimization problem: there are some number of potential solutions, and they can be ranked relative to the "unimpeded movement" heuristic, which is usually desirable when clipping around walls. That solution set is then filtered down through the collision tests, and the top ranked one becomes the answer for that timestep.

Problems of this nature come up with resource allocation, scheduling, etc. Some kind of coordinating mechanism is needed, and OS kernels tend to shoulder a lot of the burden for this.

It's different from real-time in that real-time is a specification of what kind of performance constraint you are solving for, vs allowing any kind of performance outcome that returns acceptable concurrent answers.

Re: The real realtime preemption end game

#209

Earlier quoted context omitted.

There's quite a history of capabilities-based research OS's that culminated in, but did not start with L4 (of which seL4 is a later variant).

Yes, but I believe seL4 took it to the max. I may be wrong on that count, but I think seL4 is unique in that it leverages capabilities for pretty much everything except the scheduler. (There was work in that area, but it's incomplete.)

L4 was developed in the 90's. Operating Systems like Amoeba, which were fundamentally capability-based to a degree that even exceeds L4, were a hot research topic in the 80's.

L4's contribution was speed. It was assumed that microkernels, and especially capability-based microkernels were fundamentally slower than monolithic kernels. This is why Linux (1991) is monolithic. Yet L4 (1994) was the fastest operating system in existence at the time, despite being a microkernel and capability based. It's too bad those dates aren't reversed, or we might have had a fast, capability-based, microkernel Linux :(

Re: The real realtime preemption end game

#210
post #84

Earlier quoted context omitted.

This kind of makes the same point I made though -- apps without hard realtime requirements aren't "really realtime" applications

No -- soft realtime applications are things like video conferencing, where you care mostly about low latency in the audio/video stream but it's ok to drop the occasional frame. These are still realtime requirements, different from what your typical browser does (for example): who cares if a webpage is rendered in 100ms or 2s? Hard realtime is more like professional audio/video recording where you want hard guarantees…

> who cares if a webpage is rendered in 100ms or 2s?

Do you really stand by the statement of this rhetorical question? Because if yes: this attitude is a big reason for why web apps are so unpleasant to work with compared to locally running applications. Depending on the application, even 16ms vs 32ms can make a big difference.

Post reply on HN