141–150 of 291 posts
I apologize for not reading much of it yet, but could someone give a quick comparison to the exokernel idea?
At least in database kernels, we noticeably reached this threshold around five years ago with typical server hardware. This is an interesting computer science problem in that virtually all of our database literature is based on the presumption that I/O is much slower than CPU. If you cleanroom a database kernel design based on the assumption that I/O performance is not the bottleneck, you end up with an architecture…
Are you aware of any efforts by existing databaes or entirely new databases built with this realization in mind?
Earlier quoted context omitted.
I would like to see an updated file system architecture that's closer to a database, with tagging and all that. And I could see that extending to processes too. But what are you imagining as alternatives for i/o and terminals?
Wasn't this kind of filesystem an abandoned plan for one of the (now old) versions of Windows? Like Windows 2000 or XP? I don't remember why they ditched this plan. Maybe it wasn't such a good idea after all, who knows.
I think there are a few reasons it hasn't happened:
1. Filesystems are hard. Every new filesystem architecture ends up requiring a large pool of talented developers.
2. Nobody wants to break backwards compatibility. Current filesystem design is so integral to all kinds of software that you just can't expect all software in the world to be updated just to work with a new filesystem paradigm.
3. Databases are also hard, so a database-like filesystem is doubly so.
4. Current filesystem architecture is good enough for most stuff. The pain of continuing to use it isn't as great as the pain of changing it.
None of these reasons make a database-like filesystem inherently bad. It's just not practical right now.
I think we're moving in that direction though, with things like object storage.
Look to GPUs for better solutions: make more discrete -- less functional overall but higher performance -- cores, and move them closer to the data; then let the CPU just handle coordination of the discrete processors. Think of SIMD but on a massive scale. Think of blocks of RAM with math processors. Or the same in your NVMe/NIC/etc.
HP announced their ambition to do that a few years ago, framing it as a grand vision of the future of computing. I haven't heard anything since. This kind of offloading or distributed computing isn't quite a new idea, but it hasn't materialized yet. I suspect that it is too tough a nut to crack for the general case.
Is this true for most real life workloads? There's that famous rule-of-thumb indicator for latencies: https://www.prowesscorp.com/computer-latency-at-a-human-scal... It doesn't seem to be that the orders of magnitude are so close as to require totally rethinking mainstream kernels. Or am I looking at this the wrong way?
> Is this true for most real life workloads? No. It's true if you care about NVMe drives, or high-speed networking--which is to say, it's true if you care about a few kinds of server workloads, but it's absolutely not true for most consumer hardware.
Earlier quoted context omitted.
> But there's still a lot of software out there for which "restart the application" (or even, "restart the stinking OS") is the only practical solution when it's behaving badly. Which, in principle, doesn't prevent a program from crashing or misbehaving when it encounters a corrupted file left over from its last run. In the imagined system, the application developer would be fully aware whether he is putting a data s…
> Why would you need to communicate with any process? Maybe I need to communicate with the currently running instance of "HTTP Server App" or "Database Engine App", not with an unrelated process my program knows nothing about. Today, I piped the output of some process I was running into grep. Neither of these programs knew about the existence of the other. Later, I used an image editor to create an image, then used a…
Only because "an unstructured stream of bytes" is the way for Unix processes to communicate. It allows for composability, but a very fragile and unsafe one, requiring programs to spit out and read free text, with all the crazy filtering and guesswork needed.
The paper reads like it's suggesting moving the burden of complexity in dealing with varying hardware interfaces from the kernel to userland so that userland can take direct advantage of higher performance hardware when it's available. I could see that for some very small niches, but in general I think it would be a terrible development for the industry. Hardware vendors don't like to share. They don't share code, th…
As I was reading this, I remembered the days of my youth setting the IRQ and DMA address for my soundblaster (compatible) soundcard.
The paper reads like it's suggesting moving the burden of complexity in dealing with varying hardware interfaces from the kernel to userland so that userland can take direct advantage of higher performance hardware when it's available. I could see that for some very small niches, but in general I think it would be a terrible development for the industry. Hardware vendors don't like to share. They don't share code, th…
What you don't get however is sharing the disk between multiple processes.