Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

penberg.org

Post reply on HN
Print view View original

101–110 of 291 posts

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#101

Post by bluetomcat »

thaumaturgy wrote: post #88
bluetomcat wrote: post #70

Earlier quoted context omitted.

What if we surpass file systems completely? Imagine every process having its own "non-volatile" memory area, where all persistent state is kept between restarts, with a clean OS interface to share that state to other processes? Terminals and process hierarchy are a complete no-go in a fresh design. Every entity in the system can be identified through unique IDs which can be handed down to other processes based on dif…

> Imagine every process having its own "non-volatile" memory area, where all persistent state is kept between restarts... I'll treat this seriously: as software development practices are just now beginning to mature, with more emphasis on test coverage being considered best practice, sure ... maybe. But there's still a lot of software out there for which "restart the application" (or even, "restart the stinking OS")…

> But there's still a lot of software out there for which "restart the application" (or even, "restart the stinking OS") is the only practical solution when it's behaving badly.

Which, in principle, doesn't prevent a program from crashing or misbehaving when it encounters a corrupted file left over from its last run. In the imagined system, the application developer would be fully aware whether he is putting a data structure in the "volatile" or in the "non-volatile" memory areas, with some safety guarantees from the OS. Restarting would zero only the volatile area, enabling a clean starting state.

> Okay, but how do you write a process which is capable of communicating with any other process?

Why would you need to communicate with any process? Maybe I need to communicate with the currently running instance of "HTTP Server App" or "Database Engine App", not with an unrelated process my program knows nothing about.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#102

Post by [deleted] »

thaumaturgy wrote: post #82

The paper reads like it's suggesting moving the burden of complexity in dealing with varying hardware interfaces from the kernel to userland so that userland can take direct advantage of higher performance hardware when it's available. I could see that for some very small niches, but in general I think it would be a terrible development for the industry. Hardware vendors don't like to share. They don't share code, th…

[deleted]

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#103

Post by robbyt »

thaumaturgy wrote: post #82

The paper reads like it's suggesting moving the burden of complexity in dealing with varying hardware interfaces from the kernel to userland so that userland can take direct advantage of higher performance hardware when it's available. I could see that for some very small niches, but in general I think it would be a terrible development for the industry. Hardware vendors don't like to share. They don't share code, th…

As I was reading this, I remembered the days of my youth setting the IRQ and DMA address for my soundblaster (compatible) soundcard.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#105

Post by olliej »

bluetomcat wrote: post #42

We need entirely new OS abstractions to replace the dated notions of hierarchical file systems built around the metaphor of file cabinets, I/O as streams of bytes, terminals, process hierarchy. Essentially, say goodbye to the Unix model after 50 years. It would open up an entirely new world of software experimentation and craftsmanship.

...except humans instinctively organize things by hierarchies (even something as counting is done hierarchically).

Just saying “we need to change this” without saying what the short comings you want to address isn’t a super useful statement. Multiple platforms have attempted to have the user interface to their data be tag based, but that simply doesn’t scale for the amount of data people can manage with hierarchies.

Finally what the heck are you talking about in that last sentence: how does changing representation result in a new age of experimentation and (???) craftsmanship? Why is craftsmanship gated on the user level abstraction to bytes? Experimentation already happens today, what does this change?

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#106

Post by robbyt »

bluetomcat wrote: post #42

We need entirely new OS abstractions to replace the dated notions of hierarchical file systems built around the metaphor of file cabinets, I/O as streams of bytes, terminals, process hierarchy. Essentially, say goodbye to the Unix model after 50 years. It would open up an entirely new world of software experimentation and craftsmanship.

Correct me if I'm wrong, but isn't this what Plan 9 does?

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#107

Post by vetinari »

racuna wrote: post #43

On Exadata architecture (and the recent versions of Oracle Database) the search is made by the hardware, not the software. I'm not a fan of Oracle, but things like that are awesome.

Really? What I vaguely remember that the storage nodes were embedded linux boxes. Yes, they understood indices and would return only minimum needed, but it was still software.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#108

Post by jandrewrogers »

shereadsthenews wrote: post #33
jandrewrogers wrote: post #12

At least in database kernels, we noticeably reached this threshold around five years ago with typical server hardware. This is an interesting computer science problem in that virtually all of our database literature is based on the presumption that I/O is much slower than CPU. If you cleanroom a database kernel design based on the assumption that I/O performance is not the bottleneck, you end up with an architecture…

I don't know ... we used to interleave our hard disk formats because a hard disk could stream data faster than an i386 CPU could ingest it, and there was plenty of database research done prior to 1990.

Database algorithms are typically designed for worst case storage throughput i.e. the bandwidth floor, which for hard disks is not sequential access patterns. In the spinning hard disk days you assumed a few MB/s per disk, because you would often see that in real systems. Even with a disk array, it usually wasn't enough to keep the CPU busy if real-world access patterns kept disk throughput near the floor.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#109

Post by loeg »

blihp wrote: post #76
aloer wrote: post #21

Earlier quoted context omitted.

What was the situation like before SSDs? Were the fastest hard drives at the time (15k rpm?) faster than the fastest CPUs?

Individually no, collectively (i.e. storage arrays) yes.

Sort of. Intel's server parts top out at on the order of 48 PCIe 3 lanes (Skylake-SP). Those parts have 6-channel DDR4 memory (rated to DDR4-2666), giving a theoretical DRAM throughput limit of about 128 GB/s.

Meanwhile, a PCIe 3.0 lane has a theoretical throughput limit of about 985MB/s; 48 lanes makes 47 GB/s.

Both are theoretical numbers and in practice will be lower, and you're right that it doesn't give you a lot of free CPU to do any kind of processing on the data, but it seems like the CPU can keep up (as long as Intel continues to have an anemic number of PCIe lanes and until PCIe 4 comes along).

(FWIW, on a dual-socket Skylake-SP, Anandtech only achieved 122 GB/s (of theoretical 256 GB/s DRAM maximum). But Intel claims the dual-socket can reach 199 GB/s with an AVX-512 workload.[1] Scaling both of these numbers down by a factor of two gives a very crude estimate for single-socket DRAM bandwidth on a conventional workload (61 GB/s) and AVX-512 (99 GB/s).)

Meanwhile, AMD's Zen server parts have more than double the PCIe lanes but only a few more memory channels, so I expect you're correct there — huge/fast PCIe arrays could be bottlenecked on DRAM on that platform.

[1]: https://www.anandtech.com/show/11544/intel-skylake-ep-vs-amd...

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#110

Post by sp33der »

josephg wrote: post #99
thaumaturgy wrote: post #82

The paper reads like it's suggesting moving the burden of complexity in dealing with varying hardware interfaces from the kernel to userland so that userland can take direct advantage of higher performance hardware when it's available. I could see that for some very small niches, but in general I think it would be a terrible development for the industry. Hardware vendors don't like to share. They don't share code, th…

I don’t think anyone is suggesting that every application explicitly code for each network device. All of that tricky logic could be put into a userland library instead of the kernel. If we wanted, we could even replicate a kernel device driver style API in userland that network device drivers program against, done as a set of userland library files loaded dynamically based on detected hardware. The tricky part would…

Microkernels tend to do a lot of what you're talking about. They accomplish this via interrupts and virtual mapping of the device memory to userspace for the userland app to access.

"If we wanted, we could even replicate a kernel device driver style API in userland that network device drivers program against, done as a set of userland library files loaded dynamically based on detected hardware."

This is how many microkernel based embedded RTOS's work. Common API's written for a hardware device, say a NOR flash chip, and the driver for the hardware onboard implements the interface specific to that hardware. You can then dynamically link the driver specific to your hardware. That's what I was doing as an embedded C developer some 10 years ago, and that solution had already been implemented for at least a decade (or 2) before that.

Post reply on HN