Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

penberg.org

Post reply on HN
Print view View original

191–200 of 291 posts

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#191

Post by rhizome »

virgilp wrote: post #140
acdha wrote: post #116

Earlier quoted context omitted.

I never saw it portrayed that way: it was always higher-level programmer ease of use pitches along the lines of “isn’t this easier than doing a JOIN?” pitch shortly before the speaker had a valuable learning experience about the value of data integrity and consistency.

You really don't remember the "web scale" memes about Mongo? It was always primarily about "performance".

Performance was an imposed callsign of Mongo, the reason everybody started using it was the declarative schema stuff and so the dev didn't have to know how to start a terminal, which was a not-insignificant hurdle. It doesn't have significant performance differences over other simple KVs. "Faster" was primarily a measurement of learning curve.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#192

Post by gmueckl »

inetknght wrote: post #145
gmueckl wrote: post #134

Earlier quoted context omitted.

HP announced their ambition to do that a few years ago, framing it as a grand vision of the future of computing. I haven't heard anything since. This kind of offloading or distributed computing isn't quite a new idea, but it hasn't materialized yet. I suspect that it is too tough a nut to crack for the general case.

My understanding is that GPUs have moved toward what I described: thousands of discrete cores with large amounts of math performance but generally terrible (or even non-existent) branching...

And transferring data to and from the GPU is also a source of overhead.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#193

Post by kazinator »

Animats wrote: post #169
kazinator wrote: post #153

Earlier quoted context omitted.

> Channels have some privileged functions through which the OS tells them where the data is supposed to go in memory. So the channel controller system has DMA; just not the peripheral. > Minicomputers of the 1970s had low transistor counts and slow CPUs. So peripherals were usually put directly on the memory bus, with full access to memory. That's "bus mastering" DMA. There is such a thing as "third party" DMA, which…

Those are more like add-on features a driver can use if present. They don't push peripheral interfaces into a standard channel-like format. Nor are they close to one that can be exposed to application programs.

We can't expose a DMA controller directly to programs, but an API could be devised for it. Such a thing is needed even just from the mere portability point of view: making different DMA controllers on different systems look the same.

Applications will typically specify transfers using virtual addresses, but DMA controllers will understand physical memory (usually, but maybe there are virtually mapped ones out there). Plus there are other issues like restricted ranges: DMA that cannot acces all physical memory but just a certain range. The API will have to convert ranges of virtual addresses to one or more physical ranges, which are then queued as one or more DMA operations and somehow deal with the problem that not just any old memory allocation in the application is DMA-able.

Applications talking to channel hardware directly using dedicated CPU instructions is only possible in some vendor-locked IBM mainframe world. It's not otherwise feasible simply for business/market/economic reasons having nothing to do with technical feasibility.

An operating system API is basically a set of extensions to the instruction set available in the application's virtual machine; it's no different from some dedicated I/O instructions, just perhaps less efficient (which may or may not matter).

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#194

Post by kazinator »

dkersten wrote: post #187
Animats wrote: post #169

Earlier quoted context omitted.

Those are more like add-on features a driver can use if present. They don't push peripheral interfaces into a standard channel-like format. Nor are they close to one that can be exposed to application programs.

So would it be correct to say that functionally, it’s similar to DMA, but the API is different because “channels” expose a consistent interface while DMA doesn’t?

Channels only expose a consistent interface as long as there is no other mainframe other than an IBM one, or compatible.

Once you have two, and you want portable applications, using channel I/O instructions directly inline goes out the window; you need an API.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#195

Post by the8472 »

loeg wrote: post #189
the8472 wrote: post #137

Earlier quoted context omitted.

The means may be different, but the ends they aim for seem to be similar. The ends here are improved latency, throughput, parallelism, non-blocking APIs and security. The above-mentioned improvements aim to address the first four without completely bypassing the kernel, instead changing APIs so they step out of the way most of the time, limiting them to coordination tasks and then either offloading to the hardware or…

> It's basically the difference between a green field design and tacking all those innovations onto the glueball that is linux. The latter may be ugly and complex, but it has the advantage of being backwards-compatible. I'm definitely not trying to argue in favor of the paper's opinion. :-) I just believe the author of the article would disagree that the glueball your earlier remarks describe addresses the author's c…

Well, and all I am saying is that the paper, including the section you mention, don't take these latest developments into account. They're not kernel bypasses. They're driver level optimizations and traditional APIs operating on file descriptors and virtual memory aiming to solve similar problems.

This stuff is very new of course, so we have to wait for something to actually integrate all those pieces and then for benchmarks.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#196

Post by transpute »

Animats wrote: post #118

Mainframe designers had this problem under control by 1970. Mainframes had, and have, "channels". A channel is part of the processor architecture. It takes commands, sends them to a peripheral, and manages the data transfer in both directions. Channels have some privileged functions through which the OS tells them where the data is supposed to go in memory. The architecture of channels is well defined, and peripheral…

> So there's no standardization on how to talk to devices at the hardware level. Some CPUs have protection systems, an "I/O MMU", and there have been various channel-like interfaces, especially from Intel, but they have never caught on.

From the paper:

> NVMe SSDs perform I/O faster than the OS can accept new I/O requests and notify their completion. Furthermore, the current POSIX AIO implementations in Linux are ugly and adhoc, and they have limited support from file systems. This has lead to new AIO interfaces that eliminate system calls and leverage polling. However, polling for completion is not suitable for large I/O transfers.

AWS Nitro SR-IOV I/O virtualization [1] uses Intel VT-d Posted Interrupts [2] to avoid polling, but this CPU feature is price-segmented to Xeon E5 and higher. For other CPUs, polling [3] is necessary to achieve high IOPS with NVME. Intel and AMD should consider making this feature available on all CPUs, to support NVME and NVDIMM.

[1] SR-IOV: https://www.snia.org/sites/default/files/RonEmerick_PCI_Expr... & https://www.twosixlabs.com/running-thousands-of-kvm-guests-o...

[2] IOMMU: https://www.linux-kvm.org/images/7/70/2012-forum-nakajima_ap...

[3] Polling: https://events.static.linuxfound.org/sites/events/files/slid... & https://www.snia.org/sites/default/files/SDC/2018/presentati...

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#197

Post by laughinghan »

scroot wrote: post #176
Spivak wrote: post #162

Earlier quoted context omitted.

Okay so you have a device driver that exposes your underlying hard disk as a single contiguous segment of blocks of a given size. You have objects which are blobs of heterogeneously sized arbitrary data with an addressing scheme. The software that marries these two things is basically a filesystem driver (in that you can implement filesystem semantics on top of it -- hell Ceph does it right now). Nothing about a mode…

Of course, you need some agent that can communicate with the hard disk and that knows how to manage memory. Maybe its one driver, maybe its several, but these can also just be objects. Live objects, I mean. Not just blobs of data, but actors that have behaviors and are always "running." You don't even need the concept of files or filesystems. And if you do, better left to a higher level

Isn't the whole point of persistent storage like hard drives to deal with the fact that computers shut off sometimes, and nothing can be "always 'running'"?

Something below the object level (possibly part of the object system, possibly a layer below that) needs to read from disk and bootstrap all those agents/live objects/actors into existence.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#198

Post by Ericson2314 »

pjmlp wrote: post #72
pzs wrote: post #57

Earlier quoted context omitted.

Would you care to elaborate why Unix, in your opinion, was never a good design?

It was re-written on an unsafe language that completely disregarded what has happened outside Bell Labs since 1961, at Burroughs, IBM, MIT, Xerox and other research cathedrals. It only took off thanks to Bell Labs being forbidden to sell it, so it got offered for a symbolic price, alongside source code to major universities, which then decided to build on top, instead of paying OS street prices. Had UNIX been sold in…

Thank you for being a breadth of wisdom in the UNIX-worshiping wilderness.

You can always get mindshare being first massively underpriced thing to market.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#199

Post by [deleted] »

the_af wrote: post #135
dllthomas wrote: post #83

Earlier quoted context omitted.

There were always objections. "Unix went from being the worst operating system available, to being the best operating system available, without getting appreciably better." https://news.ycombinator.com/item?id=19416485 Which isn't to say that those POV were necessarily correct, just that it isn't all hindsight.

To be fair, there are always objections to any working piece of technology. It's not helpful to claim "it was never good design". It was a working design that drove technology to the point it is now, and in that sense it was hugely successful. What kind of perfect and pure tech do some people want, anyway? Pick anything, whatever they like -- say, Plan 9 or OS/2 or whatever -- and I can bet you in a parallel universe…

[deleted]

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#200

Post by Ericson2314 »

wongarsu wrote: post #139
Ericson2314 wrote: post #45

Earlier quoted context omitted.

Amen to that. Unix was never a good design, and now is severely out of date. We can no longer afford to hack around it.

Which is why we have Windows with the NT family of kernels. It's a reimagining that contains great ideas like a flexible file system stack that allows you to dynamically add drivers that add encryption, compression, antivirus checks etc, and a few bad ideas like a flexible file system stack that's orders of magnitude slower when working with small files than unix implementations. Windows is a mainstream OS that is ve…

The VMS people Microsoft poached and then straightjacketed with legacy concerns did have good ideas, yes.
Post reply on HN