I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf] • Hacker News Board

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#171

Post by stormbeard » Fri, May 03, 2019, 7:08 PM UTC

MagicPropmaker wrote: post #120

pjc50 wrote: post #11

Earlier quoted context omitted.

Is there a good concise explanation of the 360 architecture available on the web? It seems historically important but I've not seen such a thing.

Why, yes there is! The IBM 360 Principles of Operation. When I got my C.S. degree we spent a whole semester studying this. http://bitsavers.trailing-edge.com/pdf/ibm/360/princOps/A22-...

This is a treasure trove of information; however, it's not what I would call concise. Did you run across anything else to supplement this during your program?

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#172

Post by baybal2 » Fri, May 03, 2019, 7:14 PM UTC

saltcured wrote: post #165

thaumaturgy wrote: post #82

The paper reads like it's suggesting moving the burden of complexity in dealing with varying hardware interfaces from the kernel to userland so that userland can take direct advantage of higher performance hardware when it's available. I could see that for some very small niches, but in general I think it would be a terrible development for the industry. Hardware vendors don't like to share. They don't share code, th…

There is a constant dance at the fringe of high performance systems. It leads to a recurring pattern of "revolutionizing" with some kind of bypass or coprocessor architecture, then eventually reverting to traditional structures as the new performance realities reach commodity levels. Part of it is the economics at the fringe can pursue speed at any cost. And part is the heady appeal of doing things differently for re…

> There is a constant dance at the fringe of high performance systems. It leads to a recurring pattern of "revolutionizing" with some kind of bypass or coprocessor architecture, then eventually reverting to traditional structures as the new performance realities reach commodity levels.

Truly this is what DC hardware been about for 20+ years.

Look at HPC space, and than look for "uncle Joe hosting." The moment Joe starts looking for HPC and invest heavily for some super duper RDMA powered disk, a mainstream hardware maker comes and spoils everything by releasing a mass market product doing just that if not better, thus ruining the grand scheme of gaining competitive advantage through some "secret sauce hardware"

Look at video transcoding or that new "AI" neural network thing. After few years of entertaining themselves with idea of purpose made hardware, big boys simply went the route of just using a lot of mainstream hardware.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#173

Post by PeterCorless » Fri, May 03, 2019, 7:15 PM UTC

Jach wrote: post #96

owaislone wrote: post #22

Earlier quoted context omitted.

Are you aware of any efforts by existing databaes or entirely new databases built with this realization in mind?

ScyllaDB is probably in the right direction. It at least bypasses the kernel for networking (uses Intel's DPDK).

Yes. Pekka works at ScyllaDB, and the paper references Avi Kivity's work at Scylla twice in the endnotes, as well as the underlying Seastar engine that Scylla is built upon (that's where DPDK is embedded).

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#174

Post by baybal2 » Fri, May 03, 2019, 7:17 PM UTC

thaumaturgy wrote: post #82

The paper reads like it's suggesting moving the burden of complexity in dealing with varying hardware interfaces from the kernel to userland so that userland can take direct advantage of higher performance hardware when it's available. I could see that for some very small niches, but in general I think it would be a terrible development for the industry. Hardware vendors don't like to share. They don't share code, th…

> Hardware vendors don't like to share. They don't share code, they don't share common interfaces, they don't even share documentation. As it is now, these are all problems which most userland developers don't have to care about -- those problems get dealt with in the kernel, by developers who specialize in building support for uncooperative hardware.

Thus more money for us :) I think this fact almost begs to be taken advantage of. See, how things work in corporate storage products, with EMC being the prime example

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#175

Post by kabdib » Fri, May 03, 2019, 7:18 PM UTC

Animats wrote: post #118

Mainframe designers had this problem under control by 1970. Mainframes had, and have, "channels". A channel is part of the processor architecture. It takes commands, sends them to a peripheral, and manages the data transfer in both directions. Channels have some privileged functions through which the OS tells them where the data is supposed to go in memory. The architecture of channels is well defined, and peripheral…

IIRC the IBM Microchannel stuff was perceived as a ploy to recover control of the PC industry by licensing a new bus. The licensing terms were pretty bad (it could be that "free" was the expected bar at the time, though) and the implementation was expensive, too. Unfortunately for IBM the industry was pretty happy using the existing buses, and when more performance was needed the industry was equally happy to invent new buses that didn't involve paying royalties to IBM.

So regardless of MC's attractiveness from a systems standpoint, it was a nonstarter in an industry that was headed towards an ecology of clones and racing to the bottom in manufacturing costs. Many PC manufacturers ditched parity memory, too.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#176

Post by scroot » Fri, May 03, 2019, 7:21 PM UTC

Spivak wrote: post #162

scroot wrote: post #149

Earlier quoted context omitted.

I'm not sure there has to be a lower level at all, aside from asm and machine language

Okay so you have a device driver that exposes your underlying hard disk as a single contiguous segment of blocks of a given size. You have objects which are blobs of heterogeneously sized arbitrary data with an addressing scheme. The software that marries these two things is basically a filesystem driver (in that you can implement filesystem semantics on top of it -- hell Ceph does it right now). Nothing about a mode…

Of course, you need some agent that can communicate with the hard disk and that knows how to manage memory. Maybe its one driver, maybe its several, but these can also just be objects. Live objects, I mean. Not just blobs of data, but actors that have behaviors and are always "running." You don't even need the concept of files or filesystems. And if you do, better left to a higher level

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#177

Post by PeterCorless » Fri, May 03, 2019, 7:22 PM UTC

gambler wrote: post #170

jandrewrogers wrote: post #159

Earlier quoted context omitted.

How are you defining "very large"? I've used HANA and even SAP doesn't make claim of real scalability -- the practical limitations are in their own documentation. And in real scalability testing, it struggles long before you reach those limits. For sensor data, I wouldn't use it for more than a few terabytes, and even then it has a few sharp edges when it comes to performance if you are not careful. No one is going t…

I missed the part about "petabytes a day". I'm curious, which companies stores petabytes of data a day in a single database?

Facebook was hitting 4 petabytes a day of user data back in 2014. Back then, there were only a handful of giants needing that. Today, multiple systems are now being scaled to handle that sort of data volume. Think automotive fleet systems and other IIoT applications. Mobile networks (and security systems for them) managing tens of millions of devices.

https://research.fb.com/facebook-s-top-open-data-problems/

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#178

Post by magicalhippo » Fri, May 03, 2019, 7:24 PM UTC

jandrewrogers wrote: post #12

At least in database kernels, we noticeably reached this threshold around five years ago with typical server hardware. This is an interesting computer science problem in that virtually all of our database literature is based on the presumption that I/O is much slower than CPU. If you cleanroom a database kernel design based on the assumption that I/O performance is not the bottleneck, you end up with an architecture…

Just skimmed the article, but it reminded me very much of how graphics APIs have evolved, due to similar motivation.

Just compare what it takes to draw a triangle in OpenGL 1.x[1] vs Vulkan[2].

[1]: http://nehe.gamedev.net/tutorial/your_first_polygon/13002/

[2]: https://vulkan-tutorial.com/Overview

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#179

Post by inetknght » Fri, May 03, 2019, 7:29 PM UTC

convolvatron wrote: post #154

inetknght wrote: post #114

Look to GPUs for better solutions: make more discrete -- less functional overall but higher performance -- cores, and move them closer to the data; then let the CPU just handle coordination of the discrete processors. Think of SIMD but on a massive scale. Think of blocks of RAM with math processors. Or the same in your NVMe/NIC/etc.

'systems code' like protocol implementations and device drivers tend to be very control-flow centric. on a simd machine in the worst case this means narrowing the 'vector length' to effectively 1. since these are throughput machines, thats often pretty bad. I do agree with you that lots of little processors is a good way forward here with a careful eye towards reducing sharing of state, but maybe its useful in this c…

I thought separate cores do have their own separate instruction streams -- sometimes even completely different architectures and/or supported instructions? Is that not the case?

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#180

Post by PeterCorless » Fri, May 03, 2019, 7:30 PM UTC

worble wrote: post #74

strictfp wrote: post #64

Earlier quoted context omitted.

One idea I've been throwing around is to replace files with HTTP resources; everything is a resource. Effectively Plan9s idea, but the time might be right.

I don't know much about the OS and designs, but is that similar to how Redox is approaching it with "everything is a url"[0]? [0] https://doc.redox-os.org/book/design/urls_schemes_resources....

If you go that route, bind unique objects to Uniform Resource Names (URN), and then have a mapping to instantiations (URLs), so that the same resource can be found in its multiple locations. Would help immeasurably for a distribution model.