Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

penberg.org

Post reply on HN
Print view View original

281–290 of 291 posts

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#281

Post by AtlasBarfed »

yingw787 wrote: post #13

There was a great blog post I read a while back about constructing a caching layer across network by Dan Luu: https://danluu.com/infinite-disk/ I asked a friend who works in a quant firm and he was like yes it’s true, and it is pretty insane. I think there’s research Microsoft and Google are doing for RDMA over 100G Ethernet for intra data center communication as well. Pretty neat.

all that huge scale distributed storage is going to run into CAP concerns with imperfect network links, nodes, storage units. I get a lot of that is consumer/social network crapdata with low guarantees to the end user. Dropbox would probably like a little more of a guarantee, and enterprise even more so.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#282

Post by blihp »

loeg wrote: post #109
blihp wrote: post #76

Earlier quoted context omitted.

Individually no, collectively (i.e. storage arrays) yes.

Sort of. Intel's server parts top out at on the order of 48 PCIe 3 lanes (Skylake-SP). Those parts have 6-channel DDR4 memory (rated to DDR4-2666), giving a theoretical DRAM throughput limit of about 128 GB/s. Meanwhile, a PCIe 3.0 lane has a theoretical throughput limit of about 985MB/s; 48 lanes makes 47 GB/s. Both are theoretical numbers and in practice will be lower, and you're right that it doesn't give you a lo…

The question was about spinning rust drives ('before SSDs') and the corresponding CPUs that were available at the same time.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#283

Post by MathematicalArt »

bluetomcat wrote: post #42

We need entirely new OS abstractions to replace the dated notions of hierarchical file systems built around the metaphor of file cabinets, I/O as streams of bytes, terminals, process hierarchy. Essentially, say goodbye to the Unix model after 50 years. It would open up an entirely new world of software experimentation and craftsmanship.

See: “Designing better file organization around tags, not hierarchies”[1]

The author has done a thorough preliminary exploration on this matter. [1] https://www.nayuki.io/page/designing-better-file-organizatio...

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#284

Post by bogomipz »

deRerum wrote: post #225
bogomipz wrote: post #222

Earlier quoted context omitted.

>"If memory speed is 100ns then you would notice the memory bottleneck around the time when your processor speed is 10Mhz." Sorry I'm not following the math there, whats the relation between 100ns and 10Mhz? Why is that the tipping point?

A 10Mhz processor has a clock cycle of 100ns (0.1 millionths of a second). Those are just rough representative numbers I picked...any particular RAM delays would be different and the actual latency would be complicated by bus speeds and protocols etc.

Thanks for the explanations that math makes sense to me now. Cheers.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#285

Post by sinisa_cyprus »

Only mainframes had channels. Best implementation thereof is in IBM machines. It is nothing like DMA or that Intel chip or anything else. No Unix machines nor PC and not even specialised hardware, like Tangent, had anything similar.

It is something like separate FPU or MMU units, built for the total control of the peripherals, so that CPU had little or no work to do. Don't forget that device drivers run on CPU.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#286

Post by Rafuino »

bogomipz wrote: post #223
tracker1 wrote: post #47

Earlier quoted context omitted.

The problem is, it varies... you have what are effectively key/value stores to document databases, column stores and everything in between. You have systems built on other systems. RethinkDB and Cockroach have different approaches than Redis, Mongo, Cassandra or others. CockroachDB gives an SQL interface over the top of a distributed data store with better consistency and relations. Cassandra has no real relations ov…

>"I think with what's coming out of SSD/NVMe and even Optain DIMMS, that there will be databases directly tuned to control/set their own data storage in these environments." I think this is already happening. I think it was Aerospike that used the FTL of of NANd/NVMe drives for a direct key value store and I think another vendor maybe Fusion had an SDK for this as well. The Optane stuff looks really interesting, are…

Not OP but, yes, server vendors are shipping with Optane DC Persistent Memory, like Lenovo [1] and Supermicro [2], among others. Google also has a testing program [3, 4] with access to instances with 7TB of total memory.

[1] https://lenovopress.com/lp1066-intel-optane-dc-persistent-me... [2] https://www.storagereview.com/supermicro_superserver_with_in... [3] https://cloud.google.com/blog/topics/partners/available-firs... [4] https://docs.google.com/forms/d/e/1FAIpQLSeX1tN6Qt-aQUK2iVVi...

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#287

Post by _msw_ »

_msw_ wrote: post #250
transpute wrote: post #196

Earlier quoted context omitted.

> So there's no standardization on how to talk to devices at the hardware level. Some CPUs have protection systems, an "I/O MMU", and there have been various channel-like interfaces, especially from Intel, but they have never caught on. From the paper: > NVMe SSDs perform I/O faster than the OS can accept new I/O requests and notify their completion. Furthermore, the current POSIX AIO implementations in Linux are ugl…

Disclosure: I work at AWS and played a part in building the Nitro system The Nitro system does use SR-IOV, and it can take advantage of hardware virtualization features like VT-d posted interrupts to lower interrupt virtualization overhead. But VT-d posted interrupts isn't a factor in avoiding polling. I'd go further to say that the Two Six Labs blog post has some inaccuracies, so I wouldn't depend on the information…

See also this thread: https://news.ycombinator.com/item?id=19843464

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#289

Post by warrenm »

temac wrote: post #190
masklinn wrote: post #35

Earlier quoted context omitted.

I assume the IO you're talking about is (/was) sequential. And DBs are specifically engineered to sequentialise their IO (e.g. clustered indexes). But even if SSDs do blow HDDs out the water on sequential IO, it's on random IO that the difference is most stark, the cost difference between sequential and random is much lower on SSD than on HDDs, and both random IO throughput and random IOPS shoot through the roof rela…

> low-mid tens microseconds From a modern CPU point of view, that's an eternity.

Still 1000x faster than milliseconds for HDD

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#290

Post by warrenm »

bluetomcat wrote: post #70
thaumaturgy wrote: post #52

Earlier quoted context omitted.

I would like to see an updated file system architecture that's closer to a database, with tagging and all that. And I could see that extending to processes too. But what are you imagining as alternatives for i/o and terminals?

What if we surpass file systems completely? Imagine every process having its own "non-volatile" memory area, where all persistent state is kept between restarts, with a clean OS interface to share that state to other processes? Terminals and process hierarchy are a complete no-go in a fresh design. Every entity in the system can be identified through unique IDs which can be handed down to other processes based on dif…

>Imagine every process having its own "non-volatile" memory area, where all persistent state is kept between restarts, with a clean OS interface to share that state to other processes?

So ... something Docker-esque?

Post reply on HN