Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

penberg.org

Post reply on HN
Print view View original

271–280 of 291 posts

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#271

Post by deRerum »

soup10 wrote: post #238
deRerum wrote: post #217

In the past (like around the time most programming languages were invented) memory speeds were faster than processor speeds. So all variable accesses were instantaneous. Languages like C did not have to worry about memory hierarchies. If memory speed is 100ns then you would notice the memory bottleneck around the time when your processor speed is 10Mhz. This point was reached in the mid 1980s with the 286 processor.…

more of a hardware problem... if CPU's let you directly address and manage L1, L2, L3 cache memory high performance programmers would love it and languages like C++ would immediately add support.

Anyone can prefetch data into the cache, but letting programmers control cache eviction would open the door to all kinds of user errors which would kill performance.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#272

Post by wmu »

BTW, does anybody know a paper about doing some DB ops directly on disc controllers? The other day my former colleague mentioned that he come across such paper (maybe a blog post?), but we couldn't find it. It's really interesting idea and I believe it's doable, although under very specific circumstances (disc vendor specific, sectors layout aligned to DB needs, etc.).

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#273

Post by [deleted] »

ncmncm wrote: post #239
CamperBob2 wrote: post #236

Earlier quoted context omitted.

But you aren't waiting on L1, except in the sense that you're waiting on line fills from main memory. If the problem is that you don't have enough time to do something with the data you're getting, how is that solvable by architectural changes external to the CPU? In the real world, if you need to deal with 400 Gb/s, you aren't going to send it to a single general-purpose CPU using any type of bus or channel. The CPU…

It is only solvable by architectural changes external to the CPU. Even at 40Gbps, which we live with now, that's 20 ns per packet, or 80 ns at a pokey 10Gbps, certainly not enough time to do a system call per packet. So you need the hardware to divvy traffic up to multiple queues -- rings, really -- and a core for each ring. The packets just show up in memory, and the cores had just better keep up. If that looks to y…

[deleted]

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#274

Post by watersb »

timzentu wrote: post #218
dbcurtis wrote: post #161

Earlier quoted context omitted.

That is an IBM- and Univac-centric view of I/O. Control Data Corp had the "peripheral processor", or PP. PP's only ran OS code, usually called "driver overlays". It was actually very elegant. There were 10 copies of PP state (20 in the 7600) and only 1 actual execution unit (2 in 7600). Hardware multi-threading in 1959! So there were 10 PP's executing PP overlay code (drivers) at 1/10 the instruction rate of the main…

Oh man, CDS/CDC (Control Data Systems/ Control Data Corporation) is where my Dad worked. Don't know if you knew him (Ed), but he was one of the last 2-4 employees for the company. He was working up in Michigan until 1996. I still remember we took a vacation to DC and in the Smithsonian they had 2 of the machines he worked on. [Him swearing at some hardware while in a museum because it had cut him is one of my best me…

Friend of mine ran the CDC 6600 at the University of Texas in Austin.

Now I need to ask him about DMA architecture...

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#275

Post by soup10 »

deRerum wrote: post #271
soup10 wrote: post #238

Earlier quoted context omitted.

more of a hardware problem... if CPU's let you directly address and manage L1, L2, L3 cache memory high performance programmers would love it and languages like C++ would immediately add support.

Anyone can prefetch data into the cache, but letting programmers control cache eviction would open the door to all kinds of user errors which would kill performance.

true but it could also be very efficient, especially in cases where you have the exact hardware to test on like a game console

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#276

Post by ignoramous »

hueving wrote: post #246
ignoramous wrote: post #85

Earlier quoted context omitted.

CMU's Andy Pavlo is probably at the forefront in this area, here's what they were working on (they're re-doing it all over again now as a yet to be announced project): https://news.ycombinator.com/item?id=14391523 Also, I'd encourage you to go through their Advanced Database class: https://www.youtube.com/playlist?list=PLSE8ODhjZXjYutVzTeAds... Michael Stonebreaker, a Turing awardee, called for complete destruction o…

Why do you suppose nothing useful has come out of the CMU group for actual use? Is it just a case of limited payoff for marginal gain?

I'm in no position to comment on that, unfortunately. Not sure if Andy's group has detailed why their first-take didn't come off apart from this terse note on GitHub which notes that building an autonomous-db is what they're main goal is (rather than a nvme-first db, which seems like a sub-goal):

> The Peloton project is dead. We have abandoned this repository and moved on to build a new DBMS. There are a several engineering techniques and designs that we learned from this first system on how to support autonomous operations that we are doing a much better job at implementing in the second system.

Here's their alleged unannounced second-take: https://github.com/cmu-db/terrier

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#277

Post by mypalmike »

jstimpfle wrote: post #266
pjmlp wrote: post #254

Earlier quoted context omitted.

Sure it does, not using C to start with. Or in another form, C should have done the same as other systems languages, do proper bounds checking, arrays and strings without implicit decay into pointers. Second, having a proper UI story like NeXTSTEP or NeWS, and not the X11 Frankenstein.

Equating kernel design with choice of implementation is a stretch. Also, Unix is not X11, and weren't the other ones running on top of Unices as well?

Just walk away.

"The language runtime is the OS." - https://news.ycombinator.com/item?id=15468395

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#278

Post by Veedrac »

theincredulousk wrote: post #212
yingw787 wrote: post #13

There was a great blog post I read a while back about constructing a caching layer across network by Dan Luu: https://danluu.com/infinite-disk/ I asked a friend who works in a quant firm and he was like yes it’s true, and it is pretty insane. I think there’s research Microsoft and Google are doing for RDMA over 100G Ethernet for intra data center communication as well. Pretty neat.

Yes! Was surprised the paper didn't specifically mention RDMA, or to a lesser degreee SRIOV, with all their focus on NICs. Also, there may be ongoing research, but it isn't theory at all. HPCs, HFTs, and the could providers have been leveraging RDMA for a long time - e.g. Infiniband. Doing it over Ethernet (RRoCE) is relatively new, and it isn't necessarily any big leap that is happens over 100G instead of 40 or 1. H…

The paper does mention RDMA.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#279

Post by AtlasBarfed »

owaislone wrote: post #22
jandrewrogers wrote: post #12

At least in database kernels, we noticeably reached this threshold around five years ago with typical server hardware. This is an interesting computer science problem in that virtually all of our database literature is based on the presumption that I/O is much slower than CPU. If you cleanroom a database kernel design based on the assumption that I/O performance is not the bottleneck, you end up with an architecture…

Are you aware of any efforts by existing databaes or entirely new databases built with this realization in mind?

IMO the biggest problem is that SQL joins don't scale on commodity hardware networks due to the CAP theorem, assuming you want availability. Network IO is a lot better these days, and per Scylla it can get to a pretty small instruction window, but still it isn't what XPoint and SSD crunch.

Database rearchitecture at the single node level can help optimize this, and maybe improve the megamachine SQL node to scale up to fatter sizes, but really big data is still a network IO and reliability challenge.

And it may be true we can pump a single fat node up to really huge throughput once some db rearchitecture fully leverages SSDs and I/O, but that just exposes more data to downtime if there is a network partition.

Re: I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]

#280

Post by AtlasBarfed »

fulafel wrote: post #259
dorlaor wrote: post #127

Earlier quoted context omitted.

The advantage of in-memory it's also its biggest disadvantage - it's costly. NVMe can serve almost as fast as ram and is 100x cheaper. Scylla offers single digit msec (usually 1msec) 99% latency and way more cost effective

Most databases are small though.

And don't need performance, so clearly we aren't talking about an Access database on a laptop.
Post reply on HN