Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Non-volatile Storage: CPUs no longer more performant than I/O devices

queue.acm.org

Post reply on HN
Print view View original

51–60 of 121 posts

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#51

Post by Scramblejams »

pklausler wrote: post #14
thrownaway2424 wrote: post #8

Performant: still not a word.

Seymour Cray never said "performant". Engineers say "fast" or "fast enough". Marketing types and nontechnical management seem to prefer this neologism. But it might also be a generational thing. A new coinage that I noticed in the past year that also grates on my ears: "learning" as a substitute for "lesson", as in "what were your learnings from the hackathon?" Anyone else caught this one?

Also noticed it several years ago. I may be wrong but to me it seems to have arrived from the same group that's imposed corporate-speak on boardrooms everywhere, including high-profile retreadings of words like synergy and paradigm.

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#52

Post by matt_d »

mikehollinger wrote: post #35
Animats wrote: post #15

I've made that point before on YC.[1] We need to view fast storage as something other than a disk accessed through the OS, and other than slow RAM accessed as raw memory. Access through the OS is too slow, and access as raw memory is too risky. What's probably needed is something like a GPU sitting between the CPU and the fast persistent storage. Call this an SPU, or "storage processing unit." What would such a devic…

Being up front: this is what I work on for IBM Systems. A buddy wrote this blog ( https://www.ibm.com/developerworks/community/blogs/fe313521-... ) with a little more info. What we have is an IO offload accelerator that knows how to drive high bandwidth IOs to some external storage device. A user app doesn't interact with the device - they make shared library calls to read or write data from a particular buffer, and…

Interesting!

Incidentally (since this may be somewhat related), I'm wondering, what are your thoughts on the Persistent Memory Manager approach, as in the following:

Justin Meza, Yixin Luo, Samira Khan, Jishen Zhao, Yuan Xie, and Onur Mutlu: "A Case for Efficient Hardware/Software Cooperative Management of Storage and Memory." Workshop on Energy-Efficient Design, 2013.

Context: "emerging high-performance NVM technologies enable a renewed focus on the unification of storage and memory: a hardware-accelerated single-level store, or persistent memory, which exposes a large, persistent virtual address space supported by hardware-accelerated management of heterogeneous storage and memory devices. The implications of such an interface for system efficiency are immense: A persistent memory can provide a unified load/store-like interface to access all data in a system without the overhead of software-managed metadata storage and retrieval and with hardware-assisted data persistence guarantees."

The stated goals/benefits include eliminating operating system calls for file operations, eliminating file system operations, and efficient data mapping.

Paper: http://justinmeza.com/bin/meza_weed13.pdf

Presentation: https://users.ece.cmu.edu/~omutlu/pub/mutlu_weed13_talk.pdf

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#53

Post by fungos »

mikehollinger wrote: post #35
Animats wrote: post #15

I've made that point before on YC.[1] We need to view fast storage as something other than a disk accessed through the OS, and other than slow RAM accessed as raw memory. Access through the OS is too slow, and access as raw memory is too risky. What's probably needed is something like a GPU sitting between the CPU and the fast persistent storage. Call this an SPU, or "storage processing unit." What would such a devic…

Being up front: this is what I work on for IBM Systems. A buddy wrote this blog ( https://www.ibm.com/developerworks/community/blogs/fe313521-... ) with a little more info. What we have is an IO offload accelerator that knows how to drive high bandwidth IOs to some external storage device. A user app doesn't interact with the device - they make shared library calls to read or write data from a particular buffer, and…

That is really interesting, but I see this as a short term solution to a new and amazing world. What we are doing is trying to hammer something with potential to change most of CS to the shape of our current reality - what is understandable due to the commercial nature of these solutions.

But the CS community should think about this with a fresh point of view, maybe get back to the origins and start over with this kind of technology. Or maybe we do this already and I just do not know?

For myself, since I've got out of university, I've always thought about how things would be different if we hadn't disk+ram, but a storage that solved the two with the best of each (top speed and large and cheap capacity) - extrapolate this thing to a kind of SoC with 40+ cores and 40TB+ LD1 cache - and I tried to imagine what would be needed in terms of a new OS made from scratch for this thing. This still keeps me thinking on new designs, new algorithms, etc. Sadly, I've never tried or even theorized anything interesting apart of entirely killing the file-system concept and having _always loaded applications_ running (equivalent to processes) or suspended (equivalent to app binary files)... :)

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#54

Post by rsp1984 »

Please excuse my ignorance on this matter but will this technology have any impact on the hierarchy levels below disk (i.e. RAM and CPU caches)? Compared to Register, L1 and L2 access RAM access is still really slow. Will non-volatile storage latencies rival or exceed those of standard RAM? From how I understand the article it's primarily disk IO speed that's affected, correct?

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#55

Post by fungos »

matt_d wrote: post #52
mikehollinger wrote: post #35

Earlier quoted context omitted.

Being up front: this is what I work on for IBM Systems. A buddy wrote this blog ( https://www.ibm.com/developerworks/community/blogs/fe313521-... ) with a little more info. What we have is an IO offload accelerator that knows how to drive high bandwidth IOs to some external storage device. A user app doesn't interact with the device - they make shared library calls to read or write data from a particular buffer, and…

Interesting! Incidentally (since this may be somewhat related), I'm wondering, what are your thoughts on the Persistent Memory Manager approach, as in the following: Justin Meza, Yixin Luo, Samira Khan, Jishen Zhao, Yuan Xie, and Onur Mutlu: "A Case for Efficient Hardware/Software Cooperative Management of Storage and Memory." Workshop on Energy-Efficient Design, 2013. Context: "emerging high-performance NVM technolo…

Thanks, that is something that I was looking for! Nice to see it really happening!

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#56

Post by AnimalMuppet »

Animats wrote: post #18
ricw wrote: post #16

Earlier quoted context omitted.

Just to clarify: this is to enable programs to have more a more direct and faster interface to data, while retaining data consistency and safety that typically would be managed by the OS?! Is this not just another form of DMA (direct memory access)? And if so, how would it differ from current implementations? Sometimes DMA only refers to ram, though on many systems this is fluid between different data storage types (…

"this is to enable programs to have more a more direct and faster interface to data, while retaining data consistency and safety that typically would be managed by the OS?" No, managed by the database engine. The idea is to put the data-intensive operations of a database engine into a highly parallel SPU. The application would see an interface much like an SQL or NoSQL database.

But see, you're limiting your idea to databases. I don't think you should do that. The idea of a "storage accelerator" (by whatever name) could be used far more widely than that.

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#57

Post by i336_ »

mikehollinger wrote: post #35
Animats wrote: post #15

I've made that point before on YC.[1] We need to view fast storage as something other than a disk accessed through the OS, and other than slow RAM accessed as raw memory. Access through the OS is too slow, and access as raw memory is too risky. What's probably needed is something like a GPU sitting between the CPU and the fast persistent storage. Call this an SPU, or "storage processing unit." What would such a devic…

Being up front: this is what I work on for IBM Systems. A buddy wrote this blog ( https://www.ibm.com/developerworks/community/blogs/fe313521-... ) with a little more info. What we have is an IO offload accelerator that knows how to drive high bandwidth IOs to some external storage device. A user app doesn't interact with the device - they make shared library calls to read or write data from a particular buffer, and…

This is pretty interesting.

It's possible that people might confuse this bit...

> The performance of SCMs means that systems must no longer "hide" them via caching and data reduction in order to achieve high throughput.

...in the original article with your mention of caching; by "cache coherency," I assume you're referring that your addon (card?) can introspect into the CPU cache? That's pretty awesome if that's what's happening.

Some hopefully relevant questions from someone totally unfamiliar with this particular area:

- The original article mentioned "RAM emulation" (to put it crudely) as "unstable." Do you have any comment on this?

- Do you happen to have any performance figures you can release?

- From the blog article and video I get the idea that this is POWER-specific. :) Are you aware of any alternative offerings for x86 that offer similar performance?

- What does this thing (I have no idea if it's a card, a module...) look like? Being able to see "the thing" is generally really cool :)

My last question about POWER8 in general is arguably both on- and off-topic and might be a question for a different team, but do you know...

a) if/when POWER8 will manage to escape from the datacenter and become accessible to developers in the hobbyist/student sector? My understanding is that the architecture as it stands at the moment requires lots of different components that unavoidably require a lot of space; are you aware of any scaling-down efforts to produce (even (E)ATX-sized) POWER8 SBCs people can play with?

b) if/when full-scale POWER8 systems will be available in the style of Heroku/OpenShift, both of which have free tiers that allow for entry-level poking? I understand that RunAbove provided something along these lines with (1-?) POWER system(s), but that dried up some time ago, and I'm not aware of any replacements.

All in all, this Flash system looks pretty cool, and I can definitely say I wouldn't mind being a fly on the wall for a day in your office, what with getting to play with 40TB of Flash (SSDs...?) - wow. :D

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#58

Post by erichocean »

My sense is this is only true today because OS kernels are ridiculously slow relative to what the hardware can achieve.

Most of my recent designs treat RAM as if it were (what we used to considered to be) disks, i.e. all computation and in-process data is in cache exclusively, and "going to RAM" requires the use of a B-tree-like structure to amortize the cost.

For example, once you've opened a RAM page line on a normal four-channel Xeon server, you can read the entire 4KB page in about the same time it takes to read one byte, switch pages, and then read another byte. (Of course, you can't do that either since the entire cache line will be filled, but the overall point still stands.)

The situation we're in today with RAM is pretty much the identical situation with the disks of yore. Anyway…interesting article nonetheless.

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#59

Post by CamperBob2 »

eloff wrote: post #43
amluto wrote: post #30

Earlier quoted context omitted.

> While you were context-switching into this OS server you missed the chance to do 10 IOPs give or take an order of magnitude. I'll believe that when I see real numbers. A system call takes something like 54 ns on my laptop. With pwritev or similar, you can do quite a few IOs in a system call. (Of course, pwritev is slower than 54 ns, but that's not a fundamental constraint.) An IO requires making the IO durable if y…

The syscall overhead isn't the problem, that's dirt cheap as you say. The problem is the context-switch overhead. Calling into the OS flushes a lot of data and instructions from the cache, and that lost performance after returning can easily add up to around 30µs.[1] [1] http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-ma...

Sounds like multiple caches are an obvious solution. No code in the OS needs access to the user code or data cache, and vice versa.

Smart cache management might be one of Intel's goals for the Altera buyout.

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#60

Post by mikeash »

ape4 wrote: post #45
mikeash wrote: post #23

Earlier quoted context omitted.

See also: "utilize" instead of "use."

Brits use "whilst" instead of "while". Thank god they didn't pick the keywords of any programming languages. utilise namespace std; whilst (performant()) { pontificate(); } Edit: bonus `problematic` operator for exceptions.

I don't know, I think a "whilst loop" would be kind of charming.
Post reply on HN