Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Non-volatile Storage: CPUs no longer more performant than I/O devices

queue.acm.org

Post reply on HN
Print view View original

101–110 of 121 posts

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#101

Post by ChuckMcM »

AndrewKemendo wrote: post #100
ChuckMcM wrote: post #75

This is so true, the world has changed greatly and not everyone has gotten the memo. I saw a really cool device made by Texas Memory systems which was a "ram disk" that was all ram with disk backing, and when you lost power it flushed to disk. I wanted something that worked better for a storage paradigm and designed/invented a network accessible memory appliance[1]. Basically using ethernet packets you could store 8K…

That is fascinating. Could this reliably reduce the hardware footprint on any device?

If I understand the question, then yes. When you consider the amount of cache memory in clustered systems which is all holding the same stuff in every independent machine. Using it simply as a victim cache for a block storage device penciled out to a pretty significant improvement.

It gets even better with 64 bit address spaces and a bit of kernel code to 'fault in' from the device.

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#104

Post by derefr »

fungos wrote: post #53
mikehollinger wrote: post #35

Earlier quoted context omitted.

Being up front: this is what I work on for IBM Systems. A buddy wrote this blog ( https://www.ibm.com/developerworks/community/blogs/fe313521-... ) with a little more info. What we have is an IO offload accelerator that knows how to drive high bandwidth IOs to some external storage device. A user app doesn't interact with the device - they make shared library calls to read or write data from a particular buffer, and…

That is really interesting, but I see this as a short term solution to a new and amazing world. What we are doing is trying to hammer something with potential to change most of CS to the shape of our current reality - what is understandable due to the commercial nature of these solutions. But the CS community should think about this with a fresh point of view, maybe get back to the origins and start over with this ki…

Even more interesting with NUMA: imagine 10K slow/cheap cores, each with their own non-shared bit of NVMe (~10MB would do) for their heaps to live on. Perfect for running Erlang.

There might not even be a point in a "classical" CPU cache hierarchy in such a system, if the NVMe is fast enough, and has its own "internal" writeback cache (e.g. some volatile battery-backed memory) protecting it, so that cycling a bit at 3GHz doesn't burn it out. At that point you may as well say you have a CPU with ten million nonvolatile registers.

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#105

Post by ori_b »

eloff wrote: post #43
amluto wrote: post #30

Earlier quoted context omitted.

> While you were context-switching into this OS server you missed the chance to do 10 IOPs give or take an order of magnitude. I'll believe that when I see real numbers. A system call takes something like 54 ns on my laptop. With pwritev or similar, you can do quite a few IOs in a system call. (Of course, pwritev is slower than 54 ns, but that's not a fundamental constraint.) An IO requires making the IO durable if y…

The syscall overhead isn't the problem, that's dirt cheap as you say. The problem is the context-switch overhead. Calling into the OS flushes a lot of data and instructions from the cache, and that lost performance after returning can easily add up to around 30µs.[1] [1] http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-ma...

> OS flushes a lot of data and instructions from the cache

Only if you actually switch contexts. For I/O, this isn't necessary -- you don't need to touch a thing in the TLB.

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#107

Post by baq »

dogma1138 wrote: post #4

So NVDIMM's... Is any one actually making those except Viking and is anyone actually supporting them in servers except SuperMicro? These are basically DDR3/DDR4 DIMM's with onboard flash and a supercap/battery pack to provide persistence incase of system reboots and power failures. They are also a bit odd as they would ignore various system event calls from the BIOS/UEFI and then have to be specifically managed by va…

see https://en.wikipedia.org/wiki/3D_XPoint

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#108

Post by jdright »

derefr wrote: post #104
fungos wrote: post #53

Earlier quoted context omitted.

That is really interesting, but I see this as a short term solution to a new and amazing world. What we are doing is trying to hammer something with potential to change most of CS to the shape of our current reality - what is understandable due to the commercial nature of these solutions. But the CS community should think about this with a fresh point of view, maybe get back to the origins and start over with this ki…

Even more interesting with NUMA: imagine 10K slow/cheap cores, each with their own non-shared bit of NVMe (~10MB would do) for their heaps to live on. Perfect for running Erlang. There might not even be a point in a "classical" CPU cache hierarchy in such a system, if the NVMe is fast enough, and has its own "internal" writeback cache (e.g. some volatile battery-backed memory) protecting it, so that cycling a bit at…

NVM as fast as registers, that is a bold ~1cycle per write/read! But would be really awesome.

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#109

Post by brudgers »

Serious but naive question: does a bottleneck curve trending toward CPU subsystems suggest micro kernel based approaches replacing spinning up virtual machines as a future trend due to the possibility of reduced overhead at the CPU?

tldr; Does increasing use of Storage Class Memory imply increasing use of microkernals?

Re: Non-volatile Storage: CPUs no longer more performant than I/O devices

#110

Post by mikeash »

datadata wrote: post #78
mikeash wrote: post #67

Earlier quoted context omitted.

I have no idea what difference is implied between (1) and (2) there. What's the difference between speed and performance here?

(1) The bandwidth is being entirely consumed. It is all needed and the process is performing nominally. (2) The bandwidth is being entirely consumed, but it might only be because the process is misbehaving and consuming more bandwidth than necessary. Speed only translates to performance if the resource being quickly consumed is actually needed. Example-- a game might always use all CPU cycles, but only when rendering…

Is that difference how people would actually understand the word? I've never (knowingly) heard it used like that. Certainly terms like "CPU utilization" and "network utilization" are frequently used to denote a simple use-divided-by-capacity ratio.

For your game example, I don't see what isn't encompassed by "speed" (how many FPS you get out) or "efficiency" (how much CPU/GPU you use to do it).

Post reply on HN