Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rest in Peace, Optane

specbranch.com

Post reply on HN
Print view View original

21–30 of 122 posts

Re: Rest in Peace, Optane

#21

Post by ddalex »

zinodaur wrote: post #14
hardware2win wrote: post #6

What was interesting about Optane was that it was kinda an attempt to get rid of disk and ram distinction. You had single device (optane stick) which served as both - ram and disk. After decades of current approach we finally could physically get rid of disks - hdds, ssds or even nvme sticks. https://www.theregister.com/2022/08/01/optane_intel_cancella... >Optane presented a radical, transformative technology but bec…

Yeah - the article was talking about mmap... but what i wanted was to not have to define persistence boundary. I wanted the entire in-memory state of my program to be persisted - perhaps even duplicated and moved elsewhere

But then you can't just reset the state when crashes happen and data corrupts

Re: Rest in Peace, Optane

#22

Post by znpy »

ritwikgupta wrote: post #13
Dalewyn wrote: post #11

Earlier quoted context omitted.

So the winning move now is for Apple to announce iOptane and sweep the data storage and cache industries by storm?

Apple may already be headed in that direction. They already have unified CPU and GPU RAM. It doesn’t seem far-fetched to imagine that they could unify persistent storage and memory.

Knowing apple’s marketing moves, they could definitely do that: just use a single number to describe memory. And then pretend it’s a big number.

Re: Rest in Peace, Optane

#23

Post by jared_hulbert »

buildbot wrote: post #3
mikewarot wrote: post #2

My HP laptop came with optane memory, as far as I could tell, it was nothing special. I eventually pulled it to upgrade to a Terabyte ssd

That was probably a hybrid optane setup, like the h10 ssd. It was a tiny optane ssd and a normal ssd on the same m.2 card. Not very good compared to real optane. A really noticeable improvement when running apt, or dnf, or any windows program install.

Silly hack of a product. x2 PCIe for Optane, 2x for NAND. Meant compromised bandwidth for all! The fact that it actually had value in some cases highlights how cool it could have been if it was a proper hybrid design.

Re: Rest in Peace, Optane

#24

Post by tanelpoder »

Yeah I'm sad to see Optane go. I bought a few DCPMMs myself (and had to upgrade my workstation CPUs to support them) to test out the capabilities and maybe even write a little toy OS/KV-store engine that runs as a multithreaded process in Linux using mmap'ed Optane memory as its persistent datastore (never got to the latter part). The "do not use mmap for databases" argument would not apply here as you wouldn't cache persistent block devices to RAM, but the "RAM" itself is persistent, no writeback or block disk I/O is needed.

Intel discontinued/deprecated Optane, before I could do anything really cool with it. But Intel can probably still reuse lots of their cache coherency logic for external CXL.mem device access.

One serious early adaptor and enterprise user of Optane tech was Oracle. More specifically Oracle's Exadata clusters (where database compute nodes are disaggregated from storage nodes that contained Optane) and connected via InfiniBand or RoCE. And since Optane is memory (memory addressable), they could skip OS involvement and some of the interrupt handling when doing RDMA ops directly to/from Optane memory located inside different nodes of the Exadata cluster. I think they could do 19 microsecond 8kB block reads from storage cells and WAL sync/commit times were also measurable in microseconds (if you didn't hit other bottlenecks). They could send concurrent RDMA write ops (to remote Optane memory) for WAL writes into multiple different storage nodes, so you could get very short commit times (of course when you need sync commits for disaster recovery, you'd have to pay higher latency).

With my Optane kit I tested out Oracle's Optane use in a single-node DB mode for local commit latency (measured in a handful of microseconds, where the actual log file write "I/O" writes were sometimes even sub-microsecond). But again, if you need sync commits across buildings/regions for disaster recovery and have to pay 1+ ms latency anyway, then the local commit latency advantage somewhat diminishes. I have written a couple of articles about this, if anyone cares: [1][2].

Fast forward a few years, Oracle is not selling Exadata clusters with PMEM anymore, but with XMEM, which is just RAM in remote cluster nodes and by using syscall-less RDMA ops, you can access it with even lower latency.

[1] - https://tanelpoder.com/posts/testing-oracles-use-of-optane-p... [2] - https://tanelpoder.com/posts/testing-oracles-use-of-optane-p...

Re: Rest in Peace, Optane

#25

Post by jamiek88 »

hlandau wrote: post #7
hardware2win wrote: post #6

What was interesting about Optane was that it was kinda an attempt to get rid of disk and ram distinction. You had single device (optane stick) which served as both - ram and disk. After decades of current approach we finally could physically get rid of disks - hdds, ssds or even nvme sticks. https://www.theregister.com/2022/08/01/optane_intel_cancella... >Optane presented a radical, transformative technology but bec…

Still waiting for the Memristor...

Oh jeez I was so excited for that!

Re: Rest in Peace, Optane

#26

Post by tanelpoder »

zinodaur wrote: post #14
hardware2win wrote: post #6

What was interesting about Optane was that it was kinda an attempt to get rid of disk and ram distinction. You had single device (optane stick) which served as both - ram and disk. After decades of current approach we finally could physically get rid of disks - hdds, ssds or even nvme sticks. https://www.theregister.com/2022/08/01/optane_intel_cancella... >Optane presented a radical, transformative technology but bec…

Yeah - the article was talking about mmap... but what i wanted was to not have to define persistence boundary. I wanted the entire in-memory state of my program to be persisted - perhaps even duplicated and moved elsewhere

mmap on Optane direct-access-aware (DAX) filesystems like EXT4/XFS now, is not like mmap on block devices where the OS gets in your way and pages stuff in from disk and (maybe) later syncs it back to persistent storage. Optane is the persistent storage, it's just usable/addressable as regular RAM as it's plugged in to DIMM slots.

And in the later Xeon architecture (Xeon Scalable 3rd gen, I think), intel expanded the persistence domain to CPU caches too. So, you didn't even have to bother with CLFLUSH and CLWB instructions to manually ensure that some cache lines (not 512B blocks, but 64B cache lines) get persisted. You could operate in the CPU cache and in the event of power loss, the CPU/mem controllers/and the capacitors on Optane DCPMMs ensured that the dirty cache lines got persisted to Optane before the CPU lights went off. But all this coolness a bit too late...

Another note: Intel's marketing had terrible naming for Optane stuff. Optane DCPMMs are the ones that go into DIMM slots and have all the cool features. Optane Memory SSDs (like Optane H10) are just NAND SSDs with some Optane cache in front of them. These are flash disks, installed in PCIe slots but Intel decided to call these disks "Optane Memory" ...

Re: Rest in Peace, Optane

#27

Post by eternityforest »

Why was everyone so obsessed with the idea of Optane as universal memory?

From a consumer perspective, it was a fast small SSD that doesn't wear out. That was enough to justify it's value at the prices it went for(Or at least it seems like it, I never personally used it). It didn't need any radical rethink of computing to be worth it.

I'm not even a fan of the nonvolatile RAM idea. Rebooting is often the first thing we do to fix stuff. Modern computing trends towards regenerating from descriptions, not adding more persistent state. Persistent state can get messed up, better to be able to wipe and start over.

Re: Rest in Peace, Optane

#28

Post by jared_hulbert »

zinodaur wrote: post #14
hardware2win wrote: post #6

What was interesting about Optane was that it was kinda an attempt to get rid of disk and ram distinction. You had single device (optane stick) which served as both - ram and disk. After decades of current approach we finally could physically get rid of disks - hdds, ssds or even nvme sticks. https://www.theregister.com/2022/08/01/optane_intel_cancella... >Optane presented a radical, transformative technology but bec…

Yeah - the article was talking about mmap... but what i wanted was to not have to define persistence boundary. I wanted the entire in-memory state of my program to be persisted - perhaps even duplicated and moved elsewhere

memverge.com does some cool work around making that happen.

Re: Rest in Peace, Optane

#29

Post by jandrewrogers »

Optane suffered the same fate as every other storage technology over the decades with a similar value proposition. Broadly, the notional value of storage technology like Optane is that it offers a drop-in improvement in storage performance without requiring any software modification, albeit at a significant cost premium. The theory that this market exists in any significant way never pans out in practice.

On one hand, you have a large market of people that are not sensitive enough to storage performance for the extra cost to be worth it. Ordinary storage is perfectly adequate for their needs and they would see little benefit in transparently making the storage their software was designed for faster.

On the other side, there are people who care about storage performance a lot. So much so, that they will design and modify their software to take full advantage of the storage and system characteristics. They use the storage so well that the gains by putting something in the middle will be marginal or non-existent, and certainly not worth the extra cost —- they would gain just as much by adding more storage devices.

The effective target market always ends up being “people who care a lot about storage performance but use their storage in the most naive way possible”, which isn’t that big of a market in practice. Using good software design to achieve similar results on commodity hardware is almost always the better option.

Re: Rest in Peace, Optane

#30

Post by buildbot »

tanelpoder wrote: post #26
zinodaur wrote: post #14

Earlier quoted context omitted.

Yeah - the article was talking about mmap... but what i wanted was to not have to define persistence boundary. I wanted the entire in-memory state of my program to be persisted - perhaps even duplicated and moved elsewhere

mmap on Optane direct-access-aware (DAX) filesystems like EXT4/XFS now, is not like mmap on block devices where the OS gets in your way and pages stuff in from disk and (maybe) later syncs it back to persistent storage. Optane is the persistent storage, it's just usable/addressable as regular RAM as it's plugged in to DIMM slots. And in the later Xeon architecture (Xeon Scalable 3rd gen, I think), intel expanded the…

There are also fully optane based SSDs such as the 905p or the 5800x
Post reply on HN