Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RAM Is the New Disk

medium.com

Post reply on HN
Print view View original

51–60 of 74 posts

Re: RAM Is the New Disk

#51

Post by ksec »

tw04 wrote: post #11

The price of disk has dropped at nearly the same pace as ram. As has the cost of compute. At the same time data growth has increased faster than either has dropped... so I'm not really sure the price argument holds water. If I can buy ram at 1/100th the cost but I need to store 500x more data... that isn't a net win on cost. From ~$1000.00/gb to $0.03/gb http://www.mkomo.com/cost-per-gigabyte-update

Data Growth is faster then RAM price decrease, and at the moment it is actually increasing.

While I dont believe in Infinite growth of data, I still think a RAM only DB isn't as good if we have SSD that is ridiculously fast. My thinking is that RAM / SSD should always be 1:5 or 1:10.

Re: RAM Is the New Disk

#52

Post by FractalNerve »

zeusk wrote: post #29
vbezhenar wrote: post #24

May be with very high-end servers it is. But generally it's not. I can buy 4TB HDD For $200. I think, I'll have to add 2-3 zeros for 4TB RAM machine, and I'm not even saying only about RAM, I need some server motherboard, some server processor, while I can use 4TB RAM HDD with pretty much any computer. And SSD isn't going to be even with HDD in near future as well for $/Byte. So optimizing software for HDD won't go a…

NVMe drives which are approaching RAM speeds are a good compromise if RAM and server components are outside of your financial reach.

@Zeusk is right, I wish costs of SSD-RAMs like Diablo Memory1 [1] were more affordable or transparent.

Summary performance comparison by Storage Class: http://xitore.com/what-is-nvm-x

    NVM-X RAM ≈ DDR4-3200 ↧Availability|↥Cost| ↥4TB@25,6 GB/s
    Diablo Memory1        ↥Availability|↥Cost|↱256GB@10GB/s
EDIT, added source: [1] http://www.diablo-technologies.com/memory1/

Re: RAM Is the New Disk

#53

Post by FractalNerve »

arielweisberg wrote: post #5

... I was the third engineer at VoltDB and spent six years making that bet. It's not a good bet. Maybe there are other factors, but if VoltDB could page out cold data to disk I think it would be at least 2x if not more successful. No one agreed with me so it never happened. I saw so many use cases go out the door because hey you know what? RAM is expensive and it's cheaper to page out cold data. The scale where that…

Given a time-series history of DRAM/SSD prices, by "what value" do you need to be able to buy 1TB RAM (or anything approaching RAM speeds) in order to make VoltDB and in-Memory Databases competitively advantageous?

So, given this insider knowledge of yours, can we make a prediction by what date predicively DRAM/SSD-NVMe prices may make in-Memory Database Startups lucrative again?

--

Offtopic:

I feel empathy for you, being overrun in decisions as an engineer in a field you're the expert in market and technology by decision makers can be heart-breaking.

EDIT:

removed irrelevant personal experience

Re: RAM Is the New Disk

#54

Post by ars »

FeepingCreature wrote: post #45
old-gregg wrote: post #44

Earlier quoted context omitted.

I don't recommend this anymore. With a typical developer machine containing 16GB of RAM, and especially on Linux, you will that all of your daily-touched files are in FS cache after a few minutes of work. Even with default kernel settings Linux is pretty good with eating up all of your unused RAM for speeding up disk access. Here's my anecdote based on 16GB workstation with NVMe SSD (Samsung 960 Pro): Watching my pro…

One memory leak and your cache is gone. [edit] Not sure if ramdisks are pinned though.

> Not sure if ramdisks are pinned though.

Ramdisks will go to swap. A memory leak will force the entire ramdisk into swap, and reading it back into memory afterward is 10 to 100 times slower than reading normal files off of a disk.

Re: RAM Is the New Disk

#56

Post by olavgg »

dis-sys wrote: post #41
zeusk wrote: post #29

Earlier quoted context omitted.

NVMe drives which are approaching RAM speeds are a good compromise if RAM and server components are outside of your financial reach.

many NVMe drives on the market are useless jokes. try some from the now biggest semiconductor company, test their fsync() performance and don't get a heart attack for seeing those ugly numbers. ;)

This!

https://forums.servethehome.com/index.php?threads/did-some-w...

Re: RAM Is the New Disk

#57

Post by bitL »

lathiat wrote: post #20
0xbear wrote: post #12

And 99% of developers spend their entire careers without giving as much as a passing thought to cache locality. Quick, how long, in cycles, does it take to retrieve data from RAM? About 200 cycles. 200 cycles is a very long time if you miss cache often. Scattered RAM reads can be _slower_ than sustained linear disk reads (that is, once the disk actually gets around to reading, which takes a while).

It's even worse when you get into interpreted languages like Python and Ruby. Bad efficiency at that level which translates to dreadful if not totally broken efficiency at the cache level.

Bbbut JIT compiler solves everything, right?

BTW, this is a blind spot/elephant in the room of most *VM-based and functional languages nobody wants to talk about...

Re: RAM Is the New Disk

#58

Post by amelius »

If only programming languages supported "offsetted pointers", we could use mmapped files and store arbitrary data structures in them without hassle.

Re: RAM Is the New Disk

#60

Post by dis-sys »

olavgg wrote: post #56
dis-sys wrote: post #41

Earlier quoted context omitted.

many NVMe drives on the market are useless jokes. try some from the now biggest semiconductor company, test their fsync() performance and don't get a heart attack for seeing those ugly numbers. ;)

This! https://forums.servethehome.com/index.php?threads/did-some-w...

hi olavgg, I was searching for fast fsync & low cost NVMe SSD a few months ago, so I looked into consumer NVMe SSDs. Samsung 960 Pro was the first I tested, the results were just shockingly bad. It was so bad to the extent that I started to question whether my kernel/installation caused the slowness issue. Searched online and found a few of your posts talking about the exact same problem you saw. That saved me quite a bit time. :)

Yes, totally agree with the conclusion in your link above, consumer SSD (NVMe or not, high end or cheap) doesn't worth a dime. Cheers!

Post reply on HN