Where's my petabyte disk drive? • Hacker News Board

Re: Where's my petabyte disk drive?

#61

Post by douche » Sun, Mar 27, 2016, 9:18 PM UTC

Razengan wrote: post #55

jsprogrammer wrote: post #29

Earlier quoted context omitted.

So, the rate if increase (in density) is slowed by lack of demand for storage?

> lack of demand for storage and in turn, demand for storage is slowed by lack of need for storage. The real question should be: Where's my 4K video? My 10-bit-channel images? My lossless audio? Yes they are here but not nearly as common as content delivered in ancient formats from over a decade ago. For example go on a wallpapers subreddit, the vast majority is still in 8-bit 1920x1080 JPEG. Video is still mostly 10…

For a lot of things, that extra resolution is not really any benefit. Particularly for content that wasn't ever recorded at that level of fidelity anyway - I don't need 4k versions of Seinfeld re-runs, or SpongeBob SquarePants for the kiddos. Most of that stuff doesn't even need to be 720p.

Besides, pirated video content was really the only thing that most normal people could fill up their hard drives with (okay, maybe GoPro people or people with huge Steam libraries, too), but the Netflix and YouTube and Amazon Prime have taken a huge chunk out of that.

Re: Where's my petabyte disk drive?

#62

Post by capitalsigma » Sun, Mar 27, 2016, 9:21 PM UTC

abecedarius wrote: post #25

> Is this notion of merging memory and storage an attractive prospect or a nightmare? I’m not sure. There are some huge potential problems. For safety and sanity we generally want to limit which programs can alter which documents. Those rules are enforced by the file system, and they would have to be re-engineered to work in the memory-mapped environment. This was done back in the 80s in http://www.cis.upenn.edu/~Key…

I'm not sure his idea about "merging memory and storage" really makes sense. He says that he wants load instructions to be able to hit the disk in order to avoid "calls to input/output routines in the operating system." But you can't avoid the input/output routines --- he's effectively saying that we should hardcode our filesystems into a single machine instruction and let the processor figure it out. If anything, we're moving farther from this model, since VMs give us virtual address spaces inside virtual address spaces.

I don't think this is just a security issue; it really breaks all of the assumptions that we like to make in modern programming languages.

Re: Where's my petabyte disk drive?

#63

Post by capitalsigma » Sun, Mar 27, 2016, 9:22 PM UTC

brianwawok wrote: post #6

archiebunker wrote: post #4

Excellent post. Great information. I have a question about SSDs, though. Google has published their information about hard drive and SSD survival in their data centers. It can be viewed here: http://www.datacenterdynamics.com/servers-storage/googles-ss... So my question is how we mere mortals can deal with all the maintenance. It may well be that spinning hard drives are best for us.

You assume HDs will die. Keep 3 copies of all data, at least 1 far away, and diff regularly. If a HD lasts 2 years vs 10 in average, doesn't really change the best practice to keep your data.

Except you let the machine handle that for you with RAID.

Re: Where's my petabyte disk drive?

#64

Post by scottlocklin » Sun, Mar 27, 2016, 9:33 PM UTC

rgbrenner wrote: post #10

The growth in hard drive space is called Kryder's Law [0] (like Moore's law). There's a paper from 2012 on the cost of long-term storage [1].. and here's a quote from the researchers blog[2]: Here is a graph I got from Dave Anderson [director of strategic planning at Seagate] years ago. It shows that what looks like a smooth Kryder's Law curve is actually the superposition of a series of S-curves, one for each succes…

FWIIW; worked in Kryder's lab at Hammerschlag Hall as an undergraduate in the 90s: he's one of the best people I've met in the sciences. The E.O. Lawrence of hard drive technology.

Re: Where's my petabyte disk drive?

#65

Post by jstsch » Sun, Mar 27, 2016, 9:35 PM UTC

Razengan wrote: post #55

jsprogrammer wrote: post #29

Earlier quoted context omitted.

So, the rate if increase (in density) is slowed by lack of demand for storage?

> lack of demand for storage and in turn, demand for storage is slowed by lack of need for storage. The real question should be: Where's my 4K video? My 10-bit-channel images? My lossless audio? Yes they are here but not nearly as common as content delivered in ancient formats from over a decade ago. For example go on a wallpapers subreddit, the vast majority is still in 8-bit 1920x1080 JPEG. Video is still mostly 10…

If VR gets popular, it might be a driver for higher storage demands. For non-foveated imagery 4K is even quite limited.

Re: Where's my petabyte disk drive?

#66

Post by Houshalter » Sun, Mar 27, 2016, 9:38 PM UTC

douche wrote: post #61

Razengan wrote: post #55

Earlier quoted context omitted.

> lack of demand for storage and in turn, demand for storage is slowed by lack of need for storage. The real question should be: Where's my 4K video? My 10-bit-channel images? My lossless audio? Yes they are here but not nearly as common as content delivered in ancient formats from over a decade ago. For example go on a wallpapers subreddit, the vast majority is still in 8-bit 1920x1080 JPEG. Video is still mostly 10…

For a lot of things, that extra resolution is not really any benefit. Particularly for content that wasn't ever recorded at that level of fidelity anyway - I don't need 4k versions of Seinfeld re-runs, or SpongeBob SquarePants for the kiddos. Most of that stuff doesn't even need to be 720p. Besides, pirated video content was really the only thing that most normal people could fill up their hard drives with (okay, may…

Yes that's an important point. There are diminishing returns with better resolutions. Twice the resolution doesn't mean it appears twice as good, and at higher resolutions it can even be hard to tell. At some point you exceed the resolution of the human eye and ear.

But I think storage space still matters on mobile. I haven't come close to filling up my laptop after 4 years of use, but I have to be careful about putting stuff on my iPod. Still it can store dozens of hours of audio, so it's not the biggest issue.

Re: Where's my petabyte disk drive?

#67

Post by coldtea » Sun, Mar 27, 2016, 9:54 PM UTC

Razengan wrote: post #55

jsprogrammer wrote: post #29

Earlier quoted context omitted.

So, the rate if increase (in density) is slowed by lack of demand for storage?

> lack of demand for storage and in turn, demand for storage is slowed by lack of need for storage. The real question should be: Where's my 4K video? My 10-bit-channel images? My lossless audio? Yes they are here but not nearly as common as content delivered in ancient formats from over a decade ago. For example go on a wallpapers subreddit, the vast majority is still in 8-bit 1920x1080 JPEG. Video is still mostly 10…

>Yes they are here but not nearly as common as content delivered in ancient formats from over a decade ago. For example go on a wallpapers subreddit, the vast majority is still in 8-bit 1920x1080 JPEG. Video is still mostly 1080. The leading music services still deliver in 2-channel lossy formats. And this is 2016.

Ever experienced YouTube buffering on an average connection? And that's for the metropolitan US -- consider the developing world, or even rural US, the mobile internet caps, etc.

Not to mention that even if you have enough speed, still you don't get much benefit from 4K video for 99% of stuff out there. Diminishing returns. I should know, I have projected good 1080p video to movie theaters, and nobody would call it bad or inadequate. On an average 15-27" monitor? It's not even an issue...

Re: Where's my petabyte disk drive?

#68

Post by LoSboccacc » Sun, Mar 27, 2016, 10:07 PM UTC

jsprogrammer wrote: post #29

rgbrenner wrote: post #10

The growth in hard drive space is called Kryder's Law [0] (like Moore's law). There's a paper from 2012 on the cost of long-term storage [1].. and here's a quote from the researchers blog[2]: Here is a graph I got from Dave Anderson [director of strategic planning at Seagate] years ago. It shows that what looks like a smooth Kryder's Law curve is actually the superposition of a series of S-curves, one for each succes…

So, the rate if increase (in density) is slowed by lack of demand for storage?

Also form factor is getting smaller

Re: Where's my petabyte disk drive?

#69

Post by lwf » Sun, Mar 27, 2016, 10:20 PM UTC

ladzoppelin wrote: post #58

ChrisDutrow wrote: post #38

Earlier quoted context omitted.

I'm not sure spinning hard drives are more reliable outside of data centers - at least not for laptops because you are carrying them around (anecdotaly, I recently had en employee break a hard drive in a laptop after dropping it). It probably doesn't really matter as much as your intuition might suggest anyway because each drive can and will fail. And they fail at similar rates (as opposed to an exponential differenc…

"The I put all the information I don't want to lose in dropbox." I don't think its healthy to consider cloud storage backup. For example I am pretty sure you could loose everything to Ransomware.

In re the Ransomware point, Dropbox provides all users with access to snapshots of their files going back 30 days:

https://www.dropbox.com/en/help/11

So, as long as you notice before the clock runs out, you should be able to recover from ransomware via file history.

[I work for Dropbox, but am not speaking on behalf of my employer.]

Re: Where's my petabyte disk drive?

#70

Post by ValentineC » Sun, Mar 27, 2016, 10:33 PM UTC

lwf wrote: post #69

ladzoppelin wrote: post #58

Earlier quoted context omitted.

"The I put all the information I don't want to lose in dropbox." I don't think its healthy to consider cloud storage backup. For example I am pretty sure you could loose everything to Ransomware.

In re the Ransomware point, Dropbox provides all users with access to snapshots of their files going back 30 days: https://www.dropbox.com/en/help/11 So, as long as you notice before the clock runs out, you should be able to recover from ransomware via file history. [I work for Dropbox, but am not speaking on behalf of my employer.]

I know Dropbox sells Extended Version History [1] as an add-on, but it'll be awfully nice if Pro users had maybe 60 days of file history, versus 30 days for free. Just a thought.

[1] https://www.dropbox.com/en/help/113