Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where's my petabyte disk drive?

bit-player.org

Post reply on HN
Print view View original

31–40 of 101 posts

Re: Where's my petabyte disk drive?

#31

Post by rgbrenner »

jsprogrammer wrote: post #29
rgbrenner wrote: post #10

The growth in hard drive space is called Kryder's Law [0] (like Moore's law). There's a paper from 2012 on the cost of long-term storage [1].. and here's a quote from the researchers blog[2]: Here is a graph I got from Dave Anderson [director of strategic planning at Seagate] years ago. It shows that what looks like a smooth Kryder's Law curve is actually the superposition of a series of S-curves, one for each succes…

So, the rate if increase (in density) is slowed by lack of demand for storage?

Sure, that's at least part of it.. since if demand is lower, that increases the time required to recoup the investment on new storage technology.

Re: Where's my petabyte disk drive?

#32

Post by virtuallynathan »

spullara wrote: post #20

He is wrong about 6TB being the biggest hard drive in the market. Now that SSDs have become the standard they are taking up the curve. Samsung announced a 16TB SSD last year. I don't expect people to devote as much effort to making spinning disks bigger. http://arstechnica.com/gadgets/2015/08/samsung-unveils-2-5-i...

There are also 8, 10, and 14TB SMR spinning disks on the market.

Re: Where's my petabyte disk drive?

#33

Post by simcop2387 »

jfoutz wrote: post #15
brianwawok wrote: post #6

Earlier quoted context omitted.

You assume HDs will die. Keep 3 copies of all data, at least 1 far away, and diff regularly. If a HD lasts 2 years vs 10 in average, doesn't really change the best practice to keep your data.

Just to elaborate a bit, it's pretty easy to keep three copies of stuff you care about. First, you have the working copy, on the machine itself. A big backup disk is pretty straightforward. I use time machine, which isn't super reliable, but it's very very easy. Now, when it comes down to it, do you really need to backup the OS? or your installed software? if the machine and the backups fail, you're going to be reins…

> Now, when it comes down to it, do you really need to backup the OS? or your installed software? if the machine and the backups fail, you're going to be reinstalling anyway (probably)

I'd counter this with what I do with my laptop. The OS is considerably smaller than the data I actually care about (<20gb), it takes almost no time to backup and so it leaves me with a very quick ability to restore the system to a known good state in the event of some kind of failure. I don't do constant backups of it, but maybe once a month i'll update the backup I have of the OS.

Re: Where's my petabyte disk drive?

#35

Post by tamana »

dalke wrote: post #23
PaulHoule wrote: post #2

Note in the early 1990s you could read hard drive content pretty easily with a scanning probe microscope, in fact the bits were lozenge shaped and if you looked close you might find the edges of bits that had been written before and weren't perfectly aligned. (i.e. scientists had tools in their labs that were better at reading a hard drive than the read head) Back then it was possible you could smash the plates and s…

"in the early 1990s you could read hard drive content pretty easily with a scanning probe microscope" I believe that is an urban legend. http://all.net/ForensicsPapers/2012-12-07-OverwrittenMagneti... describes attempts to track down such cases: > To date I have found no example of any instance in which digital data recorded on a hard disk drive and subsequently overwritten was recovered from such a drive since 1985,…

Parent said reading a drive:s current data is possible. You said that reading erased data is impossible.

Re: Where's my petabyte disk drive?

#37

Post by chris11 »

edward wrote: post #36

You can buy a 8TB HDD for $219.99, that's $27.50 per TB. I made a page that shows hard drives and SSDs sorted by price per TB: https://edwardbetts.com/price_per_tb/

PCPartPicker is also usually a really good source for comparing part prices. https://pcpartpicker.com/parts/internal-hard-drive/#sort=a7&...

Re: Where's my petabyte disk drive?

#38

Post by ChrisDutrow »

archiebunker wrote: post #4

Excellent post. Great information. I have a question about SSDs, though. Google has published their information about hard drive and SSD survival in their data centers. It can be viewed here: http://www.datacenterdynamics.com/servers-storage/googles-ss... So my question is how we mere mortals can deal with all the maintenance. It may well be that spinning hard drives are best for us.

I'm not sure spinning hard drives are more reliable outside of data centers - at least not for laptops because you are carrying them around (anecdotaly, I recently had en employee break a hard drive in a laptop after dropping it).

It probably doesn't really matter as much as your intuition might suggest anyway because each drive can and will fail. And they fail at similar rates (as opposed to an exponential difference of a factor of 10x or more). So you need to take similar precautions for each type of drive.

For personal use - I use exclusively SSDs because they are much faster. The I put all the information I don't want to lose in dropbox.

For servers, all data that is important goes in a database cluster (Cassandra) with a replication factor of 3. Those drives are backed up daily offsite. For data that cannot be lost at all (even a days worth), I also copy each record to Amazon S3 every time it is changed. - I'm sure there are many other ways to tackle this problem.

Re: Where's my petabyte disk drive?

#39

Post by mrob »

13of40 wrote: post #21
Kenji wrote: post #8

The reason why spinning disks still stick around is not just because of the price. Often, it is the case that you do not need 200-500MBit/s and ~100+ are enough. I have my OS on a (small) SSD and my spinning disks filled with my documents, music, movies. All of this does not need high speed, so it would be wasted money to buy an SSD for it. I'd rather buy twice the storage and do RAID1 (which I did).

It's a mystery to me that hybrid SSD+HD drives aren't more ubiquitous. I can guess what data is going to be read off the drive more frequently, but the computer can collect statistics and make a way more accurate prediction than I can.

I don't think statistics will make better predictions than I can. For example, consider a moderately large file (eg. 100MB), which is only ever read occasionally, by sequential low speed streaming. It seems reasonable to place it on the HD, but it's music, and I want it on the SSD so I can have the HD powered down for lower background noise when I'm listening to it. And I could have another almost identical file which is a podcast instead, and that should go on the HD because I don't care about minimum background noise when listening to podcasts, so even looking at the file type won't help. The correct place for a file depends on what you intend to do with it, not any measurable property.

Re: Where's my petabyte disk drive?

#40

Post by AnthonyMouse »

evmar wrote: post #9

Massive hard drives only useful for archival purposes, under this argument: if your hard drive is being used for live queries, you want to access all the data on it. Even if you have have all the pieces in place to stream 1gig per sec off of the drive, a 1pb drive would still take 1 million seconds = ~11.5 days to read. So in practice in production it's more useful to have smaller hard drives in more places to work o…

> Massive hard drives only useful for archival purposes, under this argument: if your hard drive is being used for live queries, you want to access all the data on it. Even if you have have all the pieces in place to stream 1gig per sec off of the drive, a 1pb drive would still take 1 million seconds = ~11.5 days to read.

Your premise is that anything you don't use on a regular basis belongs in archival storage, presumably in some kind of central archive.

Suppose there is 1PB of static data of which you access a different 50GB every day. You can call it archival if you like but it's still going to save you 50GB/day of network traffic to have a local copy.

So for example, Netflix could make a box that came loaded with all their content and new content is added using IP multicast or P2P during off-peak hours. The peak hours bandwidth savings would be immense and you would be completely immune to crappy or unreliable network connections.

Post reply on HN