Live data from Hacker News

Where's my petabyte disk drive?

bit-player.org

11–20 of 101 posts

Re: Where's my petabyte disk drive?

#11
post #9

Massive hard drives only useful for archival purposes, under this argument: if your hard drive is being used for live queries, you want to access all the data on it. Even if you have have all the pieces in place to stream 1gig per sec off of the drive, a 1pb drive would still take 1 million seconds = ~11.5 days to read. So in practice in production it's more useful to have smaller hard drives in more places to work o…

This is an important point. The Google white paper does address it, though only in the abstract: They want to maximize both capacity and IO bandwidth. One can imagine large disks with multiple independently actuated head assemblies. But quite possibly you're right that it's not worth the both.

Re: Where's my petabyte disk drive?

#12
Why did he write '\(2\frac{1}{2}\) or \(3\frac{1}{2}\)' (which presumably uses JavaScript to be rendered), when he could have just written '2½ or 3½'? It'd have taken 8 bytes to store instead of 36 — and that's not even counting the size of the JavaScript!

As long as folks keep on reinventing the wheel, only bigger, hard drives are going to have to keep increasing in size.

Re: Where's my petabyte disk drive?

#13
post #8

The reason why spinning disks still stick around is not just because of the price. Often, it is the case that you do not need 200-500MBit/s and ~100+ are enough. I have my OS on a (small) SSD and my spinning disks filled with my documents, music, movies. All of this does not need high speed, so it would be wasted money to buy an SSD for it. I'd rather buy twice the storage and do RAID1 (which I did).

It's still the price in a way. If they would cost the same, you would choose the SSD.

Re: Where's my petabyte disk drive?

#14
post #7

We appear to be slashdotting the server; here's some cached copies: Just text (loads instantly): http://webcache.googleusercontent.com/search?q=cache:Jr34hZX... Images etc. loaded from the site (seems to work, albeit slowly): http://webcache.googleusercontent.com/search?q=cache:Jr34hZX...

>

I found the problem.

Re: Where's my petabyte disk drive?

#15

Excellent post. Great information. I have a question about SSDs, though. Google has published their information about hard drive and SSD survival in their data centers. It can be viewed here: http://www.datacenterdynamics.com/servers-storage/googles-ss... So my question is how we mere mortals can deal with all the maintenance. It may well be that spinning hard drives are best for us.

You assume HDs will die. Keep 3 copies of all data, at least 1 far away, and diff regularly. If a HD lasts 2 years vs 10 in average, doesn't really change the best practice to keep your data.

Just to elaborate a bit, it's pretty easy to keep three copies of stuff you care about. First, you have the working copy, on the machine itself. A big backup disk is pretty straightforward. I use time machine, which isn't super reliable, but it's very very easy.

Now, when it comes down to it, do you really need to backup the OS? or your installed software? if the machine and the backups fail, you're going to be reinstalling anyway (probably) So, for the third copy i rely on 3rd parties. Different people have different needs, you might want to do something fancy in house.

It pretty much boils down to finding a service for your stuff. I have a couple of private github repos. Photos on iCloud and whatever Alphabet is calling Picasa these days. 20 gigs of music to Amazon or Alphabet (or both). Administrative stuff, like taxes, i just email to myself. It's probably smarter to keep that in dropbox or something along those lines.

The key point is, there are the things you make or capture that are irreplaceable, save those lots of places. There's a bunch of other crap on your computer to make it be useful. That stuff is trivial to reinstall. Well, ok, it might cost you a day or two to redownload and reconfigure emacs just so - but with a little planning you can put that config in git, so it's easy to restore or set up on a new machine.

It's almost better to think in terms of, if i had to upgrade tomorrow, what would i need to copy over? that's the stuff to be really fussy about.

Re: Where's my petabyte disk drive?

#16
post #12

Why did he write '\(2\frac{1}{2}\) or \(3\frac{1}{2}\)' (which presumably uses JavaScript to be rendered), when he could have just written '2½ or 3½'? It'd have taken 8 bytes to store instead of 36 — and that's not even counting the size of the JavaScript! As long as folks keep on reinventing the wheel, only bigger, hard drives are going to have to keep increasing in size.

If you already have the maths rendering script in place for more complicated uses, and you don't already know a quick way to insert the ½ character, then it's just the path of least resistance.

Re: Where's my petabyte disk drive?

#17
post #12

Why did he write '\(2\frac{1}{2}\) or \(3\frac{1}{2}\)' (which presumably uses JavaScript to be rendered), when he could have just written '2½ or 3½'? It'd have taken 8 bytes to store instead of 36 — and that's not even counting the size of the JavaScript! As long as folks keep on reinventing the wheel, only bigger, hard drives are going to have to keep increasing in size.

I'm not sure that using LaTeX markup, a format that predates HTML, counts as "reinventing the wheel".

Re: Where's my petabyte disk drive?

#18

Excellent post. Great information. I have a question about SSDs, though. Google has published their information about hard drive and SSD survival in their data centers. It can be viewed here: http://www.datacenterdynamics.com/servers-storage/googles-ss... So my question is how we mere mortals can deal with all the maintenance. It may well be that spinning hard drives are best for us.

You assume HDs will die. Keep 3 copies of all data, at least 1 far away, and diff regularly. If a HD lasts 2 years vs 10 in average, doesn't really change the best practice to keep your data.

And prefer filesystems which actively check data integrity.

Re: Where's my petabyte disk drive?

#19
the main speed boost back then was innovations landing to usable drives. We are currently in a innovation phase with a whole new space to explore, once the winners of that innovating gets picked, refining happens, cost savings are found, etc, I think we'll probably see some return to the expectations

that being said, for the slashdotting issue, I can't see how bad the graph is

Re: Where's my petabyte disk drive?

#20
He is wrong about 6TB being the biggest hard drive in the market. Now that SSDs have become the standard they are taking up the curve. Samsung announced a 16TB SSD last year. I don't expect people to devote as much effort to making spinning disks bigger.

http://arstechnica.com/gadgets/2015/08/samsung-unveils-2-5-i...

Post reply on HN