Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

15TB HDDs: Western Digital Unveils the Ultrastar DC HC620

anandtech.com

Post reply on HN
Print view View original

101–110 of 123 posts

Re: 15TB HDDs: Western Digital Unveils the Ultrastar DC HC620

#101

Post by quasarj »

Dylan16807 wrote: post #100
quasarj wrote: post #97

So does anyone know of a way to take a set of files and write them to an HDD, in NTFS or exFAT format, in a single sequential write? Essentially building the FS on-the-fly (because we're talking about datsets that are much too large to fit into memory)?

So basically a zip file? I will note that you could easily build the MFT before you start transferring data. That's really your active 'dataset' here, and it's not very big.

Building the MFT first is pretty much what I want to do. But I'm not aware of any utilities that can handle it, nor where to start with writing it myself...

I have a project where I routinely need to copy large amounts of data (3 to 8 TB) to a hard drive. Problem is, my files are all 512kb. So this is much slower than it could be...

If I write it as a single tar file I get excellent throughput, but the users who need to be able to work with the drive are unable to handle a tar file. They need to be able to plug the drive into a Windows computer and have it "just work".. which presents some problems.

Re: 15TB HDDs: Western Digital Unveils the Ultrastar DC HC620

#102

Post by femto »

The 15TB drive packs in 1108 Gbit/inch2. That is, each bit is a square of side 8.5nm. This is small but the transistors in flash are smaller [1]. As mind blowing as the numbers (for both technologies) are in the referenced article, that article is now 2.5 years old. Is anyone aware of more recent numbers?

[1] https://www.computerworld.com/article/3030642/data-storage/f...

Re: 15TB HDDs: Western Digital Unveils the Ultrastar DC HC620

#103

Post by tfigment »

tracker1 wrote: post #92
nine_k wrote: post #24

Earlier quoted context omitted.

RAID6 should be fine rebuilding online (in RAID5 mode) even under a moderate write load. Of course one should source RAID disks form 3 different vendors, to ensure that they are from different batches, and are not going to fail at approximately the same time.

Good advice, though I once had about half a dozen drives (12 drive RAID Z2 with 2 as hot spares) fail within a few weeks of each other in separate batches from sourcing. (Seagate 3TB drives, I think there's been articles on how bad that series was).

I don't know how i survived those Seagates. Lasted maybe a year and started dropping like flies. Synology seems to recommend identical drives as i recall but work fine with different sizes and makers afaict.

Re: 15TB HDDs: Western Digital Unveils the Ultrastar DC HC620

#104

Post by proverbialbunny »

quasarj wrote: post #97

So does anyone know of a way to take a set of files and write them to an HDD, in NTFS or exFAT format, in a single sequential write? Essentially building the FS on-the-fly (because we're talking about datsets that are much too large to fit into memory)?

Using dd.

http://man7.org/linux/man-pages/man1/dd.1.html

Re: 15TB HDDs: Western Digital Unveils the Ultrastar DC HC620

#106

Post by bradknowles »

dktoao wrote: post #88
tallanvor wrote: post #26

Earlier quoted context omitted.

HAMR and then HDMR are expected to allow data densities to increase by 5 to 10 times what is currently achievable. HAMR will probably start showing up in a year or two. Spinning drives are definitely not going away anytime soon unless there is a much more significant drop in the cost of SSDs.

Multi-actuator drives are also coming in the next few years: https://blog.seagate.com/craftsman-ship/multi-actuator-techn... That will close the performance gap a bit too I suspect. I have to wonder about power consumption though.

Back when I was an intern at Imprimis (before they got bought by Seagate), I worked with the Manufacturing Engineers for the Wren series of 5.25” drives, including the Wren VII, which was the first consumer SCSI drive with a capacity of 1GB or more.

I looked at the drive actuators at the time, and I was incredulous when the guys told me that all operations were serial. I asked why they didn’t do parallel reads and writes, and I was told that technology was already common for mainframes but too expensive for consumer gear.

So, fast forward from 1989 to now, and I’m sure that idea will come back — sooner or later.

Re: 15TB HDDs: Western Digital Unveils the Ultrastar DC HC620

#107

Post by p1necone »

blihp wrote: post #61
johngalt wrote: post #53

Earlier quoted context omitted.

Tapes have a use case that hard drives do not. Tapes are the lowest cost/GB stored and are more shelf stable than hard drives. SSDs are higher performance than HDDs and have none of the packaging constraints. Flash storage is going to be put into everything and the economies of scale look quite good. Storage is scaling but the r/w speeds of hdds aren't keeping up. Following the trend line and we see huge hdds that ar…

You state that with an unwarranted degree of certainty. You're making the same argument, and mistake, that proponents of 'X is going to kill hard drives' have made for decades. There have been many 'this will be the death of hard drives' technologies over the decades: zip drives, optical drives, tape drives (there was a time when they were predicted to be everywhere... never happened), CD (then DVD) writers, etc. Not…

I don't know, SSDs have already replaced HDDs in consumer systems to a far greater extent than any of the things you've listed.

Re: 15TB HDDs: Western Digital Unveils the Ultrastar DC HC620

#108

Post by Dylan16807 »

proverbialbunny wrote: post #104
quasarj wrote: post #97

So does anyone know of a way to take a set of files and write them to an HDD, in NTFS or exFAT format, in a single sequential write? Essentially building the FS on-the-fly (because we're talking about datsets that are much too large to fit into memory)?

Using dd. http://man7.org/linux/man-pages/man1/dd.1.html

That's how you copy an existing filesytem. It's not how you take an arbitrary subset of files and put them into a new filesystem with minimal write amplification.

Re: 15TB HDDs: Western Digital Unveils the Ultrastar DC HC620

#109

Post by Lramseyer »

femto wrote: post #102

The 15TB drive packs in 1108 Gbit/inch2. That is, each bit is a square of side 8.5nm. This is small but the transistors in flash are smaller [1]. As mind blowing as the numbers (for both technologies) are in the referenced article, that article is now 2.5 years old. Is anyone aware of more recent numbers? [1] https://www.computerworld.com/article/3030642/data-storage/f...

From my experience in the HDD industry I remember that a magnetic bit of data is about 13-15nm long and about 40-60nm wide (narrower tracks for SMR.) The length of a bit is constrained by the grain size of the magnetic media. However, the width of a bit (prior to SMR) is actually constrained by the size of the write head. I don't remember why, but I think it has to do with the fact that the write current is like 40 mA, and the magnetic flux density on the write element is like 1.5T (no that's not a typo)

I'm not an expert on transistor pitches, but here's a chart from Wikipedia for the 10nm - https://en.wikipedia.org/wiki/10_nanometer It's kind of impressive for HDDs considering that it's a 2 inch long mechanical arm that is able to move with that level of precision.

Re: 15TB HDDs: Western Digital Unveils the Ultrastar DC HC620

#110

Post by AgentME »

quasarj wrote: post #101
Dylan16807 wrote: post #100

Earlier quoted context omitted.

So basically a zip file? I will note that you could easily build the MFT before you start transferring data. That's really your active 'dataset' here, and it's not very big.

Building the MFT first is pretty much what I want to do. But I'm not aware of any utilities that can handle it, nor where to start with writing it myself... I have a project where I routinely need to copy large amounts of data (3 to 8 TB) to a hard drive. Problem is, my files are all 512kb. So this is much slower than it could be... If I write it as a single tar file I get excellent throughput, but the users who need…

Maybe you could use the same type of filesystem that file CD/DVDs use (UDF?). They're written sequentially and are commonly supported.
Post reply on HN