ST3000DM001 • Hacker News Board

Re: ST3000DM001

#171

Post by ta988 » Mon, Jun 07, 2021, 1:22 PM UTC

madaxe_again wrote: post #138

edward wrote: post #19

In 2012 I was working at the Internet Archive. I remember the 2011 Thailand floods restricted the supply of hard disks and pushed up prices. The best deal we could find was buying these Seagate hard disks as external drives at retail from Costco. We picked up a truck load. Then had somebody with a screw driver remove them all from the enclosures and install them in new servers. The error rate was high, the drives did…

I did precisely this too, as I needed storage for a few thousand hours of video footage I’d shot for a documentary, needed it now, and nobody had anything other than external HDDs in any kind of useful size. I was really careful - I had two NASes, each with raid 5 volumes holding a full copy of the footage, as it was irreplaceable. I got up one morning to find a disc had failed in one of them. Swapped it. During the…

That's the reason I never put the same brands/models in RAID. It is logistically a bit more complicated, but I've often noticed failures like that really close in time.

Re: ST3000DM001

#172

Post by flyinghamster » Mon, Jun 07, 2021, 1:22 PM UTC

geocrasher wrote: post #31

dheera wrote: post #26

Earlier quoted context omitted.

Back in the day I used IBM drives and they were super reliable. Too bad they stopped making hard drives and all the Maxtors, Seagates, Hitachis, and WDs came along with all their crap.

In the early to mid 90's, Maxtor was my go-to hard drive for any build. They went downhill after that, sadly.

Back in those days, Maxtor was on my good list and WD was definitely not. At work, we had to deal with prematurely failed WDs on our desktops all too often. It took a very long time before I'd consider a WD drive again, and then in the 2000s I finally tried them again. I've been mostly happy, but now I look kind of side-eyed at them after WD Red SMR fiasco.

I was none too pleased when Seagate bought out Maxtor and turned it into their lowball brand.

Re: ST3000DM001

#173

Post by rightbyte » Mon, Jun 07, 2021, 1:30 PM UTC

madaxe_again wrote: post #138

edward wrote: post #19

In 2012 I was working at the Internet Archive. I remember the 2011 Thailand floods restricted the supply of hard disks and pushed up prices. The best deal we could find was buying these Seagate hard disks as external drives at retail from Costco. We picked up a truck load. Then had somebody with a screw driver remove them all from the enclosures and install them in new servers. The error rate was high, the drives did…

I did precisely this too, as I needed storage for a few thousand hours of video footage I’d shot for a documentary, needed it now, and nobody had anything other than external HDDs in any kind of useful size. I was really careful - I had two NASes, each with raid 5 volumes holding a full copy of the footage, as it was irreplaceable. I got up one morning to find a disc had failed in one of them. Swapped it. During the…

Ye copying sick HDs always make me nervous. It is like taking your really old and scrappy car for a last ride to the salvage yard going over the whole country.

Re: ST3000DM001

#174

Post by px43 » Mon, Jun 07, 2021, 1:34 PM UTC

Sebb767 wrote: post #148

K0balt wrote: post #99

Earlier quoted context omitted.

At the time it was kind of an open secret in the industry. The floods were a convenient explanation, but a very large order from a client with a confidentiality clause ( very common in the industry) was widely known to be the main reason for short supply. It is possible that this client was someone else other than the NAS er... NSA but it is not very likely and at the time it was conventional wisdom among insiders th…

> but a very large order from a client with a confidentiality clause ( very common in the industry) was widely known to be the main reason for short supply. So, just for the fun of it: Forbes estimates a capacity (at the time) of up to 12 EB [1]. Assuming they used 3TB drives, they'd need 4 million drives for this [2]. This article [3] shows the market for consumer drives at about 150 million in 2013 (figure 7). So t…

Does 12 EB seem super low to anyone else?

I know of people who were delivering software to certain customers that were required/intended to work on EB scale datasets over a decade ago, and this was well before the Utah data center was a thing.

Re: ST3000DM001

#175

Post by rightbyte » Mon, Jun 07, 2021, 1:36 PM UTC

jeegsy wrote: post #147

monadgonad wrote: post #84

Earlier quoted context omitted.

Colordrops said, "The Thailand story was a cover." This is the allegation. Not that the NSA bought a lot of drives and kept it secret, but that this purchase alone rather than the floods were responsible for the drive storage, and the flood story was fabricated. That is the alleged conspiracy.

I find that a number of conspiracy theories are 'debunked' by creating an obviously ridiculous strawman/rendition of the theory that nobody actually claimed.

Also counspiracy nutcases are a conspiracy to hide conspiracies by ridiculing.

Re: ST3000DM001

#176

Post by salawat » Mon, Jun 07, 2021, 1:38 PM UTC

pas wrote: post #93

betwixthewires wrote: post #89

Earlier quoted context omitted.

> The plausibility of a conspiracy theory diminishes as the number of people necessary to keep the secret increases. While I don't know anything about the theory being discussed here, the pervasive belief in Santa Clause among children disproves your point entirely.

Because of Big Santa (ie. parents). And that promptly fails when the cost of maintaining the Santa system gets too big (ie. as the child's mental faculties develop, plus as the child interacts with the outside world not controlled by the parents). So unless we all are living in a very well orchestrated NSA Truman show, it's unlikely that your argument applies.

Perhaps you should read Hogfather or Making Money by Terry Pratchett. In which the importance of the lies that society tells itself is touched upon.

"All right," said Susan. "I'm not stupid. You're saying humans need... fantasies to make life bearable."

REALLY? AS IF IT WAS SOME KIND OF PINK PILL? NO. HUMANS NEED FANTASY TO BE HUMAN. TO BE THE PLACE WHERE THE FALLING ANGEL MEETS THE RISING APE.

"Tooth fairies? Hogfathers? Little—"

YES. AS PRACTICE. YOU HAVE TO START OUT LEARNING TO BELIEVE THE LITTLE LIES.

"So we can believe the big ones?"

YES. JUSTICE. MERCY. DUTY. THAT SORT OF THING.

"They're not the same at all!"

YOU THINK SO? THEN TAKE THE UNIVERSE AND GRIND IT DOWN TO THE FINEST POWDER AND SIEVE IT THROUGH THE FINEST SIEVE AND THEN SHOW ME ONE ATOM OF JUSTICE, ONE MOLECULE OF MERCY. AND YET—Death waved a hand. AND YET YOU ACT AS IF THERE IS SOME IDEAL ORDER IN THE WORLD, AS IF THERE IS SOME...SOME RIGHTNESS IN THE UNIVERSE BY WHICH IT MAY BE JUDGED.

"Yes, but people have got to believe that, or what's the point—"

MY POINT EXACTLY -Hogfather

For a TL:DR of Making Money, there's

https://www.ft.com/content/1aebc9f8-70bb-11dc-98fc-0000779fd...

I'd still recommend the book though. There are so many things that are little more than flaky veneers taken for granted in adult human life that the fact anything works at all generally comes down to an intentioned group of people 'conspiring' to make it so, and no one ever bothers to ask about it, as it 'just werkz' and creates value. It's only a problem when too many externalities get ignored, in which case, everyone starts to notice.

Once you understand this, the beauty of the 'conspiracy theorist' meme coming out of an intelligence agency to get people to stop taking those who ask inconvenient questions starts to take on a stroke of genius. You get people who rock the boat discredited out the gate, and reinforce the status quo.

...Until that document gets declassified and blows up in your face anyway.

Re: ST3000DM001

#177

Post by userbinator » Mon, Jun 07, 2021, 1:40 PM UTC

londons_explore wrote: post #76

So why did this drive fail? Has nobody dissected it to figure out exactly which component no longer functioned correctly?

A search through data recovery forums like hddguru.com (they tend to have a secretive "trade union" attitude, but some useful stuff leaks through...) and the Internet reveals some more info like this:

https://www.recoveryforce.ca/forums/viewtopic.php?f=12&t=157

Most of them seem to be head problems, and there are some outright head crashes too. Seems like bad media (leading to head failure) is mainly to blame:

https://www.cheadledatarecovery.co.uk/2016/02/seagate-dm001-...

It sounds vaguely reminiscent of the IBM DeathStars, which failed in a similar fashion.

Re: ST3000DM001

#178

Post by DominoTree » Mon, Jun 07, 2021, 1:43 PM UTC

edward wrote: post #19

In 2012 I was working at the Internet Archive. I remember the 2011 Thailand floods restricted the supply of hard disks and pushed up prices. The best deal we could find was buying these Seagate hard disks as external drives at retail from Costco. We picked up a truck load. Then had somebody with a screw driver remove them all from the enclosures and install them in new servers. The error rate was high, the drives did…

During the current IC shortage, I've been buying quantities of assembled breakout boards that have chips I need and desoldering them.

In many cases it's actually cheaper than buying the bare ICs at their normal price.

Re: ST3000DM001

#179

Post by userbinator » Mon, Jun 07, 2021, 1:46 PM UTC

flyinghamster wrote: post #146

londons_explore wrote: post #100

Earlier quoted context omitted.

I think it's sad that the hacker/maker community doesn't have enough knowledge/skill/inclination to disassemble this kind of thing and figure it out for themselves. It's a bad sign when tech complexity gets out of the expertise of a hobbyist - I believe that's the start of a stagnation of that technology as new young engineers never get excited about disassembling and modding hard drives in their free time, and in tu…

Equipment cost is a huge problem there. You need a proper clean room if you're going to tear apart HDDs and expect them to work again when you reassemble them. Not to mention, the extremely precise tolerances involved would make mechanical mods a tall order indeed. Firmware mods would be more doable, assuming it isn't locked down.

WDs tend to be a bit more durable... especially in smaller capacities:

http://www.brendangregg.com/blog/2021-05-09/poor-disk-perfor...

Re: ST3000DM001

#180

Post by wholinator2 » Mon, Jun 07, 2021, 1:47 PM UTC

Sebb767 wrote: post #148

K0balt wrote: post #99

Earlier quoted context omitted.

At the time it was kind of an open secret in the industry. The floods were a convenient explanation, but a very large order from a client with a confidentiality clause ( very common in the industry) was widely known to be the main reason for short supply. It is possible that this client was someone else other than the NAS er... NSA but it is not very likely and at the time it was conventional wisdom among insiders th…

> but a very large order from a client with a confidentiality clause ( very common in the industry) was widely known to be the main reason for short supply. So, just for the fun of it: Forbes estimates a capacity (at the time) of up to 12 EB [1]. Assuming they used 3TB drives, they'd need 4 million drives for this [2]. This article [3] shows the market for consumer drives at about 150 million in 2013 (figure 7). So t…

I have a feeling that just comparing the numbers without looking at any of the logistics is just a convenient way to dismiss a theory you don't like.

There's all kinds of things to think about like how 2% may be less than 100% but in an elastic supply of a globally distributed good, 2% of (total yearly) supply called upon all at once can be a _huge_ departure from the norm. Maybe the shortage isn't just a hard disk shortage but a shortage of shipping space to take those hard drives anywhere else but to the NSA. Maybe it's that the factory(s) don't make 100% of the drives in an instant and then just distribute them for the rest of the year, but slowly make a constant stream of drives, all of which were swallowed up for a period of time by the massive order, causing the shortage.

There's a million other factors besides just "this number smaller that this number".