Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Storing 25 petabytes of Megaupload data costs us $9,000 a day

arstechnica.com

Post reply on HN
Print view View original

91–100 of 138 posts

Re: Storing 25 petabytes of Megaupload data costs us $9,000 a day

#91

Post by jlawer »

Tuna-Fish wrote: post #89
jlawer wrote: post #22

The costs of moving that amount of data is crazy. I am surprised the government hasn't seized the hardware, and chucked it in a warehouse. I did some back of an envelope calculations... and its absolutely crazy. Tape would require over 17,000 Ultrium tapes. Now you could De-dupe... but the hardware to process and dedupe that much data.... not really an option. Not to mention the time to write that many tapes... Somet…

> Now you could De-dupe... but the hardware to process and dedupe that much data.... not really an option. FYI, the data is already comprehensively de-duped.

I know they dedupe on the file level, but I wonder if they are doing block level deduping... as without a big shared storage infrastructure block level deduping becomes pretty hard to serve at high speed from as the reads potentially become distributed across hundreds of nodes...

To build out web scale systems you generally use commodity gear and accept the overhead of duplication, heavy deduping requires massive IO, and there is no way i can see you can be dealing with that much data have that level of IOPS and be profitable charging what they charge.

Re: Storing 25 petabytes of Megaupload data costs us $9,000 a day

#92

Post by ordinary »

fruchtose wrote: post #27
dos1 wrote: post #2

In my mind, the MPAA is certainly the best choice to pay these costs. They're the ones with the problem, they should be the ones to bear the burden. Especially considering Megaupload offered to take the data and they explicitly forbade it. If the MPAA didn't like the solutions offered, but can't come up with something better, then I think Carpathia should get to do what it wants. Edit: The opinion above has NO legal…

There's no legal basis for the MPAA to bear the cost of storing the data. The MPAA is not even a party to the proceedings, which, by the way, are criminal and not at all civil.

So this confuses me a little, I would be glad if someone could explain it to me: why does the MPAA get a voice in the proceedings, if they are not a party to it?

Re: Storing 25 petabytes of Megaupload data costs us $9,000 a day

#93

Post by samstave »

samstave wrote: post #77
VikingCoder wrote: post #70

Help me out with the math here: 1 terabyte costs them $128.41 per year, right? Amazon S3 would cost them roughly $444 per year, if they were using the Reduced Redundancy Storage. The cheapest HD that I see on pcpartpicker (in terms of Price/GB) is the Western Digital Caviar Green 2.5 TB (5400 RPM) for $135.43, which is $0.054/GB. That's $54.17 per TB. If you want a single backup, that's $108.34 per TB. Two backups (3…

Yeah, I think you are exactly right. This, at 9K/day for that much data appears really good. That is ~$.028 cents PER TB PER DAY to store it. Christ, I am paying Dropbox charges me, at $10/MONTH for 50GB the equivalent of $6.67 PER DAY in storage costs. For my 50GB, I am charged $.33 per day.

I don't know why I was downvoted, but the $6.67 per day cost is PER TERABYTE

I think whomever downvoted that thought I was referring to the per GB cost.

Re: Storing 25 petabytes of Megaupload data costs us $9,000 a day

#94

Post by Duff »

shrike wrote: post #81

The federal government does have a process for this sort of thing, if they seize an alleged drug dealer's house and that house has a mortgage the United States Marshals Service will pay the mortgage. If they seize cars, furniture, other assets the government is responsible for the storage of those items until the case has been resolved. [1] I would guess that MegaUpload's lawyers will make the claim that the data on…

You hit it on the head. I used to have to deal with alot of litigation-related preservation of data... we started calling discovery "Mutually Assured Destruction".

The lawyers basically try to make things more and more onerous in order to encourage a settlement. It's amusing, as long as you aren't accountable for the data!

Re: Storing 25 petabytes of Megaupload data costs us $9,000 a day

#95

Post by lotu »

Karunamon wrote: post #66
jlawer wrote: post #32

Earlier quoted context omitted.

You want to pull 1,103 server's disks? The compatibility problems you have trying to get data off a disks in a hardware raid make it impractical (do you have the EXACT SAME version hardware revision & firmware; without this you can't guarantee you can read it back)? Its either that or you have to pay data recovery guys to rebuild it. Not to mention hard drive costs are still high, post thai floods. For enterprise gea…

>The compatibility problems you have trying to get data off a disks in a hardware raid make it impractical (do you have the EXACT SAME version hardware revision & firmware; without this you can't guarantee you can read it back)? Its either that or you have to pay data recovery guys to rebuild it. Forgive me for for sounding like a member of Anonymous but.. So? That's the government's problem. I don't see why a privat…

Carpathia wants to be paid $9000 a day like they were before the Megaupload case started and pulling the drives and telling the FBI that it's their problem doesn't do that.

Re: Storing 25 petabytes of Megaupload data costs us $9,000 a day

#96

Post by Duff »

rmc wrote: post #50
xymostech wrote: post #19

Earlier quoted context omitted.

But instead of just losing use of those servers, they have to actively maintain them. It looks like they just want someone to pay for the $9000 per day that it costs to keep the servers running, they aren't looking for money that they lost from not being able to use the servers. To go with your analogy, sure building owners aren't allowed to rent the space back out, but they most certainly not asked to pay for usual…

Why are the servers still running? Surely it's easier and cheaper to just turn off the servers, power down the racks and lock the cages?

The running cost is just part of the equation... those servers are depreciating away and generating no revenue!

Re: Storing 25 petabytes of Megaupload data costs us $9,000 a day

#97

Post by ericd »

Why are options that would destroy any chance at Megaupload conducting business in the future even on the table before a trial is finished? I suppose a large amount of damage is already done, but it would be a gross injustice to kill their business before anything started. The government should pay to keep this up until they've conducted their trial. If they don't, and they lose somehow, I hope they get hit with a massive countersuit.

Re: Storing 25 petabytes of Megaupload data costs us $9,000 a day

#98

Post by mbell »

devicenull wrote: post #83
samstave wrote: post #49

Earlier quoted context omitted.

Assuming 2TB internal HDD's at $120.00 per piece, you would need $1.5MM worth of drives to transfer this to. Not including the man-hours associated with doing so. The $3.2MM per year is actually sounding a lot more reasonable to keep it up and running.

Don't forget replacement RAID controllers! The RAID config is usually dependent on the exact drive/controller combination, so removing the drives and not saving the controller may result in them becoming inaccessible. At the very least, they would almost certainly not pass requirements to be used as evidence if that happened.

I really hope they weren't backing 25TB of data with RAID controllers, this is well out of RAID league and well into the territory of distributed file system techniques. I ditched RAID years ago for ZFS for only an 18TB file server at home. Start talking about thousands of servers and RAID is pretty useless.

That said, IANAL but I think the interesting thing here is that just making copies of the hard drives is likely not enough from a evidence standpoint. With such a data set there are likely layers of data abstraction software that would be need to be replicated to make any sense of it.

Re: Storing 25 petabytes of Megaupload data costs us $9,000 a day

#99

Post by Retric »

deadmike wrote: post #67
rmc wrote: post #50

Earlier quoted context omitted.

Why are the servers still running? Surely it's easier and cheaper to just turn off the servers, power down the racks and lock the cages?

I'm wondering this, too. The data would still be there, and they wouldn't have to worry about maintenance costs at least, right?

Shutting them down could easily destroy evidence depending on how it's done. EX: Many redundant systems try replicating data from machines that are taken off line. What happens when the full network is taken off line quickly has probably not been tested.

Re: Storing 25 petabytes of Megaupload data costs us $9,000 a day

#100

Post by Karunamon »

lotu wrote: post #95
Karunamon wrote: post #66

Earlier quoted context omitted.

>The compatibility problems you have trying to get data off a disks in a hardware raid make it impractical (do you have the EXACT SAME version hardware revision & firmware; without this you can't guarantee you can read it back)? Its either that or you have to pay data recovery guys to rebuild it. Forgive me for for sounding like a member of Anonymous but.. So? That's the government's problem. I don't see why a privat…

Carpathia wants to be paid $9000 a day like they were before the Megaupload case started and pulling the drives and telling the FBI that it's their problem doesn't do that.

No, but it stops the costs from accruing. Legal action could then be brought to recover their other costs.
Post reply on HN