Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dropbox’s Exodus from the Amazon Cloud

wired.com

Post reply on HN
Print view View original

131–140 of 252 posts

Re: Dropbox’s Exodus from the Amazon Cloud

#131

Post by jamwt »

photonwins wrote: post #126

I am curious to know, with so many disks densely packed in a 4U configuration, I am guessing there is definitely increased heat generation and not to mention vibration. How do you handle these? Also, does it have any effect on MTBF?

Yeah, our hardware teams put a lot of qualification time into hardware profiles before we buy in bulk. And part of what we test is power utilization, heat, etc, under normal software loads vs extreme loads, to make sure we don't exceed power budget, don't spin everything up at the same time, etc. This process can take months.

A lot of the system design is about talking closely to vendors about MTBF for specific loads, and we use a lot of metric data to characterize our loads accurately in those calculations. Then, of course, we measure once we're in the field to make sure we were on target, so the next round of estimations is better.

Re: Dropbox’s Exodus from the Amazon Cloud

#132

Post by gshx »

Sanddancer wrote: post #127
riobard wrote: post #37

> Measuring only one-and-half-feet by three-and-half-feet by six inches, each Diskotech box holds as much as a petabyte of data This number is very interesting. Basically Diskotech stores 1PB in 18" × 6" × 42" = 4,536 cubic inch volume, which is 10% bigger than standard 7U (17" × 12.2" × 19.8" = 4,107 cubic inch). 124 days ago Dropbox Storage Engineer jamwt posted here ( https://news.ycombinator.com/item?id=10541052…

Not to knock Dropbox's engineering, because it is sexy, but there are off the shelf enclosures right now that will fit 90 drives in a 4u case. Given that said enclosures have full rack rails and all, and are narrower, I don't doubt that a more densely designed server is feasible in the least with dropbox's design.

There are other concerns to keep in mind, eg. the upgradability of storage to use next gen cards that use 3d nand, etc. I'm sure they thought through a list of concerns before going this route.

Re: Dropbox’s Exodus from the Amazon Cloud

#133

Post by dexterdog »

yardie wrote: post #122
jamwt wrote: post #107

Earlier quoted context omitted.

Sometimes; but depreciation costs are usually more than either of those (for storage).

Did you build your own centers? Running our hosting we had far more constraints on our power than anything else. Even though we only had 20U of equipment we ended up taking a 44u rack. Our disk arrays being the largest consumers. This made the case for moving to SSDs even stronger.

That was the problem I always had in my pre-cloud days. We were usually running racks half-full because of power restrictions in the DC. It was convenient because you could keep you cold spares and tools in your expensive space, but it always seemed like you should be putting more useful stuff in the racks.

Re: Dropbox’s Exodus from the Amazon Cloud

#135

Post by [deleted] »

skuhn wrote: post #90
jamwt wrote: post #86

I want to point out one other thing about this project, if only to assuage my guilt. And this community of hackers and entrepreneurs seems as good a place as any to clear the air. There is a necessary abridgement that happens in media like this, wherein a few individuals in lead roles act as a vignette for the entire effort. In particular, on the software team, James and I are highlighted here, and it would be easy t…

Nice of you to call this out. I think everyone understands that not all of the people involved will get mentioned by name in a Wired article. A project like this clearly isn't a four person operation, and hopefully most people realize there are always a bunch of people working behind the scenes. My only disappointment is that by trying to set a PR friendly narrative, Dropbox greatly compressed the time spent on the p…

[deleted]

Re: Dropbox’s Exodus from the Amazon Cloud

#136

Post by Spooky23 »

planetjones wrote: post #4

Oh dear I didn't realise Dropbox had invested all of that time and money moving into their own data centre. From my perspective the future of Dropbox looks bleak. Mass storage with Amazon is much cheaper [edit: from a consumer perspective]. I know Dropbox has superior software that works (as opposed to the poor apps by Amazon and Google) but I imagine a lot of people are like me i.e. Store most of the stuff at the ch…

Not at all. I work at a decent size enterprise...

Buying off the shelf hardware providing an s3 style API saves us a significant amount of money, and while our scale is large, it isn't a fraction of what Dropbox is.

Re: Dropbox’s Exodus from the Amazon Cloud

#137

Post by jamwt »

Spooky23 wrote: post #134

I'm a recent Dropbox Pro subscriber -- I've been a user for many years. Really glad to see what's been going on from behind the scenes, and I'm looking forward to seeing what the future will bring to the front end of Dropbox. Thanks guys!

Awesome! Thanks for being a customer. We'll continue to work hard to make Dropbox better and better.

Re: Dropbox’s Exodus from the Amazon Cloud

#138

Post by pbarnes_1 »

planetjones wrote: post #4

Oh dear I didn't realise Dropbox had invested all of that time and money moving into their own data centre. From my perspective the future of Dropbox looks bleak. Mass storage with Amazon is much cheaper [edit: from a consumer perspective]. I know Dropbox has superior software that works (as opposed to the poor apps by Amazon and Google) but I imagine a lot of people are like me i.e. Store most of the stuff at the ch…

No normal consumer is writing directly to S3, so the entire premise of your debate is kind of moot, that's why you're being downvoted.

You're missing something here, which is that DropBox are probably now spending roughly 10-20% of what they were paying Amazon on an annual basis. The savings probably grow as DropBox grows. Those savings over your hypothetical 5 years could probably keep DropBox on life support for another 5 years.

Re: Dropbox’s Exodus from the Amazon Cloud

#139

Post by T2_t2 »

wpietri wrote: post #71
planetjones wrote: post #4

Oh dear I didn't realise Dropbox had invested all of that time and money moving into their own data centre. From my perspective the future of Dropbox looks bleak. Mass storage with Amazon is much cheaper [edit: from a consumer perspective]. I know Dropbox has superior software that works (as opposed to the poor apps by Amazon and Google) but I imagine a lot of people are like me i.e. Store most of the stuff at the ch…

I think this is where your mistake comes in: > I imagine a lot of people are like me i.e. Store most of the stuff at the cheapest location and use Dropbox just for docs that you want to sync on multiple devices I don't believe that's a correct intuition. You're describing optimizer behavior, but for most things people are saticficers [1]. I happily pay Dropbox $100 a year to make a problem go away. Is it the best dea…

It is hard enough to get staff to use DropBox at all, let alone optimise the spend.

Most of the world is not HN readers, it is McDonalds workers (gross generalisation for affect, not accuracy). Dropbox is the greatest tool ever when dealing with the vast majority of workers, because it just WORKS, without a lot of explanation. Work in this folder always or get written up. DropBox is still my favourite SAAS service, and every time a co-worker loses a file I just want to punch them.

Re: Dropbox’s Exodus from the Amazon Cloud

#140

Post by discodave »

gvb wrote: post #108
jamwt wrote: post #2

Hi HN! A couple of us from the Magic Pocket software team are around to answer questions if anyone has some.

Do you know what Amazon is going to do with their excess 500PB of capacity? Are they scaling up fast enough that it isn't a big deal? Is Glacier selling your vacated storage? You must have left a large hole in AWS' revenue stream. I assume their fees for transferring your data out of their system helped cover that short-term, but 500PB of data is a lot of storage capacity to sell. Your other answers indicate your sep…

One of the AWS people at Re:Invent mentioned that Amazon is storing 8EB across all storage services (S3, Glacier etc.)

So 500PB is ~6% of that... Although I'm sure Amazon would have liked to keep Dropbox as a customer it's probably a pretty small percentage of their revenue. Also, if you assume that AWS is growing at ~50% year over year then 6% of S3 is only about one or two months worth of growth.

I think it was this talk but I'm not sure: https://www.youtube.com/watch?v=3HDQsW_r1DM

Post reply on HN