Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dropbox’s Exodus from the Amazon Cloud

wired.com

Post reply on HN
Print view View original

81–90 of 252 posts

Re: Dropbox’s Exodus from the Amazon Cloud

#81

Post by rbranson »

james_cowling wrote: post #44
jamwt wrote: post #29

Earlier quoted context omitted.

> Dropbox rewrote Magic Pocket in Golang, and then rewrote it again in Rust, to fit on their custom built machines. Actually, full disclosure, we really just rewrote a couple of components in Rust. Most of Magic Pocket (the distributed storage system) is still written in golang. > No word on perf improvements, cost savings, stability, total number of servers, amount of data stored, or how the data was moved. Performa…

Yup, Dropbox Infra is mostly a Go shop. It's our primary development language and we don't plan to switch off any time soon. We're always about the right tool for the job tho, and there are definitely use cases where Rust makes a lot of sense. We've been really happy with it so far.

Curious about the specifics with how you interfaced Go with Rust.

Re: Dropbox’s Exodus from the Amazon Cloud

#82

Post by james_cowling »

atYevP wrote: post #78
james_cowling wrote: post #61

Earlier quoted context omitted.

We were pushing over a terabit of data transfer at peak, so 4-5PB per day.

Yev from Backblaze here -> we'd be curious to hear how! A lot of folks are looking for quick migration assistants for our B2 service, that's pretty fast!

We have a big fleet of what we call "backfill nodes" which handle the transfer but the most difficult component of the transfer is just the network capacity. We worked pretty closely with Amazon here since we have a long-standing relationship and a lot of data transfer between the two companies.

Re: Dropbox’s Exodus from the Amazon Cloud

#83

Post by mathattack »

ignoramous wrote: post #24

Key points: 1. Dropbox moved from AWS to its own datacenters after 8 months of rigourous testing. They didn't exactly build a S3 clone, but something tailored to their needs, they named it Magic Pocket. 2. Dropbox still uses AWS for its European customers. 3. Dropbox hired a bunch of engineers from Facebook to build its own hardware heavily customised for data-storage and IOPS (naturally) viz. Diskotech. Some 8 Disko…

It would seem to me that if you're primarily in the business of storage, you should go down the stack as far as you can. Outsourcing to AWS leaves you at parity to anyone else who can do the same.

If Box is in the "Business Services" business, then perhaps it's different.

Re: Dropbox’s Exodus from the Amazon Cloud

#84

Post by jamwt »

djcapelis wrote: post #77
jamwt wrote: post #2

Hi HN! A couple of us from the Magic Pocket software team are around to answer questions if anyone has some.

Who's part of your security team over there? Most of the big cloud companies have people I trust working for them and I've been pretty comfortable telling most of my clients that cloud services are often going to have a better security team than they will. :) But now that you've moved away from AWS, have you expanded your security team to help make up for the fact that you need folks to cover all your infrastructure…

We have several dozen very, very good engineers working on a few discrete teams--infrastructure security, product security, etc. I don't feel comfortable getting any more specific than that; we keep security information pretty close to the vest.

Re: Dropbox’s Exodus from the Amazon Cloud

#85

Post by jimjimjim »

planetjones wrote: post #6
jamwt wrote: post #5

Earlier quoted context omitted.

> Oh dear I didn't realise Dropbox had invested all of that time and money moving into their own data centre. We've had our own datacenters for a very long time; this project just represents moving "blob storage" into our own datacenter rather than just metadata. > Mass storage with Amazon is much cheaper As the article says, in Dropbox's particular case this is very untrue. Although I agree with your general point t…

Sorry I think you misunderstand me. Technically it sounds like a great achievement. Well done! Ending badly I mean that in 5 years time Dropbox will be left with a load of servers and a business model in tatters. It's the economics I think will end badly Also I am sure the in house data centre looks cheaper to Dropbox - but now they have their own hardware on the balance sheet. But I meant cheaper from a consumer poi…

The vast majority of businesses around the world have "assets". It is a universal practice to depreciate those assets over their expected lifetime. The difference between purchase price and resale value is then treated as a normal business expense over the course of those years. So when the servers lifetime is up there are no "load of servers" to worry about. You throw them out (or re-task them) and get new ones.

These costs are known and planned for. Dropbox wouldn't do this without running the numbers through vast herds of accountants.

Now here is the kicker. Amazon does this too (but slightly less due to economies of scale) and then makes sure their billing covers this.

Re: Dropbox’s Exodus from the Amazon Cloud

#86

Post by jamwt »

I want to point out one other thing about this project, if only to assuage my guilt. And this community of hackers and entrepreneurs seems as good a place as any to clear the air.

There is a necessary abridgement that happens in media like this, wherein a few individuals in lead roles act as a vignette for the entire effort. In particular, on the software team, James and I are highlighted here, and it would be easy to assume we did this whole thing ourselves. Especially given very generous phrasing at times like "Turner rebuilt Magic Pocket in an entirely different programming language".

The full picture is James and I were very fortunate to work with a team of more than a dozen amazing engineers on this project, that lead designs and implementations of key parts of it, and that stayed just as late at the office as we did during crunch time. In particular, the contributions of the SREs didn't make it into the article. And managing more than half a million disks without an incredible SRE team is basically impossible.

Re: Dropbox’s Exodus from the Amazon Cloud

#87

Post by benologist »

jl6 wrote: post #53
jamwt wrote: post #39

Earlier quoted context omitted.

The difference to get even higher is our system supports host-managed SMR disks that come in 10T and 14T sizes.

I've never heard of a 14TB disk drive. Is that what you mean? Who makes these?

http://www.theregister.co.uk/2016/03/03/samsung_2_5_inch_15t...

2.5" 15TB ... and SSD, and expected to get a lot bigger too!

Re: Dropbox’s Exodus from the Amazon Cloud

#88

Post by djcapelis »

jamwt wrote: post #84
djcapelis wrote: post #77

Earlier quoted context omitted.

Who's part of your security team over there? Most of the big cloud companies have people I trust working for them and I've been pretty comfortable telling most of my clients that cloud services are often going to have a better security team than they will. :) But now that you've moved away from AWS, have you expanded your security team to help make up for the fact that you need folks to cover all your infrastructure…

We have several dozen very, very good engineers working on a few discrete teams--infrastructure security, product security, etc. I don't feel comfortable getting any more specific than that; we keep security information pretty close to the vest.

To be honest, that response is less than comforting. But I understand how that'd be your answer given your position in the organization. Hopefully someone else at Dropbox feels like sharing more at some point and being more forthright about your company's practices in protecting data others entrust to them.

Feel free to refer others to this thread if you think it'd be helpful.

Re: Dropbox’s Exodus from the Amazon Cloud

#89

Post by scholia »

toomuchtodo wrote: post #33
apalmer wrote: post #23

Earlier quoted context omitted.

I don't understand your concern. Fundamentally, if you have large enough consistent need for storage/VMs/Any non proprietary resource then AWS is not the lowest cost option by a long shot. This is not at all a controversial statement. Even taking into account depreciation, administration, maintenance etc, if you have a large enough consistent need amazon is not the lowest cost option by a significant margin. This isn…

Rarely do people "get" that if you have a consistent workload or resource need, AWS is more expensive than your own gear.

But presumably Amazon benefits (saves money) from making more use of hardware resources. It has to have some spare, but not as much spare as a comparable group of non-AWS users would have in total....

Also, presumably, not all the AWS users are awake at the same time ;-)

Re: Dropbox’s Exodus from the Amazon Cloud

#90

Post by skuhn »

jamwt wrote: post #86

I want to point out one other thing about this project, if only to assuage my guilt. And this community of hackers and entrepreneurs seems as good a place as any to clear the air. There is a necessary abridgement that happens in media like this, wherein a few individuals in lead roles act as a vignette for the entire effort. In particular, on the software team, James and I are highlighted here, and it would be easy t…

Nice of you to call this out. I think everyone understands that not all of the people involved will get mentioned by name in a Wired article. A project like this clearly isn't a four person operation, and hopefully most people realize there are always a bunch of people working behind the scenes.

My only disappointment is that by trying to set a PR friendly narrative, Dropbox greatly compressed the time spent on the project from inception to completion, and that erases a lot of people's involvement. As someone who worked on MP when it actually was a four person project, it's a little frustrating to finally be able to talk about it after all this time while at the same time the public perception is that the project began after I left the company.

Post reply on HN