Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dropbox’s Exodus from the Amazon Cloud

wired.com

Post reply on HN
Print view View original

141–150 of 252 posts

Re: Dropbox’s Exodus from the Amazon Cloud

#141

Post by discodave »

james_cowling wrote: post #46
ignoramous wrote: post #24

Key points: 1. Dropbox moved from AWS to its own datacenters after 8 months of rigourous testing. They didn't exactly build a S3 clone, but something tailored to their needs, they named it Magic Pocket. 2. Dropbox still uses AWS for its European customers. 3. Dropbox hired a bunch of engineers from Facebook to build its own hardware heavily customised for data-storage and IOPS (naturally) viz. Diskotech. Some 8 Disko…

This is most accurate. A couple of comments: > Dropbox still uses AWS for its European customers. We haven't publicly launched EU storage yet but will be doing so later in the year. > Dropbox hired a bunch of engineers from Facebook to build its own hardware heavily customised for data-storage and IOPS (naturally) Facebook and Google and startup folks and people from random other places. Our IOPS demands are reasonab…

How much of your cost savings was due to the lower IOPS requirement?

Also, was the S3 "infrequent access" tier a response to customers like you or was your special bulk pricing already taking into account your low IOPS demands?

Thanks!

Re: Dropbox’s Exodus from the Amazon Cloud

#142

Post by dividuum »

bjz_ wrote: post #104
jamwt wrote: post #2

Hi HN! A couple of us from the Magic Pocket software team are around to answer questions if anyone has some.

Do you use much in the way of formal specifications? I've heard that Amazon currently uses TLA+ to verify the semantics of their systems.

Interesting. Here's the paper amazon published: http://research.microsoft.com/en-us/um/people/lamport/tla/fo...

It's only covering a few use cases and doesn't go into too much detail and shows example code but it seems like using TLA+ has been beneficial for them.

Re: Dropbox’s Exodus from the Amazon Cloud

#143

Post by pnathan »

james_cowling wrote: post #112
abhv wrote: post #102

Earlier quoted context omitted.

Does magic pocket use any non-standard error-correction algorithms, or just parity-style RAID5 or 6?

We use a variant on Reed-Solomon coding that's optimized for lower reconstruction cost. This is similar to Local Reconstruction Codes but a design/implementation of our own. The data placement isn't RAID. We encode aggregated extents of data in "volumes" that are placed on a random set of storage nodes (with sufficient physical diversity and various other constraints). Each storage node might hold a few thousand volu…

That is extremely cool. Please tell the author(s) of that system that a stranger on the internet has appreciation for that feat!

Re: Dropbox’s Exodus from the Amazon Cloud

#144

Post by luhn »

jamwt wrote: post #2

Hi HN! A couple of us from the Magic Pocket software team are around to answer questions if anyone has some.

> If a bunch of people shared some files via Dropbox, the company stored the files on Amazon’s Simple Storage Service, or S3, while housing all the metadata related to those files—who they belonged to, who was allowed to download them, and more—on its own machines inside its own data center space.

I'm surprised at this. I would have expected the opposite: using AWS for the application and metadata but managing your own storage infrastructure. Much like how Netflix runs on AWS but streams video through its self-managed CDN. Can you shed some light on why it was designed this way?

Re: Dropbox’s Exodus from the Amazon Cloud

#145

Post by samstave »

jclaybaugh wrote: post #125
dikaiosune wrote: post #118

Earlier quoted context omitted.

I assume Site Reliability Engineer from the last time I looked at their openings. I didn't look at the description, but I assume it's a more formal description for devops?

devops is not a job title. It is a way of doing things.

"devops engineer" is most certainly a job title. Devops is analogous with "agile" when talking about process...

Re: Dropbox’s Exodus from the Amazon Cloud

#146

Post by samstave »

jamwt wrote: post #52
james_cowling wrote: post #49

Earlier quoted context omitted.

To be fair to Backblaze this level of storage density is really only possible with recent advances in disk technology (higher densities, SMR storage, etc). Also not everyone wants to be packing a petabyte into a box. At that level of density you need to invest a lot of effort in replication strategies, tooling, network fabric etc to handle failures with high levels of availability/durability.

Yes, a 1PB+ failure domain only made sense because Magic Pocket is very good at automatic cluster management and repair.

Are you using spindle or ssd or flash? (admittedly I dont know if you consider flash and ssd to be the same)

What is your price per GB raw?

Re: Dropbox’s Exodus from the Amazon Cloud

#147

Post by windlep »

jamwt wrote: post #115
jabl wrote: post #109

Earlier quoted context omitted.

How do you do network io with rust? Thread-per-connection, non-blocking (using mio or?), or something else?

We have an in-house futures-based framework (inspired by Finagle) built on mio (non-blocking libevent like thing for rust). All I/O is async, but application work is often done on thread pools. Those threads are freed as soon as possible, though, so that I/O streams can be handled purely by "the reactor", and we keep the pools as small as possible.

Any plans to open-source the futures-based Rust framework? :)

Re: Dropbox’s Exodus from the Amazon Cloud

#148

Post by ChuckMcM »

james_cowling wrote: post #49
riobard wrote: post #37

> Measuring only one-and-half-feet by three-and-half-feet by six inches, each Diskotech box holds as much as a petabyte of data This number is very interesting. Basically Diskotech stores 1PB in 18" × 6" × 42" = 4,536 cubic inch volume, which is 10% bigger than standard 7U (17" × 12.2" × 19.8" = 4,107 cubic inch). 124 days ago Dropbox Storage Engineer jamwt posted here ( https://news.ycombinator.com/item?id=10541052…

To be fair to Backblaze this level of storage density is really only possible with recent advances in disk technology (higher densities, SMR storage, etc). Also not everyone wants to be packing a petabyte into a box. At that level of density you need to invest a lot of effort in replication strategies, tooling, network fabric etc to handle failures with high levels of availability/durability.

Not sure what is more amazing, the project of this scale (love the disk drawers!) or that infrastructure for managing the fleet of drives gets top billing!

Re: Dropbox’s Exodus from the Amazon Cloud

#150

Post by dikaiosune »

windlep wrote: post #147
jamwt wrote: post #115

Earlier quoted context omitted.

We have an in-house futures-based framework (inspired by Finagle) built on mio (non-blocking libevent like thing for rust). All I/O is async, but application work is often done on thread pools. Those threads are freed as soon as possible, though, so that I/O streams can be handled purely by "the reactor", and we keep the pools as small as possible.

Any plans to open-source the futures-based Rust framework? :)

From a parallel conversation[0] on the Rust subreddit:

>Are you going to open source anything?

>Probably. We have an in-house futures-based I/O framework. We're going to collaborate with the rust team and Carl Lerche to see if there's something there we can clean up and contribute.

[0]: https://www.reddit.com/r/rust/comments/4adabk/the_epic_story...

Post reply on HN