Live data from Hacker News

Why Dropbox decided to build its own infrastructure and network

techcrunch.com

131–140 of 158 posts

Re: Why Dropbox decided to build its own infrastructure and network

#131
(a) CAF ran out

(b) Without CAF, GC and AWS pricing is ridiculous - somewhat akin to paying $9.99/lb for a Purdue chicken a day before the expiration

(c) Without the scale of AWS and GC one can get vendor prices at Google/Amazon MFN + 10% any day or +5% after schmoozing.

Re: Why Dropbox decided to build its own infrastructure and network

#132

Earlier quoted context omitted.

Yeah and that's why it amazes me that Netflix uses Amazon a straight competitor that uses the money it makes from Netflix to compete with Amazon prime.

Netflix probably in a better position to do the math than random people on HN. I think they have done their homework and decided that it is overall cheaper to keep ___some___ of their infra on AWS.

Yes! Thank you! Every time some piece like this is posted 'experts' arise here and know everything better. Like there is a bunch of idiots working at Netflix who are happily burning money...

Re: Why Dropbox decided to build its own infrastructure and network

#133

Dropbox hasn't dropped AWS, they moved things off AWS as it made sense to. The article is talking about two things, the move of file storage and a network backbone. Neither of which were done recently. The file storage move from S3 to Magic Pocket is detailed in these blog posts: https://blogs.dropbox.com/tech/2016/03/magic-pocket-infrastr... https://blogs.dropbox.com/tech/2016/05/inside-the-magic-pock... https://blo…

The 'value' cloud services provide is purely for experimental services an organization does not want to commit physical assets. Those deluded to believing cloud services provide any value beyond test service deployments are propaganda poster boys for today's tech sucker awards.

Have you not worked in smaller companies? Economies of scale apply in myriad situations.

Re: Why Dropbox decided to build its own infrastructure and network

#134

I'll be 100% honest, I didn't realize Dropbox was still on AWS. You figure at a certain scale it makes more sense to run your own solution.

Yeah and that's why it amazes me that Netflix uses Amazon a straight competitor that uses the money it makes from Netflix to compete with Amazon prime.

CAF

Re: Why Dropbox decided to build its own infrastructure and network

#136

Dropbox is cool and all, but i hate their pricing model. I just want to keep about 50-100 GB there (i don't hoard stuff). I don't wanna pay $10 / month for that. At backblaze i pay less than $0.5 / month. I could pay quadruple, since Dropbox is a lot better service (and quite a different one, too). But not 20 times as much. I know that Dropbox doesn't care about my tiny dollars and all. But why not let customers pay…

what bugged me about their pricing model (perhaps it's changed?) is that things people share with me count to my cap. I had a client share stuff with me via dropbox. So I signed up. I get "2 gig free!" Yay. Another client shares stuff with me. Then another. Then another. Then I'm at my 2gig cap with just other peoples' stuff. They all think it's great, because none of them are paying for it. I needed to pay money to support their free use of the service, and I declined. Just one more $x/month service I didn't need to get hooked to. Maybe the pricing model has changed what they count now?

Re: Why Dropbox decided to build its own infrastructure and network

#137

Earlier quoted context omitted.

They run BGP and do an enormous amount of IX peering. https://www.peeringdb.com/asn/2906 AWS doesn't even let you do BGP. If you want to use them for a CDN, you're monopolized into their network and their blend of ridiculously overpriced bandwidth. They might be using AWS for the canonical store of data, but the sanity of using "the cloud" goes out the window the second you need to ship a lot of traffic to the public…

> heir blend of ridiculously overpriced bandwidth. For someone not at netflix's level, can you recommend somewhere with significantly cheaper bandwidth.

Below $10k/mo bandwidth plays are irrelevant - the cost of the support infrastructure is just too high.

At $10k/mo what you do is plop a rack in one of the well connected buildings in NYC, VA, SF, SJC, CHI and get either flat rate 10Gs or 1G commits over 10Gs. You should be paying between 0.55 and 0.75 per Mbit/sec on 95th percentile on 1G commit. You put into the same cabinet your edge nodes that actually would be pushing the traffic out and use AWS or GCP for your compute workloads.

Re: Why Dropbox decided to build its own infrastructure and network

#138

Earlier quoted context omitted.

> heir blend of ridiculously overpriced bandwidth. For someone not at netflix's level, can you recommend somewhere with significantly cheaper bandwidth.

I built a global video CDN using SoftLayer a couple years ago and ended up paying around $0.01/GB.

How? https://www.ibm.com/cloud-computing/bluemix/content-delivery... (which is linked to from http://www.softlayer.com/) says $0.12/GB.

https://aws.amazon.com/cloudfront/pricing/ says $0.085/GB.

Re: Why Dropbox decided to build its own infrastructure and network

#139
post #3

Earlier quoted context omitted.

and Netflix?

They run BGP and do an enormous amount of IX peering. https://www.peeringdb.com/asn/2906 AWS doesn't even let you do BGP. If you want to use them for a CDN, you're monopolized into their network and their blend of ridiculously overpriced bandwidth. They might be using AWS for the canonical store of data, but the sanity of using "the cloud" goes out the window the second you need to ship a lot of traffic to the public…

> you're monopolized into their network and their blend of ridiculously overpriced bandwidth.

Wouldn't a customer the size of Netflix be granted more liberty in this regard?

Re: Why Dropbox decided to build its own infrastructure and network

#140
I am curious on the hardware build out used for the storage nodes. Some of the major issues I have seen with all the appliances out there for storage are the following:

1. Network throughput on the appliance is fast but for an Enterprise level the 10gigE cards used become a bottleneck for transactions because of how the software hypervisor scales the data. 2. Power consumption of the appliances in a rack mount environment are too high and leave needed space that has to stay empty because of the facility power per rack limitations. 3. The software hypervisor scales the stack vertically and relies on the software to load balance horizontally. The performance in a high transactional environment becomes dependent on the software to scale instead of the natural horizontal distribution that can be setup on the hardware out of the box. Standard multi-purpose storage arrays scale horizontal with very little over head from traditional software storage management. I only found one company whose software does not force the stack to be vertical but they fail to meet a reasonable performance in network/power.

Streaming petabytes of data to keep a dynamic constant (static overall storage requirement that changes it's data life cycle via retention rules) becomes very hard with premade hardware.

Does anyone have any recommendations or has attempted a similar exodus from S3 that they can share?

Post reply on HN