Well unfortunately I'm seeing this in my microscopic way. I moved off all of my domains and hosting from GoDaddy (shrugging in disgust) to Google domain+GCE. yikes, with virtually no traffic outside of me ssh'ing, my 2 vm instances is costing me $45/mon just basically sitting idle. I gotta be doing something wrong!
Dropbox saved $75M over two years by building its own infrastructure (2018)
181–190 of 218 posts
Re: Dropbox saved $75M over two years by building its own infrastructure (2018)
#182Earlier quoted context omitted.
> So they saved next to nothing? Their yearly operating expenses are in the order of $1.3B, which doesn't even include cost of revenue. Wow thank god some of y'all aren't running these businesses. It still surprises me every time I see failures to think marginally.
That's assuming the numbers shown fully take into account all the tangential cost related to the transition, and not just "cost before vs cost after". If you spent 500M in developer hours doing the migration, that just wiped like 10 years worth of savings. Also it's not clear if in 2 years you really get to see the cost of maintenance and breakages. One big breakage could wipe off that 75M very quickly. Of course Clo…
> that just wiped like 10 years worth of savings.
And then the next 10 years you get pure profit? That's still an ROI and significant one.
> One big breakage could wipe off that 75M very quickly.
What specifically would cause a $75M loss? As you said, cloud is not bullet proof.
Re: Dropbox saved $75M over two years by building its own infrastructure (2018)
#183Earlier quoted context omitted.
I'm not privy to any particulars specific to GitLab's situation, but it's possible that a little sabre-rattling caused a better deal to appear which changed the calculus on migration.
My limited visibility take is that their board (or someone higher up, my memory is fuzzy) told them to use the cloud, and that was that.
I could see it. As discussed elsewhere in this thread, investing in your own metal/ops is definitely a long term play, and some combination of "focus on your actual differentiators" and "why bother when one of those cloud providers will acquire us eventually and we'll just have to migrate to them at that point anyway" could be a compelling case.
Re: Dropbox saved $75M over two years by building its own infrastructure (2018)
#184Earlier quoted context omitted.
You’ve described the historical value of cloud computing perfectly. That said, I think the days where all but the largest or most stubborn companies run their own datacenters are coming to a close. The problem will be finding skilled labor. Short-haul networking, power configurations, thermal load, hardware maintenance; these and many more are specific skills that can’t be learned overnight. Data center work used to…
I'm not in the web or cloud business, but I've filled a rack with my stuff before. My impression is that hardware has become a lot more capable even relative to its tasks. With high iops memory, many cores and obscene amounts of RAM, I would expect companies of a much larger scale (in $, FTEs, or most other metrics) can be served by one 4HE machine, or by one rack, or by one room. Thus I would expect the knowledge of…
> My impression is that hardware has become a lot more capable even relative to its tasks.
Indeed. The margins are bonkers high. As an example, the amount of ram that you can stuff into a physical machine has at least doubled in the last five years, but the price of the average virtual machine has not.
Re: Dropbox saved $75M over two years by building its own infrastructure (2018)
#185Earlier quoted context omitted.
You still want ha, failover, and disaster recovery. Then you need to set up stuff like bgp, dns, security rules, etc, etc etc. Complexity mounts pretty quickly.
Indeed. It seems that most of the people saying that cloud hosting is expensive have never run into the issues of making their own SAN, managing the provisioning of 20 different teams, etc. The organizational complexity and specialist knowledge is mind-boggling and there is zero chance that your in-house knowledge is better than what Amazon can provide.
This is true, but unrelated.
We're talking about Dropbox scale.
At that scale you can (nee should) hire all the specialists you need.
Re: Dropbox saved $75M over two years by building its own infrastructure (2018)
#186Earlier quoted context omitted.
You’ve described the historical value of cloud computing perfectly. That said, I think the days where all but the largest or most stubborn companies run their own datacenters are coming to a close. The problem will be finding skilled labor. Short-haul networking, power configurations, thermal load, hardware maintenance; these and many more are specific skills that can’t be learned overnight. Data center work used to…
I'm not in the web or cloud business, but I've filled a rack with my stuff before. My impression is that hardware has become a lot more capable even relative to its tasks. With high iops memory, many cores and obscene amounts of RAM, I would expect companies of a much larger scale (in $, FTEs, or most other metrics) can be served by one 4HE machine, or by one rack, or by one room. Thus I would expect the knowledge of…
Installing rack servers and setting up services to run a site used to be a sort of rite-of-passage 15-20 years ago, but that time period of the web was different. Still, I would consider basic familiarity with the infrastructure necessary also today.
Increasing hardware performance relative to task load created the rational for virtualization. Virtualization also turned out to be rational with respect to consistency, convenience, maintenance, and so on. At that point, outsourcing to a cloud can be rational.
But fewer people get hands-on experience with the infrastructure, and it sounds like many consider it almost mythical. For example, realizing the amount of work that can be done in 4U today. What does amazon charge for 96 cores and 256GB?
Re: Dropbox saved $75M over two years by building its own infrastructure (2018)
#187Earlier quoted context omitted.
>Definitely nothing for any medium sized company. I work at a medium sized company. Depends on who you count, but let's say around ~30 devs. Recently we basically did just this, and it's been a great success. We haven't fully migrated and still use AWS for prod, but have seen substantial savings already. We spent $2k on servers, Dell r720s. We bought a UPS and mount, and racked them in our office. I installed OpenShi…
This post proves parent's point though. You're not doing anything even remotely close to the features offered by cloud providers or even managed hosting providers. Disaster recovery? Geographically separate redundant servers with failovers? Automated (and proven to work) backups? One-stop access control for infra maintenance? Audit controls for your database and storage objects? Tape backups? Even today to support al…
> You're not doing anything even remotely close to the features offered by cloud providers or even managed hosting providers.
So what?
Who in their right mind believes in, say, you need to operate and maintain half a dozen types of RDBMS in three flavors along with two or four or eight different message brokers and your own convoluted infrastructure-as-code multiplied by three along with a repackaged FLOSS offering... And a ground station?
Let's not be mad, here. There are proper, full-blown, popular, global-scale cloud service providers. That. Only. Offer. VMs.
Are we so drunk with corporate kool-aid to believe that we are missing out because we are missing... What do you believe you're missing, actually?
I repeat: there are popular professional cloud service providers whose business consists of providing either VMs or access to bare metal. That's where real-world companies run their real-world businesses. Why are we supposed to believe that you need more to operate your own stuff?
Re: Dropbox saved $75M over two years by building its own infrastructure (2018)
#188Earlier quoted context omitted.
There are now businesses that explicitly depend on the elasticity of the cloud and can never really be moved on premise without massive up-front investment in hardware that may only be used a few times a year for their biggest customers. Trying to hybridize these workloads hasn't been very successful as of yet. It is possible that K8S could relive this problem but I haven't seen it in practice, at scale.
Instant Elasticity in Cloud is a myth. If you think you are going to get 1k hosts just like that from AWS you will have an unpleasant experience. I work at one of the decent size tech company and we are split between cloud and on prem. From our experience you have to inform AWS/GCP in advance (sometime way early) if you are looking to meaningfully increase capacity in zone/region. Sure, auto scaling few hundreds of h…
As these are generally scheduled events, end of year, end of quarter, etc. they can be planned. Beats owning the machines.
Re: Dropbox saved $75M over two years by building its own infrastructure (2018)
#189Earlier quoted context omitted.
There are now businesses that explicitly depend on the elasticity of the cloud and can never really be moved on premise without massive up-front investment in hardware that may only be used a few times a year for their biggest customers. Trying to hybridize these workloads hasn't been very successful as of yet. It is possible that K8S could relive this problem but I haven't seen it in practice, at scale.
Elasticity? Fine. So their, say, single rack will sometime have limited load and be under-utilized. About the up-front investment - most hi-tech companies are a massive initial up-front (or nearly-up-front) investment.
I was talking about at scale, not a rack. If you can get by with a rack, you will pay more for the people to support it than the incremental cost of the cloud.
Re: Dropbox saved $75M over two years by building its own infrastructure (2018)
#190Well unfortunately I'm seeing this in my microscopic way. I moved off all of my domains and hosting from GoDaddy (shrugging in disgust) to Google domain+GCE. yikes, with virtually no traffic outside of me ssh'ing, my 2 vm instances is costing me $45/mon just basically sitting idle. I gotta be doing something wrong!
You can spin up a VPS on digital ocean for $5/month