Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is your company sticking to on-premise servers? Why?

news.ycombinator.com

Post reply on HN
Print view View original

521–530 of 782 posts

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#521

Post by kortilla »

dragonwriter wrote: post #440
andrenotgiant wrote: post #333

Earlier quoted context omitted.

It sounds like you've got a good setup going with colo, but just as a way of illustrating some of the small providers lower bandwidth costs: DigitalOcean gives 3TB bandwidth on a $15 2CPU/2GBMEM/60GBSSD instance. If you ran 1350 of them it'd cost you ~$20k/month and get you your 4PB egress within bandwidth allowance.

> DigitalOcean gives 3TB bandwidth on a $15 2CPU/2GBMEM/60GBSSD instance. DigitalOcean is a VPS provider, not a cloud provider, and that's pretty similar to pricing on the nearest comparable AWS service (Amazon LightSail), not something showing DO as notably better (1Core/2GBRAM/60GBSSD/3TB transfer @ $10/mo or 2Core/4GBRAM/80GBSSD/4TB transfer @ $20/mo.) Though if you are optimizing price per TB of transfer quota, L…

> DigitalOcean is a VPS provider, not a cloud provider

From the first paragraph in https://en.wikipedia.org/wiki/DigitalOcean

> DigitalOcean, Inc. is an American cloud infrastructure provider[2] headquartered in New York City with data centers worldwide.[3] DigitalOcean provides developers cloud services that help to deploy and scale applications that run simultaneously on multiple computers.

You might have your own cute definition of “cloud”, but it doesn’t match the industry so get over it and stop correcting people.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#522

Post by davedx »

sandGorgon wrote: post #428
AgentK20 wrote: post #207

Like many others have pointed out: Cost. I'm the CTO of a moderately sized gaming community, Hypixel Minecraft, who operates about 700 rented dedicated machines to service 70k-100k concurrent players. We push about 4PB/mo in egress bandwidth, something along the lines of 32gbps 95th-percentile. The big cloud providers have repeatedly quoted us an order of magnitude more than our entire fleet's cost....JUST in bandwid…

what i have generally seen is that storage - blob (s3) or database - is the blocker. Even if you go to k8s, the hassle to manage storage is non-trivial. And because of this singular fact, startups cannot move to on-premise. You really dont want to manage snapshots, restores, backups. Anyone can manage application servers.

I don't understand. You're talking about a filesystem, with an option for backing that filesystem up?

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#523

Post by twic »

nemothekid wrote: post #305
mmmBacon wrote: post #292

Earlier quoted context omitted.

Yes, you’re paying for a ton of stuff there that you probably don’t use and then are susceptible to bugs that have nothing to do with your use case. At $5B it would not cost anywhere near that much to replicate. Your infra would be better tailored to your workloads and your sw teams which would further drive costs down. The upsides are that you only drive features you need and keep things simple. The downside is that…

Snap was spending 2B over 5 years - so 400M/yr. By comparison, Uber, who operates under managed colo, spends nearly 200M/yr alone on real estate for their datacenters. Who knows how much they are paying in engineering salaries so manage those datacenters. Personally I don't think there's a one size fits all solution, you will have to do the math (like I'm sure Snap, Netflix and others have done) to see if cloud is wo…

> Uber, who operates under managed colo, spends nearly 200M/yr alone on real estate for their datacenters

200 million a year, huh.

The cost of commercial office space in the U.S. can range from $6 per square foot in low cost regions to over $12 per square foot in New York City. On average, a 50-cabinet data center will occupy about 1,700 square feet. At a median cost of $8 per square foot, the space alone would cost about $13,600 per month. [1]

Are Uber renting on the order of two million square feet of data centre? Do they have sixty thousand cabinets of hardware?

If they do, i would absolutely love to see a quote for how much it would cost them to run in the cloud.

I think it's far more likely that number is bullshit.

[1] https://npifinancial.com/blog/data-center-pricing/

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#524

Post by twic »

avifreedman wrote: post #449
nemothekid wrote: post #305

Earlier quoted context omitted.

Snap was spending 2B over 5 years - so 400M/yr. By comparison, Uber, who operates under managed colo, spends nearly 200M/yr alone on real estate for their datacenters. Who knows how much they are paying in engineering salaries so manage those datacenters. Personally I don't think there's a one size fits all solution, you will have to do the math (like I'm sure Snap, Netflix and others have done) to see if cloud is wo…

Unless that's capex for building, or includes server capex that they depreciate, that seems way high given Uber's scale. A pretty dense cabinet should only cost ~$1400/mo at wholesale (1MW+ rooms) rates, and $200M is 143,000 cabinets. And it's public that Uber uses multiple clouds as well. Disclaimer: I haven't reviewed public Uber filings, would be very interested if there's any data that indicates they're really sp…

> A pretty dense cabinet should only cost ~$1400/mo at wholesale (1MW+ rooms) rates, and $200M is 143,000 cabinets.

The 200M is for a year - at $1400/mo, that's 11905 cabinets.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#525

Post by toyg »

kortilla wrote: post #514
toyg wrote: post #433

Earlier quoted context omitted.

The last point is something that “we”, as an industry, can often be blind to. If your business is something that is fundamentally not “pushing bytes”, you really, really don’t want to know about routers, firewalls, the OSI layer, SHA256, RAID arrays, all the way to this week’s JS framework. All of that is a big annoyance, and paying AWS to “take care of it” makes sense, even if it comes at a higher price: more often…

But if you’re incompetent to that degree, AWS isn’t for you because you can’t even write software to run on it. You should be using a fully managed SaaS at that point.

In many cases yes, but there are shades of grey. Some systems cannot be shared, for whatever reason.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#527

Post by wwn_se »

koheripbal wrote: post #265
AgentK20 wrote: post #207

Like many others have pointed out: Cost. I'm the CTO of a moderately sized gaming community, Hypixel Minecraft, who operates about 700 rented dedicated machines to service 70k-100k concurrent players. We push about 4PB/mo in egress bandwidth, something along the lines of 32gbps 95th-percentile. The big cloud providers have repeatedly quoted us an order of magnitude more than our entire fleet's cost....JUST in bandwid…

Same here - CTO of medium sized company. Our IT infra costs are 1/10th the cost of cloud, simply because I happen to be comfortable having on-premise machine and working on them (sometime myself). We have two dozen servers in two locations. It's more time to setup, but maintenance is actually quite low.

On factor for small companies that I have noticed at the places where I have been is that (as a dev consultant). The places that use the cloud often scale up to much. Since the cloud don't limit developers its very easy to just "spinn up a new xx" and not think about the long term costs.

Unless you are really small, have variable workloads then the cloud is maybe not for you. Unless the cost is a small part of the total cost of the plattform like not really related to how many users/sales you have.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#528

Post by twic »

kortilla wrote: post #514
toyg wrote: post #433

Earlier quoted context omitted.

The last point is something that “we”, as an industry, can often be blind to. If your business is something that is fundamentally not “pushing bytes”, you really, really don’t want to know about routers, firewalls, the OSI layer, SHA256, RAID arrays, all the way to this week’s JS framework. All of that is a big annoyance, and paying AWS to “take care of it” makes sense, even if it comes at a higher price: more often…

But if you’re incompetent to that degree, AWS isn’t for you because you can’t even write software to run on it. You should be using a fully managed SaaS at that point.

There are three things that make sense to me.

Firstly, if you are big enough, you can manage/architect your data centre better than a cloud provider. You can afford to hire staff to do it, and they can build something specific to your needs. Companies like this should be on-prem.

(It doesn't matter if the company is "digital" or not. It could be a huge retail chain, or a government agency, whatever. The only requirement is to be big enough that you have big needs and can afford a big spend on staff.)

Secondly, if you are small enough, you really, really don’t want to know about routers, firewalls, the OSI layer, SHA256, RAID arrays, etc. Dealing with that would mean another couple of full-time employees, and you can't afford that. Companies like this should be on a SaaS (ideally one less full of gotchas than Heroku).

Thirdly, if you are in between, you have the capacity to deal with system administration, there are some advantages to being able to shape your infrastructure to your needs, but you really don't want to get into real estate and millions of dollars of CAPEX. Companies like this should be on rented physical hardware.

What doesn't make sense, at any scale, is renting VMs.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#529

Post by xiconfjs »

XCSme wrote: post #34

I use a $5/mo DigitalOcean VPS droplet instead of AWS or other "cloud" service. I only have to host an analytics dashboard ( https://usertrack.net/ ), I don't need scaling and this way I know exactly how much I will pay. The resources are more than enough for my needs and I don't think it could be much cheaper even on the most optimized pay-per-minute of use cloud platforms. I also have some other APIs hosted in the…

off topic: do you really track the users mouse movements and personal data (IP & such) on your side without asking for permission? Kinda strange these days, isn't it?

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#530

Post by wongarsu »

user5994461 wrote: post #513
avifreedman wrote: post #449

Earlier quoted context omitted.

Unless that's capex for building, or includes server capex that they depreciate, that seems way high given Uber's scale. A pretty dense cabinet should only cost ~$1400/mo at wholesale (1MW+ rooms) rates, and $200M is 143,000 cabinets. And it's public that Uber uses multiple clouds as well. Disclaimer: I haven't reviewed public Uber filings, would be very interested if there's any data that indicates they're really sp…

The 42U rack might be only $1400 per month but the servers to put inside are $5000 upfront per U.

42U times $5000 divided by 3 years is still only $5833 per month per rack. Add the $1400 and we have a total amortized rack cost of $7233 per month. (naively assuming you could fill every spot in the rack with servers).

But servers can't be included in a "Uber spends nearly 200M/yr alone on real estate for their datacenters" figure anyway.

Post reply on HN