Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is your company sticking to on-premise servers? Why?

news.ycombinator.com

Post reply on HN
Print view View original

471–480 of 782 posts

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#471

Post by Fornax96 »

brownbat wrote: post #387
AgentK20 wrote: post #207

Like many others have pointed out: Cost. I'm the CTO of a moderately sized gaming community, Hypixel Minecraft, who operates about 700 rented dedicated machines to service 70k-100k concurrent players. We push about 4PB/mo in egress bandwidth, something along the lines of 32gbps 95th-percentile. The big cloud providers have repeatedly quoted us an order of magnitude more than our entire fleet's cost....JUST in bandwid…

I remember a post here a while back from a guy running Bitcoin mining in his dorm room. One day he realized he could offer his spare cycles to grad students with high computing workloads and undercut cloud computing prices while increasing what he made far over just mining. It's very weird that we haven't fully arbitraged $/instruction to a single (low) price yet (or storage/hosting, whichever). If only there were an…

> If only there were an Uber for unused cycles or storage... let everyone turn their unused capacity into mini AWSs with a common interface and safety and reliability guarantees

This is exactly what Sia.tech is building for storage. A competitive marketplace for storage providers where anyone can set up a host and start selling their spare storage space. It's not completely finished yet, but it's pretty close.

A bunch of companies are already building products on top of it (including me)

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#473

Post by chillfox »

flukus wrote: post #273
horsawlarway wrote: post #217

I'm slowly coming to the complete opposite opinion you seem to have. I've worked almost entirely for companies that run services in various cloud infrastructures - Azure/Heroku/Aws/GCP/Other. I recently started a tiny 1 man dev shop in my spare time. Given my experience with cloud services it seemed like a no brainer to throw something up in the cloud and run with it. Except after a few months I realized I'm in an in…

> Backups are still offsite, so if my place burns I'm just out on uptime There are plenty of exceptions, but I generally find uptime is given way too much consideration for the value it gives. Most companies up to the national level generally only require about 12 hours a day of uptime from most of their software, more than that is a nice to have, but the company won't collapse if you turn off the servers overnight b…

Yeah, I find the constant focus on ~100% uptime a bit crazy.

I used to work for one of Australia's largest bookmakers and we would regularly switch off the datacenter overnight to do some work on the servers. It didn't seem to impact the growth of that business one bit. There were times of course where uptime equalled cash, it just wasn't all the time.

For my personal hobby projects, I would actually rather have downtime than a surprise bill if something gets popular.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#474

Post by anshumania »

sidibe wrote: post #325
jedberg wrote: post #289

Earlier quoted context omitted.

Not at all. It was so “not core” that it was outsourced. The core of their business is recommendations, encoding, and authentication. All of those are done 100% on the cloud.

Sure but probably 99% of the > 30% of all internet traffic. is outside of the cloud.

Anyone know the bill Netflix has for running on the cloud ?

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#475

Post by mrweasel »

We sell hosting to large number of different customers who for whatever reason, mostly legal, are required to keep data within the borders of Denmark. There is no Google, Amazon, Azure or DigitalOcean data centers in Denmark, so cloud isn't an option for them.

Regarding cost, well it depends. We try to help customers to move to cloud hosting if it's cheaper for them. It almost always will be if they take advantage of the features provided by the cloud providers. If you just view for instance AWS as a VMware in the cloud, then we can normally host the virtual machines for you cheaper and provide better service.

You have to realize that many companies aren't developing software that's ready for cloud deployment. You can move it to EC2 instance, but that's not taking advantage of the feature set Amazon provides, and it will be expensive and support may not be what you expect. You can't just call up Amazon and demand that they fix your specific issue.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#476

Post by winter_blue »

AgentK20 wrote: post #458
winter_blue wrote: post #446

Earlier quoted context omitted.

> about 700 rented dedicated machines to service 70k-100k concurrent players Slightly off-topic, but it sounds like a single machine can't handle more than 150 concurrent players. How is this so? Is Hypixel Minecraft that resource intensive? What's the bottleneck — it is the CPU, or RAM, or the network latency(?) per machine, or is it something else? > We push about 4PB/mo in egress bandwidth With 70k players, 4PB /…

About 60 of those machines are custom L7 frontend load balancers, and an additional 40-50 are database and dev machines. As for the actual gameservers, that's about the numbers we expect on average (150/box), although it varies wildly by gametype. Simple games of Skywars or Murder Mystery with no AI and an ultra-slimmed-down world we can fit 300+ players per box and are CPU-bound. Housing or Skyblock servers with ful…

Thanks for the insightful comment.

I saw you mentioned in another thread Minecraft performs better when the CPU has better single-threaded performance. I'm guessing, going forward into the future, you'd probably want to build machines with Zen 2 AMD Ryzen CPUs (or future Zen 3 Ryzens), like the Threadripper 3960X (which has a base clock of 3.8 GHz, 24 physical CPU cores, and costs ~$1200 — so $50 for each of those cores). Or, the AMD Ryzen 9 3950X, which is about $40 per core (when you get it on sale), and despite a lower nominal base clock actually performs better on single-threaded benchmarks (e.g. https://www.cpubenchmark.net/singleThread.html).

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#478

Post by t0mas88 »

wheelerwj wrote: post #352
koheripbal wrote: post #265

Earlier quoted context omitted.

Same here - CTO of medium sized company. Our IT infra costs are 1/10th the cost of cloud, simply because I happen to be comfortable having on-premise machine and working on them (sometime myself). We have two dozen servers in two locations. It's more time to setup, but maintenance is actually quite low.

24 servers, interesting. That seems like a small enough setup that I would definitely have left it in the cloud. Much more than that and i think it makes sense to start moving to physical servers. But 24 I would have guessed would be cheaper to maintain in EC2. Are you setup with one or two racks in each location and maybe one full time IT/Sys admin at each location?

24 dedicated dual Xeon boxes can easily be comparable to 100 to 200 m5.2xlarge EC2 instances. So it's relatively small but not tiny, you're going to be paying at least 25k / month for that (inc reserved instance discounts)

So if you use a lot of bandwidth on top of it (can be another 25-50k at AWS) this could reach into to levels where it's worth it to hire two devops guys to run your own in combination with some SLA/management agreements with a colo.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#479

Post by Areading314 »

In some industries, cloud is not an option. For example, certain privacy laws like HIPAA preclude uploading data to third parties, in which case, you need things to be on-prem. There are also a lot of places in the world where internet access is limited. Sometimes you need to solve problems beyond the simple "web saas in cloud" use case

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#480

Post by nihil75 »

sudhirj wrote: post #397

The way I think about it this: not using the cloud is like building your own code editor or IDE after assembling your own laptops and desktops. It may make you happier and it’s a great hobby, but if you’re trying to run a business you need to do a cost benefit analysis. We currently have double digit petabytes of data stored in our own data centres, but we’re moving it to S3 because we have far better things to do wi…

You're not building everything from scratch on-prem. There are excellent tools for deploying and managing infrastructure like Terraform, Ansible, Puppet that funny enough are used to deploy to cloud as well. Add a self-hosted Kubernetes cluster to that and your on-prem is not that different than a cloud.

As for ops people - you might not need an engineer to replace failed hard-drives, but you'll need a DevOps person to manage Cloud Formation templates and such, and they cost more.

Post reply on HN