Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is your company sticking to on-premise servers? Why?

news.ycombinator.com

Post reply on HN
Print view View original

331–340 of 782 posts

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#331

Post by SteveNuts »

YarickR2 wrote: post #277
AgentK20 wrote: post #207

Like many others have pointed out: Cost. I'm the CTO of a moderately sized gaming community, Hypixel Minecraft, who operates about 700 rented dedicated machines to service 70k-100k concurrent players. We push about 4PB/mo in egress bandwidth, something along the lines of 32gbps 95th-percentile. The big cloud providers have repeatedly quoted us an order of magnitude more than our entire fleet's cost....JUST in bandwid…

Gaming company senior reliability engineer/officer here, with the same tune. We're operating thousands of servers on three continents (NA, Europe, Asia), running our own game distribution network and basically doing as much as possible on prem. Every time we're trying to play by the cloud provider rules, we're getting stung with outrageous storage and bandwidth bills. I'm pushing for in-house k8s deployment to increa…

Are you looking at a specific vendor for your on prem k8s?

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#332

Post by localhost »

MCthrowaway12 wrote: post #218
AgentK20 wrote: post #207

Like many others have pointed out: Cost. I'm the CTO of a moderately sized gaming community, Hypixel Minecraft, who operates about 700 rented dedicated machines to service 70k-100k concurrent players. We push about 4PB/mo in egress bandwidth, something along the lines of 32gbps 95th-percentile. The big cloud providers have repeatedly quoted us an order of magnitude more than our entire fleet's cost....JUST in bandwid…

[offtopic] Awesome seeing you here! I used to be super active on Hypixel when I was in high school (I was actually #1 on the leaderboards for one of the games for over a year). One of my first large scale programming projects ever was creating a Hypixel mod for my guild. Years later I am now a software engineer working at Google on the YouTube algorithm!

Since we're OT anyway, thank you for all the work you do on the YouTube algorithm. It has brought a tremendous amount of value into my life!

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#333

Post by andrenotgiant »

AgentK20 wrote: post #207

Like many others have pointed out: Cost. I'm the CTO of a moderately sized gaming community, Hypixel Minecraft, who operates about 700 rented dedicated machines to service 70k-100k concurrent players. We push about 4PB/mo in egress bandwidth, something along the lines of 32gbps 95th-percentile. The big cloud providers have repeatedly quoted us an order of magnitude more than our entire fleet's cost....JUST in bandwid…

It sounds like you've got a good setup going with colo, but just as a way of illustrating some of the small providers lower bandwidth costs:

DigitalOcean gives 3TB bandwidth on a $15 2CPU/2GBMEM/60GBSSD instance. If you ran 1350 of them it'd cost you ~$20k/month and get you your 4PB egress within bandwidth allowance.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#334

Post by tigerstripe »

tjbiddle wrote: post #241
horsawlarway wrote: post #217

I'm slowly coming to the complete opposite opinion you seem to have. I've worked almost entirely for companies that run services in various cloud infrastructures - Azure/Heroku/Aws/GCP/Other. I recently started a tiny 1 man dev shop in my spare time. Given my experience with cloud services it seemed like a no brainer to throw something up in the cloud and run with it. Except after a few months I realized I'm in an in…

What's the time trade-off? I've been drawing out my plans lately for a hobby project, all 100% on AWS. Being able to spin up my entire infrastructure with Terraform, build out images with Packer, setup rules for off-site backups, ensure everything is secure to the level I want it, etc. - It takes me next to no time at all. I can't imagine buying hardware, ensuring my home is setup with proper Internet, configuring ev…

What kind of setup did you have for 5 servers at $50/mo on AWS? Interested to know - our EC2 instances that are about 1/4 as powerful as a laptop cost $60+/mo

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#336

Post by mappu »

_bxg1 wrote: post #317

One of my favorite things (at least for personal projects) about using the cloud is so-called "platform as a service" systems like Heroku, where I don't have to get down in the weeds, I just push code and the process starts (or restarts). Is there something like that I could use on my own hardware? I just want to do a fresh Linux install, install this one package, and start pushing code from elsewhere, no other confi…

You're looking for Dokku

Same git push deploys, heroku-compatible buildpacks or Dockerfiles, all on your own hardware, MIT license.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#337

Post by lmeyerov »

ablekh wrote: post #313
sseagull wrote: post #283

Earlier quoted context omitted.

I can speak at least a little bit about my field (computational chemistry). A lot of academic software is free and/or open source (like NWChem). These are installed by the cluster support for use by anyone. This is helpful because the software can be optimized beyond what most end users can do. Beyond that, a lot of academics run custom software or uncommon packages. By definition, many academics are doing something…

I appreciate your feedback. My question above is not about the mechanics of deploying commercial scientific software (it certainly somewhat varies between fields and organizations, though common themes and methods are pretty standard). It is about the procurement processes within national labs ecosystem. That is, about governmental gating / filters "in front of" vendors' go-to-market strategy and processes (RFPs, pil…

We do visual GPU graph tech close to core national lab missions (sec, fraud, misinfo, genetics, social, ...) so are in a similar boat.

While I love the folks doing related work at PNNL (and am inspired!), I also recognize, done wrong, we can be existentially threatening to internal groups building related things.

Instead of aggressively persuing contracts and spending effort getting into nasty politics, we let the users come to us: we are there when the internal tools groups rather do other stuff vs do the multi-year investment. We are now focusing a lot more on enabling researchers to find and try us on public cloud / the web -- this road takes literally years longer for reaching those teams, but these are government labs, so that's the default assumption to begin with. (Likewise, not the cornerstone of our revenue.) Years later, folks are starting to write RFPs using us.

As a scientist, I love the community, and respect that we should assist it appropriately.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#338

Post by kelnos »

centimeter wrote: post #52
H8crilA wrote: post #50

Earlier quoted context omitted.

Hundreds of terabytes a day is really not that much, depends on what latency can you accept. I often run computations over datasets that are petabytes in size, just for my own needs. A big data move would be at least tens of petabytes or more like hundreds, or thousands. Also surprised about latency, latency from what to what? Big cloud providers have excellent globally spanning networks. Long distance networking is…

> Hundreds of terabytes a day is really not that much How much would it cost to move this across boxes in EC2? I actually don't know, that's not a rhetorical question. A lot of our servers have 10-40gbit links that we saturate for minutes/hours at a time, which I suspect would be expensive without the kind of topology optimization we do in our datacenters. > Also surprised about latency We've spent a surprising amoun…

> One obvious example is that EC2 has "scheduled maintenance events" where they force you to reboot your box. This would cost us a lot of money (mostly in dev time, to work around it).

It's interesting that you say this, because I think this just boils down to how you treat failures. Boxes will fail. It's inevitable, even if you run your own hardware.

We treat failure as an every-day event and have set up our systems so it doesn't matter. One box fails, an automated system notices and brings up another to replace it, while the remaining boxes take slightly more load for a few minutes.

Sure, that kind of failure tolerance doesn't come for free. But, again, you're going to have failures, no matter how much work you put into reliability (which also doesn't come for free!).

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#339

Post by jarym »

my 2cents: the 'big' clouds (AWS, GCP, Azure) and the big-brand clouds (Oracle, IBM, etc.) are attractive for BigCustomerCo because:

1. Replace capital expenditure of in-house infrastructure + staff with OpEx that can be dialled down

2. Get to benefit from the economies of scale that the cloud vendors get (those Intel CPUs get a lot cheaper when purchased in the 1000s)

3. Get to leverage big shiny new tech like Big Data and AI that's 'advertised' as 'out-of-the-box'

My only concern really is that the big cloud players are all fighting for dominance and market share. What happens in the next 5-10 years time when they start raising prices? Different kind of lock-in - customers won't have the expertise in-house to migrate stuff back.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#340

Post by RachelF »

mmmBacon wrote: post #276
AgentK20 wrote: post #207

Like many others have pointed out: Cost. I'm the CTO of a moderately sized gaming community, Hypixel Minecraft, who operates about 700 rented dedicated machines to service 70k-100k concurrent players. We push about 4PB/mo in egress bandwidth, something along the lines of 32gbps 95th-percentile. The big cloud providers have repeatedly quoted us an order of magnitude more than our entire fleet's cost....JUST in bandwid…

A few years ago I was trying to start a company and get it off the ground. We had to make decisions on our tech stack and whether we were going to use AWS and build around their infra. Our business was very data heavy and required transferring large datasets from outside to our databases. Even in our early prototypes, we realized that we couldn’t scale cost-effectively on AWS. I figured out that we could colocate and…

Dropbox did the same thing a few years back - moved everything from Amazon S3 to their own storage.

My guess is they did it for cost reasons.

Post reply on HN