Live data from Hacker News

Ask HN: Is your company sticking to on-premise servers? Why?

news.ycombinator.com

281–290 of 782 posts

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#281

Earlier quoted context omitted.

Cloud servers tend to be more reliable as well if you don't run your own datacenters. We have lost our internet connection or power 3 times in the last year in the office. Its not the end of the world since we can go to 4g for our own usage but if our servers were hosted locally this would be a huge issue.

Disagree; your still at the mercy of whatever last mile is __between__ the cloud and the office. Also any bandwidth limitations; in the US that's still a big deal. Public facing stuff should probably be colo or rented server.

In my case they are public/customer facing services. Our internal stuff like CI runs on premise.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#282

Like many others have pointed out: Cost. I'm the CTO of a moderately sized gaming community, Hypixel Minecraft, who operates about 700 rented dedicated machines to service 70k-100k concurrent players. We push about 4PB/mo in egress bandwidth, something along the lines of 32gbps 95th-percentile. The big cloud providers have repeatedly quoted us an order of magnitude more than our entire fleet's cost....JUST in bandwid…

A few years ago I was trying to start a company and get it off the ground. We had to make decisions on our tech stack and whether we were going to use AWS and build around their infra. Our business was very data heavy and required transferring large datasets from outside to our databases. Even in our early prototypes, we realized that we couldn’t scale cost-effectively on AWS. I figured out that we could colocate and…

You’re not paying Google $5B for raw infra, you’re paying for cloud services like top-tier horizontally scalable databases, global availability, CDNs, datastores of different flavors, and transparently managed monitoring and hardware fault resolution.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#283
post #264

Earlier quoted context omitted.

I used to work at Pacific Northwest National Laboratory. Their flagship computational chemistry application. NWChem, is developed mostly by their own employees. They also collaborate with researchers from other national labs and academia. Development started in the 1990s. When I was at the lab, it was free-as-in-beer but required onerous licensing agreements to get a copy of the source code. It has been open source s…

I appreciate your comment and links (BTW, I'm slightly familiar with NWChem and similar software :-). Having said that, I'm not sure how your comment addresses my question. You are talking about internally-developed software transfer from the lab to external users, whereas my use case is delivering externally-developed commercial software to national labs' users, a value transfer in the opposite direction. Care to co…

I can speak at least a little bit about my field (computational chemistry). A lot of academic software is free and/or open source (like NWChem). These are installed by the cluster support for use by anyone. This is helpful because the software can be optimized beyond what most end users can do.

Beyond that, a lot of academics run custom software or uncommon packages. By definition, many academics are doing something no one else has really done, so they are writing their own software and then running that on the clusters.

There is one large commercial package that takes up a significant amount of time on many clusters -- VASP. But no idea the details of how that ends up on HPC systems :)

My impression from running on lots of clusters is that the software installed is more by 'pull' (user request) than 'push' (cluster dictating what is used). Clusters have to be very accommodating of user software.

If you are working in computational chemistry, I would be interested in hearing more details (if you are willing).

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#284
post #210

Earlier quoted context omitted.

I consider VPSs to be cloud servers. Is this not common?

I mentioned in another comment that I use VPS and not cloud services. I think of cloud as the auto-scaling infrastructure with dynamic pricing. I think of VPS as just sharing a dedicated machine with others, so each one gets a few cores and shares other resources. The implementation of VPSs nowdays is probably more similar to cloud services, where your own space might be moved around to another physical machine witho…

So you consider cloud servers to be what most people call serverless (S3/serverless functions/etc)?

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#285

Earlier quoted context omitted.

This is kind of a big caveat (“Netflix is in the cloud but almost none of the work is done there”), and something I have to mention to non tech decision makers when they say “but Netflix!”. I even have a slide for presentations just for this (“You Are Not Netflix”).

99% of the work is done on the cloud. What comes off of those colo servers is literally just bits streaming from disk to network. There is no transformation or anything. No authentication, no user accounts, no database. Nothing. Just static files served efficiently.

The netflix CDN specifically exists because of cloud bandwidth charges.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#286
post #191

We are a 1000-2000 person company and we have probably on the order of $100M of servers and data centers and whatnot, and I think we spend about 2/3rds of that every year on power/maintenance/rent/upgrades/etc. We don't generally trust cloud providers to meet our requirements for: * uptime (network and machine - both because we are good at reliability [and we're willing to spend extra on it] and because we have lots…

> because we have lots of fancy redundant infrastructure that we can't rely on from cloud companies) Haha. This can't possibly be true.

Just to give a few examples, we are typically much more aggressive with RAID, ECC, redundant network links, redundant time sources, redundant cooling, redundant power supplies, etc. than cloud providers.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#287

We hear from our customers mostly what has been said here: cost and mental overhead. There is a bit of a paradox - companies that plan to grow aggressively are wary of AWS bills chopping their runway in half - they're very aware of _why_ cloud providers give out a year for free to most startups - they recoup that loss very fast once the cash faucet opens up. What really gets me is that most cloud providers promise sc…

I've found that Kubernetes mostly solves this problem. I say mostly because for AI/ML workloads that require GPUs, we still rely on running things on bare metal locally, and deploying with GKE's magic annotations and Deep Learning images. But for anything else, I haven't had an issue going all in on k8s at the beginning, even with very small teams.

Yep! My startup is https://kubesail.com, so I agree :)

As for ML on Kube, I agree, there have been and still are some rough edges. The kernel drivers alone make a lot of out-of-the-box Kubernetes solutions unusable. That said, we've had a lot of success helping people move entirely onto kube - the mental gain alone from ditching the bash scripts or ansible playbooks (etC) alone is pretty freeing.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#288
post #264

Earlier quoted context omitted.

I used to work at Pacific Northwest National Laboratory. Their flagship computational chemistry application. NWChem, is developed mostly by their own employees. They also collaborate with researchers from other national labs and academia. Development started in the 1990s. When I was at the lab, it was free-as-in-beer but required onerous licensing agreements to get a copy of the source code. It has been open source s…

I appreciate your comment and links (BTW, I'm slightly familiar with NWChem and similar software :-). Having said that, I'm not sure how your comment addresses my question. You are talking about internally-developed software transfer from the lab to external users, whereas my use case is delivering externally-developed commercial software to national labs' users, a value transfer in the opposite direction. Care to co…

I misunderstood your original question. When you asked how they procure domain specific application software I thought you meant "how do they get it?," which includes writing it themselves. (Not just procure as in "procurement" processes.)

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#289
post #240

Earlier quoted context omitted.

99% of the work is done on the cloud. What comes off of those colo servers is literally just bits streaming from disk to network. There is no transformation or anything. No authentication, no user accounts, no database. Nothing. Just static files served efficiently.

> What comes off of those colo servers is literally just bits streaming from disk to network So... the core of their business?

Not at all. It was so “not core” that it was outsourced.

The core of their business is recommendations, encoding, and authentication. All of those are done 100% on the cloud.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#290

Earlier quoted context omitted.

99% of the work is done on the cloud. What comes off of those colo servers is literally just bits streaming from disk to network. There is no transformation or anything. No authentication, no user accounts, no database. Nothing. Just static files served efficiently.

The netflix CDN specifically exists because of cloud bandwidth charges.

That’s not true. It exists because Akamai and friends couldn’t serve the data fast enough.
Post reply on HN