Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is your company sticking to on-premise servers? Why?

news.ycombinator.com

Post reply on HN
Print view View original

151–160 of 782 posts

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#151

Post by mattbeckman »

We spend ~$50k/mo on serverless infrastructure on AWS.

It hurts sometimes, given we were fully colocated about 4 years back, and I know how much hardware that could buy us every month.

However, with serverless infra we can pivot quickly.

Since we're still in the beta stage, with a few large, early access partnerships, and an unfinished roadmap, we don't know where the bottlenecks will be.

For example, we depended heavily on CloudSearch, until it sucked for our use case, so we shifted to Elasticsearch, and ran both clusters simultaneously until we were fully off of CS. If we were to do that on-prem, we'd have to order a lot more hardware (or squeeze in new ES cluster VMs across heavy utilization nodes).

With AWS, a few minutes to launch a new ES cluster, dev time to migrate the data, followed by a few clicks to kill the CloudSearch cluster.

Cloud = lower upfront, higher long term, but no ceiling. On-prem = higher upfront, lower long term, but ceiling.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#152

Post by twoodfin »

varenc wrote: post #78
centimeter wrote: post #62

Earlier quoted context omitted.

We would need something like an X1 instance in terms of RAM, but it's hard to find something on AWS that has a well-tuned balance of RAM/CPU/disk for our needs. A lot of the big specialized instances are tuned for one particular limiting factor (RAM/GPU/storage/CPU/bandwidth/whatever) and I don't recall them having a good selection for "really big everything". Amazon is constantly expanding the selection, so it's pos…

Just curious, can you share general details on the types of services that need really big everything? For most every workload I can think of, as their load increases it's always one resource in particular that's the limiting factor. (RAM/CPU/Storage/etc). So it makes sense to me that AWS instances focus on optimizing one particular resource type. Would be interesting to hear about types of workloads that break this p…

Redlining RAM + CPU is not difficult for the right kind of database application that needs in-memory latency. RAM is a limit on your (hot, at least) data size, and CPU is a limit on your query workload.

In my experience, harder to also balance I/O so you’re close to hitting three limits, but update-heavy transactional workloads can manage it for disk-based DBMS’s.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#153

Post by desc »

1. Our customers run our software on their own machines for security and data-control reasons. As soon as something's running on someone else's hardware, the data is out of your control. Unless you're going to accept the (often massive) cost of homomorphic encryption, AND have a workload amenable to that, it's a simple fact.

2. Everything we do in house is small enough that the costs of running it on our own machines is far less than the costs of working out how to manage it on a cloud service AND deal with the possibility of that cloud service being unavailable. Simply running a program on a hosted or local server is far far simpler than anything I've seen in the cloud domain, and can easily achieve three nines with next to no effort.

Most things which 'really need' cloud hosting seem to be irrelevant bullshit like Facebook (who run their own infrastructure) or vendor-run workflows layered over distributed systems which don't really need a vendor to function (like GitHub/Git or GMail/email).

I'm trying to think of a counterexample which I'd actually miss if it were to collapse, but failing.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#154

Post by kasey_junk »

Bandwidth is the big reason to stay on-prem. Good peering contracts can more than make up for any cloud advantages for bandwidth intensive uses.

Now the hard part is turning those cost advantages into operational improvements instead of deficiencies.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#155

Post by dathinab »

kondro wrote: post #84
2rsf wrote: post #8

Security (we're a bank)

But there are literally 100's (or more) of banks around the world that use AWS for compute. They meet all the security requirements you could want (27001, SOC2, and many others) around how they secure and limit access to servers by staff and have a variety of WORM audit tools and a very comprehensive IAM to help you satisfy these requirements are being met.

I don't know about the parent comments work place, but I don't think any non-US bank should use a AWS (or GCP or Azur).

It's way to risky that changes in local and/or US regulations will require you to move your whole infrastructure to a new provider like in a month or two (which is completely impossible).

Btw. same applies for any bank relying on the infrastructure of a foreign company.

Like e.g. just think about the whole legal mess around CLOUD act.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#156

Post by sixothree »

throwaway6845 wrote: post #37

Mostly, headspace. If I run my own server, I just need to apply my existing Ubuntu sysadmin knowledge. If I use AWS, I have to learn a whole load of AWS-specific domain knowledge, starting with their utterly baffling product names. My time is more valuable than that. Also, sheer cost. Literally everyone I know in my particular part of the industry uses Hetzner boxes. For what I do, it’s orders of magnitude cheaper th…

We're a small business and hosting costs are killing us. Spending $300+ on a small VM that I can only describe as measly really hurts.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#157

Post by skiril »

One of the reason is a legacy systems. Some companies are too tied up to old custom made systems built on old software and hardware virtually not convertible to cloud. You will be surprised but there are big corporation still using AS400 and not planning to switch anytime soon. If you heard in recent news US unemployment system was still built on COBOL... In 2020... Another reason is a cost. I love AWS! Its fantastic to be able to create and launch servers or the whole farm of servers in the matter of minutes! And ability to convert physical servers to virtual and upload them to the cloud is breathtaking! But my monthly bill started at $300 and grew to $18K per month in less than 3 years. And that was for just a few virtual servers with Windows OS and SQL. My company realized that we can have a state of the art datacenter with WmWare and SAN on premises for the fraction of that price. Put second one on the other coast (one of our other offices) and you have your own cloud with better ping and six digits figure saving a year. For the last I would name vendor lock. With vSphere its very easy to move your virtual servers between AWS, Azure and Google (assuming you can afford all 3 and licensing cost of WmWare) but have you ever tried to "download" your server back to premise? It's virtually impossible or made so hard by cloud players trying to keep you up they're in the clouds. With all said I read that Netflix (I believe its Netflix) saving hundreds of millions dollars per year by using Amazon instead of its own servers. I also read somewhere that Dropbox moved away from AWS...

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#158

Post by pojzon »

SamBam wrote: post #123
flokie wrote: post #77

Earlier quoted context omitted.

It sounds like you are operating a cloud.....

Sure... but at least they're not using the cloud! Just a whole bunch of servers!

There is no cloud, its just someone else’s computer :)

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#159

Post by Spooky23 »

It all depends. Outside of SaaS, If you have a mature data center operating model and truly understand your costs, there won’t be a strong cost savings story for many types of workflows.

If you suck and don’t understand costs, or don’t automate, or spend a lot of time eating steak with your Cisco team, you’ll save money... at first.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#160

Post by alexpotato »

zozbot234 wrote: post #150
tgamblin wrote: post #104

I work in Livermore Computing at LLNL. We manage upwards of 30 different compute clusters (many listed here: https://hpc.llnl.gov/hardware/platforms ). You can read about the machine slated to hit the floor in 2022/2023 here: https://www.llnl.gov/news/llnl-and-hpe-partner-amd-el-capita... . All the machines are highly utilized, and they have fast Infiniband/OmniPath networks that you simply cannot get in the cloud. F…

> All the machines are highly utilized, and they have fast Infiniband/OmniPath networks that you simply cannot get in the cloud. It's weird that these networking technologies are not used more in "plain" datacentre settings, since networking latency and throughput has to be a significant challenge to scaling up non-trivial workloads and achieving true datacentre-scale computing. We hear a lot about how to "scale out"…

There are a lot of people from the National Lab super computer world who end up in High Frequency Trading for just the reason you describe.

Specifically, how do you optimize a large cluster of computers to operate at the lowest possible latency. For the National Labs, those computers could be in the lab or with other labs around the world. For the HFT folks, the machines could be in an exchange or spread across multiple exchanges around the world.

Source: I used to be head of Global Latency Monitoring for a HFT.

Post reply on HN