Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is your company sticking to on-premise servers? Why?

news.ycombinator.com

Post reply on HN
Print view View original

71–80 of 782 posts

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#71

Post by centimeter »

willart4food wrote: post #68
centimeter wrote: post #36

We are a 1000-2000 person company and we have probably on the order of $100M of servers and data centers and whatnot, and I think we spend about 2/3rds of that every year on power/maintenance/rent/upgrades/etc. We don't generally trust cloud providers to meet our requirements for: * uptime (network and machine - both because we are good at reliability [and we're willing to spend extra on it] and because we have lots…

Without revealing toomuch: What industry are you in? Are you concentrated in 1 geo location or... across USA / across 1 county / global?

> What industry are you in

Something plumbing-y; the company is not well known

> 1 geo location or...

Global, although probably 50% in the US

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#73

Post by jm4 »

centimeter wrote: post #36

We are a 1000-2000 person company and we have probably on the order of $100M of servers and data centers and whatnot, and I think we spend about 2/3rds of that every year on power/maintenance/rent/upgrades/etc. We don't generally trust cloud providers to meet our requirements for: * uptime (network and machine - both because we are good at reliability [and we're willing to spend extra on it] and because we have lots…

What kind of business is that? That level of spending seems insane based on headcount. At least without knowing anything about the business. Are you running public facing services or something? What does that look like as a percentage of revenue?

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#74

Post by centimeter »

jm4 wrote: post #73
centimeter wrote: post #36

We are a 1000-2000 person company and we have probably on the order of $100M of servers and data centers and whatnot, and I think we spend about 2/3rds of that every year on power/maintenance/rent/upgrades/etc. We don't generally trust cloud providers to meet our requirements for: * uptime (network and machine - both because we are good at reliability [and we're willing to spend extra on it] and because we have lots…

What kind of business is that? That level of spending seems insane based on headcount. At least without knowing anything about the business. Are you running public facing services or something? What does that look like as a percentage of revenue?

Infrastructure. Datacenter costs are a relatively high percentage of revenue. No public facing services - only large clients.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#75

Post by nitwit005 »

We're looking at moving from AWS to having some machine space rented in two data centers. The reason is purely cost.

There are still some computers on site due to equipment being tied to it, telephony stuff, etc.

My last company was looking at "moving to the cloud", with the idea that its data centers were too expensive, but found out that the cloud solutions would be even more expensive, despite possible discounts due to the size. They still invested in it due to some Australia customers wanting data to be located there.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#76

Post by chime »

1. 500TB of storage for 3-6mo of CCTV footage.

2. Bought a hanful of $700 24 core Xeons on eBay 2 years ago for 24/7 data crunching. Equivalent cloud cost was over $3000/mo. On-Prem paid off within a month!

3. Nutanix is nice. Awesome performance for the price and almost no maintenance. Got 300+ VDI desktops and 50+ VMs with 1ms latency.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#77

Post by flokie »

centimeter wrote: post #74
jm4 wrote: post #73

Earlier quoted context omitted.

What kind of business is that? That level of spending seems insane based on headcount. At least without knowing anything about the business. Are you running public facing services or something? What does that look like as a percentage of revenue?

Infrastructure. Datacenter costs are a relatively high percentage of revenue. No public facing services - only large clients.

It sounds like you are operating a cloud.....

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#78

Post by varenc »

centimeter wrote: post #62
sneak wrote: post #56

Earlier quoted context omitted.

> We also have a number of machines with specs that would be hard to get e.g. on AWS. What specs are those? I was under the impression AWS has everything from extremely tiny to giant terabytes-of-ram-for-SAP instance types.

We would need something like an X1 instance in terms of RAM, but it's hard to find something on AWS that has a well-tuned balance of RAM/CPU/disk for our needs. A lot of the big specialized instances are tuned for one particular limiting factor (RAM/GPU/storage/CPU/bandwidth/whatever) and I don't recall them having a good selection for "really big everything". Amazon is constantly expanding the selection, so it's pos…

Just curious, can you share general details on the types of services that need really big everything?

For most every workload I can think of, as their load increases it's always one resource in particular that's the limiting factor. (RAM/CPU/Storage/etc). So it makes sense to me that AWS instances focus on optimizing one particular resource type.

Would be interesting to hear about types of workloads that break this pattern. (Or maybe it's a few different types of workloads/services that need to be tightly integrated on one machine?)

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#79

Post by anp »

aspyct wrote: post #28
reacharavindh wrote: post #25

Earlier quoted context omitted.

I wrote about it here: https://aravindh.net/post/sysadmin/ > What form do these jobs have? Mostly batch jobs written as bash scripts. Occasionally, some users run singularity containers. But, all through SLURM. > How do you manage workloads? SLURM > How do you manage resources? As a sysadmin, my inventory is via Ansible. All activity on servers happen via Ansible only. > Do users have quota for compute and storage? Y…

Great article, thanks for sharing :)

I used to work in an environment not dissimilar from what's described above. The uni has some hardware details here: https://in.nau.edu/hpc/details/. Many of my answers would be similar, although the presence of physics, astronomy, and a few other areas motivates GPUs these days apparently. I was doing genomics workloads in the 900-1200gb RAM range with 30+ cores.

There's a pretty interesting NSF-wide project for managing clusters in a more commoditized way as part of https://www.xsede.org. You might describe it as a "private heterogeneous almost-cloud" in its goals? That might be saying a bit much.

Density, latency, storage throughput, etc. were in favor of DIY (plus the pricey professionals to run it) rather than cloud offerings. When I was there (2016) I did some basic math for being able to use a cloud provider for some of our lighter workloads when the local cluster was loaded down. Astronomical without a contract, which is quite the thing to set up, etc.

Worth noting that while they do alright, NAU (first link) is hardly a top-tier university with bleeding edge technical requirements.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#80

Post by HatchedLake721 »

centimeter wrote: post #52
H8crilA wrote: post #50

Earlier quoted context omitted.

Hundreds of terabytes a day is really not that much, depends on what latency can you accept. I often run computations over datasets that are petabytes in size, just for my own needs. A big data move would be at least tens of petabytes or more like hundreds, or thousands. Also surprised about latency, latency from what to what? Big cloud providers have excellent globally spanning networks. Long distance networking is…

> Hundreds of terabytes a day is really not that much How much would it cost to move this across boxes in EC2? I actually don't know, that's not a rhetorical question. A lot of our servers have 10-40gbit links that we saturate for minutes/hours at a time, which I suspect would be expensive without the kind of topology optimization we do in our datacenters. > Also surprised about latency We've spent a surprising amoun…

Seems you’re trying to bring your on-premise structure and concepts to the cloud, that won’t work. EC2 instances are cattle, not pets.

I believe I saw a slide that average lifespan of an EC2 instance at Netflix is 14 minutes.

I’m not necessarily saying cloud will work for your setup, but you can’t compare like that.

Post reply on HN