Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is your company sticking to on-premise servers? Why?

news.ycombinator.com

Post reply on HN
Print view View original

621–630 of 782 posts

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#621

Post by StreamBright »

mmmBacon wrote: post #276
AgentK20 wrote: post #207

Like many others have pointed out: Cost. I'm the CTO of a moderately sized gaming community, Hypixel Minecraft, who operates about 700 rented dedicated machines to service 70k-100k concurrent players. We push about 4PB/mo in egress bandwidth, something along the lines of 32gbps 95th-percentile. The big cloud providers have repeatedly quoted us an order of magnitude more than our entire fleet's cost....JUST in bandwid…

A few years ago I was trying to start a company and get it off the ground. We had to make decisions on our tech stack and whether we were going to use AWS and build around their infra. Our business was very data heavy and required transferring large datasets from outside to our databases. Even in our early prototypes, we realized that we couldn’t scale cost-effectively on AWS. I figured out that we could colocate and…

> why does anyone use AWS, you can do this on your own way cheaper.

Because they can calculate TCO correctly.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#622

Post by StreamBright »

RachelF wrote: post #340
mmmBacon wrote: post #276

Earlier quoted context omitted.

A few years ago I was trying to start a company and get it off the ground. We had to make decisions on our tech stack and whether we were going to use AWS and build around their infra. Our business was very data heavy and required transferring large datasets from outside to our databases. Even in our early prototypes, we realized that we couldn’t scale cost-effectively on AWS. I figured out that we could colocate and…

Dropbox did the same thing a few years back - moved everything from Amazon S3 to their own storage. My guess is they did it for cost reasons.

How many other companies do you know that have same problem as Dropbox?

An average company that uses does not provide IT infrastructure (like storage in this case) to a massive amount of clients.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#623

Post by bsenftner »

wheelerwj wrote: post #352
koheripbal wrote: post #265

Earlier quoted context omitted.

Same here - CTO of medium sized company. Our IT infra costs are 1/10th the cost of cloud, simply because I happen to be comfortable having on-premise machine and working on them (sometime myself). We have two dozen servers in two locations. It's more time to setup, but maintenance is actually quite low.

24 servers, interesting. That seems like a small enough setup that I would definitely have left it in the cloud. Much more than that and i think it makes sense to start moving to physical servers. But 24 I would have guessed would be cheaper to maintain in EC2. Are you setup with one or two racks in each location and maybe one full time IT/Sys admin at each location?

At my own startup we ran 17 colocated servers, each a high compute resource, at an monthly expense of $600, using purchased hardware that cost $55K. That is versus $96K per month that would be the same infrastructure at AWS.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#624

Post by JanisL »

StreamBright wrote: post #621
mmmBacon wrote: post #276

Earlier quoted context omitted.

A few years ago I was trying to start a company and get it off the ground. We had to make decisions on our tech stack and whether we were going to use AWS and build around their infra. Our business was very data heavy and required transferring large datasets from outside to our databases. Even in our early prototypes, we realized that we couldn’t scale cost-effectively on AWS. I figured out that we could colocate and…

> why does anyone use AWS, you can do this on your own way cheaper. Because they can calculate TCO correctly.

Is it just me or is Total Cost of Ownership being talked about way less often these days?

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#625

Post by nickjj »

f0ff wrote: post #571
mmmBacon wrote: post #276

Earlier quoted context omitted.

A few years ago I was trying to start a company and get it off the ground. We had to make decisions on our tech stack and whether we were going to use AWS and build around their infra. Our business was very data heavy and required transferring large datasets from outside to our databases. Even in our early prototypes, we realized that we couldn’t scale cost-effectively on AWS. I figured out that we could colocate and…

Avoiding cloud works well for small requirements too - worked for a company that ran a global store; we've opted for dedicated servers on OVH, all together had about 40 of them. It was 1/10th the price of running the infrastructure on AWS. We still used S3 though.

Yeah, I'm seeing this too.

I recently talked with someone who said their costs dropped in half from switching off a major cloud provider to OVH's dedicated servers. Performance went way up too.

For $200 / month you can get a machine with a high end Xeon 8 core (16 threads) CPU, 128 GB of memory, 4.5 TB of SSD space across multiple disks with unlimited incoming / outgoing traffic and anti-DDoS protection.

No reputable cloud provider is going to come close to that price with their listed prices.

Of course there's trade offs like not being able to click a button and get it within 2 minutes for most server types but if you have consistent and predictable traffic, going with an over provisioned dedicated server seems like a very viable option.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#626

Post by StreamBright »

mmmBacon wrote: post #292
mayank wrote: post #282

Earlier quoted context omitted.

You’re not paying Google $5B for raw infra, you’re paying for cloud services like top-tier horizontally scalable databases, global availability, CDNs, datastores of different flavors, and transparently managed monitoring and hardware fault resolution.

Yes, you’re paying for a ton of stuff there that you probably don’t use and then are susceptible to bugs that have nothing to do with your use case. At $5B it would not cost anywhere near that much to replicate. Your infra would be better tailored to your workloads and your sw teams which would further drive costs down. The upsides are that you only drive features you need and keep things simple. The downside is that…

If I use S3 + EMR what exactly falls into the category of "a ton of stuff there that you probably don’t use"?

>> Your infra would be better tailored to your workloads

How do you scale elastically with an on-prem infra? What part do you tune more to your own workload when an average company cannot even tune the GC for Java for their own workload?

Your claims do not reflect on reality, it is rather your imagination. I have migrated countless companies to the cloud, and almost every single migration was driven by a single factor: cost. Everything else was an added bonus: inscrased security, availability and elasticity.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#627

Post by nemacol »

This conversation sounds a lot like the debate around globalization and outsourcing manufacturing to me. It might be a stretch but there is something here.

There is room for both Cloud and On-Prem to exist. This endless drive by industry to push everyone to cloud infrastructure and SaaS, in my humble opinion, will look exactly like the whole supply chain coming from the east during a pandemic.

The economics of it look great in a lot of use cases, but putting our whole company at the mercy of a few providers sounds terrible to me. Even more so when I see posts on HN about folks getting locked out of their accounts with little notice.

It does not take much to bring our modern cloud to a grinding halt. For example, a mistake by an mostly unheard of ISP lead to a massive outage not less than a year ago(1).

It was amazing to see the interconnections turn to cascading issues. 1 ISP goofs. 1-2 major providers have issues and the trickle down effect was such that even services that thought they were immune from cloud issues were realizing that they rely on a 3rd party that relies on a different 3th party that uses cloudflare or AWS.

So, even though I think the cloud is (usually) secure, stable, resilient, etc... I still advocate for its use in moderation and for 2 main use cases.

1 - elastic demands. Those non-critical systems that add some value or make work easier. Things we could do without for several days and not hurt the business much.

2 - DR / Backup / redundancy. We have a robust 2 data center / DR fail over system. Adding cloud components to that seems reasonable to me.

(1)https://slate.com/technology/2019/06/verizon-dqe-outage-inte...

Edit: Spelling and clarity

Edit2: New reasons to stay on prem are happening all the time. https://www.bleepingcomputer.com/news/security/microsofts-gi...

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#628

Post by dijit »

user5994461 wrote: post #584
wongarsu wrote: post #530

Earlier quoted context omitted.

42U times $5000 divided by 3 years is still only $5833 per month per rack. Add the $1400 and we have a total amortized rack cost of $7233 per month. (naively assuming you could fill every spot in the rack with servers). But servers can't be included in a "Uber spends nearly 200M/yr alone on real estate for their datacenters" figure anyway.

The rack is $2800 per month, not $1400. The lowest advertised pricing does not include enough power to supply half of what's in the rack. Add $200 per server per month to have a gigabit uplink. Add $4000 per server per lifetime for VmWare licenses. These are reasonable estimates of course. Could be multiple of that depending on what hardware is used and what colo. Physical hardware easily gets as expensive as any clo…

Not easily; you’re making the argument that there can be high expenses for collocating but you’re talking about a -lot- of computational power.

I know that hardware varies a lot for sure, but for context I put together 3 racks 50% density with an 800gbit backplane for around 18k eur/mo.

I spared no expense, official juniper QSFPs (which are egregiously overpriced) and top of the line Dell servers with full out of band licenses.

And once there: interconnected bandwidth and IOPs became “free” (or, no extra charge).

We put the same application in cloud And it costs us 40k eur/mo with a heavy amount of optimisation, with half-sized instances and an aggressive optimisation in bandwidth/iops.

Clouds “sticker price” to us is 4x that of physical. You can buy a lot of human time for that price.

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#629

Post by TuringNYC »

Here is how we went about w/ CSPs (AWS, Azure, GC, Oracle). Thoughts welcome

Getting Started --> Definitely go w/ CSPs. No need to worry about infra.

Pre Product Market Fit + Steady Growth --> On Premise, because CSPs might be expensive until you find a consistently profitable business.

Pre Product Market Fit + HyperGrowth --> CSPs since you wont be able to keep up [we never got to this stage]

Product Market Fit w/ Sustainable Good Margins --> CSPs, pay to remove the headache [we never got to this stage]

Side Note: w/ GPUs, CSPs rarely make sense

Re: Ask HN: Is your company sticking to on-premise servers? Why?

#630

Post by tjbiddle »

llarsson wrote: post #505
tjbiddle wrote: post #375

Earlier quoted context omitted.

Certainly nothing powerful :-) I can get away with t3.nano and t3.micro for what I'm doing at the moment. But the beauty of cloud, is that I can scale up when I eventually need it. 5x t3.nano will be ~$25/mo 5x t3.micro will be ~$50/mo All of my AMIs are EBS optimized and require a minimum of 8GB for the root drive (Although they only use ~1.6GB. Not bothering to hack around this to save a buck.) So that'll be 40GB E…

And if you went with reserved instances, what does it come out to? I guess that it is fair to assume that you will need at least this level of compute power for at least a year, right?

No idea. Generally reserved instances will give you 30-75% off though, I believe.

This is for a hobby project, so no way I'm committing to that, haha.

I'm also building the architecture so that I can easily completely destroy it and restore it from backups very quickly.

So, depending what I do with it, may be destroying it regularly and just bringing online when I want it.

Post reply on HN