Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS to bare metal two years later: Answering your questions about leaving AWS

oneuptime.com

Post reply on HN
Print view View original

371–380 of 513 posts

Re: AWS to bare metal two years later: Answering your questions about leaving AWS

#371

Post by bigstrat2003 »

ecshafer wrote: post #235
jsiepkes wrote: post #39

Earlier quoted context omitted.

This is exactly the rhetoric Microsoft used in the 00's with it's "Get the facts" marketing campaign against Linux and open-source: "Never mind the costs, think about the people hours you are saving!". It wasn't as simple as that then, at it's still not as simple as that now.

This is true, but also really funny considering that even today the average windows sysadmin can still barely use powershell and relies on console clicking and batch scripts. A good unix admin can easily admin 10-100x the machines as a windows admin, and this was more true back in the early 00s. So the marketing on getting the facts was absolutely false.

Citation needed on that one. I've only worked with a minority of Windows sysadmins who are as incompetent as you say. And yeah, of course a good unix admin can run circles around a bad windows one, but the converse is just as true. A good Windows admin can run circles around a bad unix one. It has nothing to do with the operating systems and everything to do with technical competence of the individual.

Re: AWS to bare metal two years later: Answering your questions about leaving AWS

#372

Post by chickensong »

Aurornis wrote: post #153
bilekas wrote: post #11

I'm so surprised there is so much pushback against this.. AWS is extremely expensive. The use cases for setting up your system or service entirely in AWS are more rare than people seem to realise. Maybe I'm just the old man screaming at cloud (no pun intended) but when did people forget how to run a baremetal server ? > We have 730+ days with 99.993% measured availability and we also escaped AWS region wide downtime…

> I'm so surprised there is so much pushback against this.. AWS is extremely expensive. I see more comments in favor than pushing back. The problem I have with these stories is the confirmation bias that comes with them. Going self-hosted or on-premises does make sense in some carefully selected use cases, but I have dozens of stories of startup teams spinning their wheels with self-hosting strategies that turn into…

All valid and important points, but missing a painful one, also rarely represented in threads like this: flaky hardware.

Almost every bare metal success story paints a rosy picture of perfect hardware (which thankfully is often the case), or basic hard failures which are easily dealt with. Disk replacement or swapping 1u compute nodes is expected and you probably have spares on hand. But it's a special feeling to debug the more critical parts that likely don't have idle spares just sitting around. The raid controller that corrupts it's memory, reboots, and rolls back to it's previous known-good state. The network equipment that locks up with no explanation. Critical components that worked flawless for months or years, then shit the bed, but reboot cleanly.

Of course everyone built a secure management vlan and has remote serial consoles hooked up to all such devices right? Right? Oh good, they captured some garbled symbols. The vendor's first tier of support will surely not be outsourced offshore or read from a script, and will have a quick answer that explains and fixes everything. Right?

The cloud isn't always the right choice, but if you can make it work, it sure is nice to not deal with entire categories of problems when using it.

Re: AWS to bare metal two years later: Answering your questions about leaving AWS

#373

Post by jdsully »

darkwater wrote: post #48

The core of this success is this, IMO: > Our workload is 24/7 steady. We were already at >90% reservation coverage; there was no idle burst capacity to “right size” away. If we had the kind of bursty compute profile many commenters referenced, the choice would be different. Which TBH applies to many, many places, even if they are not aware of it.

Even if you have that you'll find AWS is "out of stock" and wants you to create reservations that essentially cost the same as just having the machine 24/7.

Re: AWS to bare metal two years later: Answering your questions about leaving AWS

#374

Post by Hilift »

m-gasser wrote: post #158
Hilift wrote: post #50

Earlier quoted context omitted.

> The direct cost is the easy part I don't think it is easy. I see most organizations struggle with the fact that everything is throttled in the cloud. CPU, storage, network. Tenants often discover large amounts of activity they were previously unaware of, that contributes to the usage and cost. And there may be individuals or teams creating new usages that are grossly impacting their allocation. Did you know there i…

> Did you know there is a setting in MS SQL Server that impacts performance by an order of magnitude when sending/receiving data from the Cloud to your on-premises servers? It's the default in the ORM generated settings. Sounds interesting, which setting is that?

Multiple Active Result Sets (MARS). During large query responses or bulk loads, "full" packets cause an additional packet to be sent over the wire with about five bytes to hold the MARS "wrapper". The net result is one full packet, and one empty packet on the wire, alternating. The performance impact in LAN latency is negligible. However on higher latency between AWS and your premises it has a terrible performance impact.

MARS isn't strictly needed for most things. Some features that requires it are ORM (EF) proxies and lazy loading. If you need MARS, there are third party "accelerators" that workaround this madness.

"MARS Acceleration significantly improves the performance of connections that use the Multiple Active Result Sets (MARS) connection option."

https://documentation.nitrosphere.com/resources/release-note...

Re: AWS to bare metal two years later: Answering your questions about leaving AWS

#375

Post by geodel »

jmaker wrote: post #272
fulafel wrote: post #26

Earlier quoted context omitted.

The direct cost is the easy part. The more insidious part is that you're now cultivating a growing staff of technologists whose careers depend on doing things the AWS way, getting AWS certified to ensure they build your systems the AWS Well Architected Way instead of thinking themselves, and can upsell you on AWS lock-in solutions using AWS provided soundbites and sales arguments. ("Shall we make the app very resilie…

It’s a marketing trap. But also a job guarantee since everyone’s in the same trap. You got a couple cloud engineers or "DevOps" that lobby for AWS or any other hyperscaler, NaiveDate managers that write down some decision report littered with logical fallacies, and a few years in the sink cost is so high you can’t get off of it, and instead of doing productivity work you’re sitting in myriads of FinOps meetings, wher…

All great points. I have seen in company of smart people CIO/CTO would freely up admit "Look we know cloud may not be cheap or easier to manage but this is the direction we have taken since we are getting out of owning or managing hardware/datacenter"

So it is not like one can dazzle decision makers with any logic or hard data. They are just announcing the decision while calling it a robust discussion over pros and cons of on-prem vs cloud placement.

Re: AWS to bare metal two years later: Answering your questions about leaving AWS

#376

Post by kshacker »

pjdesno wrote: post #359

I'm involved in a fairly large academic cloud deployment, sited in a 15MW data center built and shared by a few large universities. There are huge advantages of scale to computer operations in a few areas: - facility: the capital and running cost of a purpose-built datacenter is far cheaper per rack than putting machines in existing office-class buildings, as long as it's a reasonable size - ours is ~1000 racks, but…

> the capital and running cost of a purpose-built datacenter is far cheaper per rack than putting machines in existing office-class buildings, as long as it's a reasonable size - ours is ~1000 racks, but you might get decent scale at a quarter of that.

Just want to confirm what I am reading. You are talking about ~1000 racks as the facility size, not what a typical university requires.

Re: AWS to bare metal two years later: Answering your questions about leaving AWS

#377

Post by Slothrop99 »

thelastgallon wrote: post #175

These are the features that AWS provides (1) Massive expansion of budget (100 - 1000x) to support empire building. Instead of one minimum-wage sysadmin with 2 high-availability, maxed-out servers for 20K - 40K (and 4-hour response time from Dell/HPE), you can have 100M multi-cloud Kubernetes + Lambda + a mix-and-match of various locked-in cloud services (DB, etc.). And you can have a large army of SRE/DevOps. You get…

> one minimum-wage sysadmin

The internet assures me there are loads of these underemployed Unix/networking experts just sitting around waiting to set up your infrastructure. But in my experience, these people are actually really difficult to hire, and not at all cheap. (Possibly the sharp ones have 'sold out' and gone the SRE route and are now one of those '3000' people.)

So I wonder if there's a certain amount of wishful thinking on both sides here, like "I wish a 'clueful' company would hire me to be their head sysadmin...", while companies who have tried to do this on the cheap usually just have terrible ops. ("Whoops, the backups haven't worked in 2 years...")

Re: AWS to bare metal two years later: Answering your questions about leaving AWS

#379

Post by infecto »

Hilift wrote: post #374
m-gasser wrote: post #158

Earlier quoted context omitted.

> Did you know there is a setting in MS SQL Server that impacts performance by an order of magnitude when sending/receiving data from the Cloud to your on-premises servers? It's the default in the ORM generated settings. Sounds interesting, which setting is that?

Multiple Active Result Sets (MARS). During large query responses or bulk loads, "full" packets cause an additional packet to be sent over the wire with about five bytes to hold the MARS "wrapper". The net result is one full packet, and one empty packet on the wire, alternating. The performance impact in LAN latency is negligible. However on higher latency between AWS and your premises it has a terrible performance im…

Is that not a client connection flag? MARS does not require a setting change on the server?

Re: AWS to bare metal two years later: Answering your questions about leaving AWS

#380

Post by _ea1k »

bilekas wrote: post #11

I'm so surprised there is so much pushback against this.. AWS is extremely expensive. The use cases for setting up your system or service entirely in AWS are more rare than people seem to realise. Maybe I'm just the old man screaming at cloud (no pun intended) but when did people forget how to run a baremetal server ? > We have 730+ days with 99.993% measured availability and we also escaped AWS region wide downtime…

> I'm so surprised there is so much pushback against this..

Same, this trend towards "AWS all the things" has really amazed me.

We've all mocked small companies copying big companies by trying to make their app super-duper scalable from the very start. After all, everyone things they are the next google, despite their 5 total users right now.

But this is really the opposite. AWS is phenomenal for the startup that would readily trade high opex for lower capex. Servers aren't the cheapest things in the world to buy and they depreciate. It makes total sense for startups to start this way.

But why are big companies, with an actual budget for staff, copying the behavior of their favorite startups?

Post reply on HN