Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We reduced our server costs by moving away from AWS

levelup.gitconnected.com

Post reply on HN
Print view View original

231–240 of 355 posts

Re: We reduced our server costs by moving away from AWS

#231

Post by taylodl »

MajimasEyepatch wrote: post #72
taylodl wrote: post #59

Earlier quoted context omitted.

Yes. If you use and manage AWS resources as you would your own on-prem resources then you're not going to have a good time. As soon as you think you need EC2 instances you need to re-think your architecture. You're probably not using AWS most effectively.

I think that statement about EC2 is a little too strong. You definitely shouldn't be managing individual EC2 instances in most cases, and you probably shouldn't be deploying directly to EC2. But if you're running EKS (Kubernetes) or ECS (non-k8s containers), then you're probably going to get more bang for your buck with EC2 nodes than Fargate nodes, especially if you have a large cluster.

That's a good distinction. Maybe instead of saying avoid EC2 what we should be saying is try to utilize AWS' managed services whenever possible.

Re: We reduced our server costs by moving away from AWS

#232

Post by Kon-Peki »

moduspol wrote: post #214
boltzmann-brain wrote: post #187

Earlier quoted context omitted.

Indeed, and in fact running own metal is an order of magnitude easier than puzzling around the Brazil nightmare that AWS is. Both people cost money. It's not like if you go with AWS things run themselves.

If that were true, then there'd be no value proposition to AWS. It is absolutely easier to use S3 than to create your own fast, highly available, infinitely scaling storage solution on your own metal. It requires more than zero knowledge / expertise to use S3, but far less than it would to implement and run yourself. If you can accept that, then we already agree in principle. It's just matter of where the line is dra…

You can buy on-prem managed storage solutions. Like PureStorage type things, where they call you up if something goes wrong, and are FedEx’ing you replacement parts before you even noticed that it had imperceptibly failed over to the standby power supply or whatever.

Re: We reduced our server costs by moving away from AWS

#233

Post by [deleted] »

joshstrange wrote: post #48

They don't mention at all what services they were using (other than slight mention of S3) which makes it very hard to respond to this. If you are running everything on EC2 then you are going to have a bad time (especially if you aren't using reserved instances). AWS (IMHO) shines with the various services they provide (S3, Lambda, CloudFront, API Gateway, SQS< SES, to name a few). AWS is a game of trying to reduce yo…

[deleted]

Re: We reduced our server costs by moving away from AWS

#234

Post by icedchai »

300bps wrote: post #150
shudza wrote: post #112

Earlier quoted context omitted.

So what should you use for let say, a backend API service? Don't tell me beanstalk/fargate/etc, because they're actually more expensive.

Lambda, SQS and SNS. That’s how just about everyone does backend API services with AWS. You’ll be shocked at how cheap, scalable and bullet-proof it can be.

The Lambda "developer experience" is mediocre at best. Performance is often highly variable. And when you reach a certain point, you'll get much better bang for the buck with EC2. Everything can be cheap when you have very little traffic.

Re: We reduced our server costs by moving away from AWS

#236

Post by dijit »

moduspol wrote: post #214
boltzmann-brain wrote: post #187

Earlier quoted context omitted.

Indeed, and in fact running own metal is an order of magnitude easier than puzzling around the Brazil nightmare that AWS is. Both people cost money. It's not like if you go with AWS things run themselves.

If that were true, then there'd be no value proposition to AWS. It is absolutely easier to use S3 than to create your own fast, highly available, infinitely scaling storage solution on your own metal. It requires more than zero knowledge / expertise to use S3, but far less than it would to implement and run yourself. If you can accept that, then we already agree in principle. It's just matter of where the line is dra…

There is a value prop for sure.

It’s hard to quantify how best you’ll be served but a lot of people are following the mantra of “nobody got fired for going AWS”.

It makes sense for some people, others are cargo-culting; yet more are fanning the flames of that cargo cult because their pay check depends on it.

Sysadmins are/were paid much less than cloud native devops people, and you need the same number of them unless you keep things very simple, which cloud providers do not incentivise. One need only look at AWS reference architectures.

Re: We reduced our server costs by moving away from AWS

#237

Post by loloquwowndueo »

citizenpaul wrote: post #203
tester756 wrote: post #201

Earlier quoted context omitted.

huge salaries + ten "ops engineers", lol. I know *data centers* that run on a few naive 20 yos admins/technicans + 1-2 "engineers" and all of them combined do receive salary of $5-10k/month in east eu

One time I was doing research on some really cheap data centers in basically 3rd world countries. Just out of curiosity to see how cheap it could really get. One of the companies had a picture of one of their "datacenters." It was something like 10 racks in a moldy unfinished basement with visible water on the floor of what I'm guessing was a residential building. Maybe they had mopped the floor for the picture? I th…

Share the company name / picture please :)

Re: We reduced our server costs by moving away from AWS

#238

Post by Corrado »

hamandcheese wrote: post #163
lbriner wrote: post #124

Earlier quoted context omitted.

Yes, exactly this. At what point do you have the time/money/confidence to invest goodness knows how much in a data centre with space to grow, to purchase an enormous amount of capital to have it all installed etc. the building alone could eat that first years saving easily. How many people are now needed to fault-find bad hardware/software/networks, to be on call for any problems? How many calls out to the Electricia…

> At what point do you have the time/money/confidence to invest goodness knows how much in a data centre with space to grow, to purchase an enormous amount of capital to have it all installed etc. the building alone could eat that first years saving easily. Co-locating has no capital investment other than hardware, and is pretty cheap. A 40U rack of compute charged as equivalent ec2 instances has a retail price easil…

I feel that a lot of posts like this might be under-representing the true costs of running your own hardware. I was only tangentially associated with a large-ish operation and I can tell you that there are loads of things that take a lot of time but are often over looked. Things like detecting and replacing bad hardware. HDs don't last forever and when they go bad it's not fun; especially if you have to source some specific model for your 7 year old server platform.

Understanding your licensing and warranties is another huge cost that people don't take into account. We used to spend hours and hours figuring out if we could replace systems and what it would cost us.

Finally, you have to dispose of all that hardware when it gets too old or out of warranty. If you've never had to do that you probably have no idea how hard it is to do it correctly and so that it satisfies your SOC2 auditor.

All of these things (plus more) really add up. It's not just purchasing the components and installing them in racks. The management is probably even more expensive than the hardware.

And all of these problems go away with a cloud provider.

Re: We reduced our server costs by moving away from AWS

#239

Post by intelVISA »

billythemaniam wrote: post #193
otabdeveloper4 wrote: post #184

Earlier quoted context omitted.

Hiring a person to do 'docker compose up' for you is orders of magnitude cheaper than whatever AWS-specific knowldege is needed to not have AWS crap its bed.

If their AWS spend is $1M/year, it is not as simple as "docker compose up" on bare metal.

Might need some systemd as well then to be fair

Re: We reduced our server costs by moving away from AWS

#240

Post by humanwhosits »

hintymad wrote: post #218
mabbo wrote: post #118

I'll always celebrate stories like this, but I also don't take some kind of anti-AWS lesson from it. This company saved $800k/year. Perfect time to go in-house with this solution. But when they were 1/10th this size, they'd only have saved $80k/year. Does that cover the cost of the engineering to build and maintain this system? Maybe not. And when they were 1/100th the size, it would have been laughable to go in-hous…

People don't consider productivity? Maybe things have gotten a lot better in the industry now. Otherwise, to rehash an older comment on HN: I'd like to remind everyone about Uber's experience: no EC2-like functionality until at least 2018, probably even now. Teams would negotiate with CTO for more machines. Uber's container-based solution didn't support persistent volumes for years. Uber's distributed database was ba…

and they had trouble moving workloads to the cloud because bringing up new capacity was a giant set of circular microservice dependencies
Post reply on HN