Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Designing a scalable API on AWS spot instances

blog.adapty.io

Post reply on HN
Print view View original

11–20 of 63 posts

Re: Designing a scalable API on AWS spot instances

#11

Post by pluies »

ocdnix wrote: post #2

Turns out this is about EC2 spot instances for ECS. How would it compare to ECS Fargate spot these days? I'm also missing a discussion about designing for interruption, either by not keeping state, or by being able to shed state quickly, to be picked up by other instances. Also, if you set up EC2 spot with a launch template or ASG with very differently-sized instance types (to reduce risk of running out), is there a…

(Not the OP, but running a fairly similar setup, albeit for EKS nodes rather than ECS nodes :) )

Fargate Spot is about a third of the price of Fargate (at least in eu-west-1 now according to: https://aws.amazon.com/fargate/pricing/ ); so the savings are roughly identical.

Re risk of running out, our current strategy is to use different-but-closely-similar instance groups; so for example we have an autoscaling group running a mix of:

- m5.large - m5dn.large - m5n.large - m5ad.large - m5d.large

Which are the same price on Spot instances, but I'd wager it'd be pretty rare to have all these families reclaimed at once.

(We also use some on-demand only ASGs with lower priority in the cluster-autoscaler to ensure that if it _does_ happen, then we'll have a fallback)

Re: Designing a scalable API on AWS spot instances

#13

Post by CSDude »

makkesk8 wrote: post #3

"Our backend system is built on AWS. Today I’m going to tell you how we had cut costs" This is a recurring topic here on HN and it boggles me and makes me wonder if people know that there are other platforms than aws, azure and google cloud out there that are very capable and much much cheaper. Unless any of the big 3 has a feature or certification you need I don't see any reason to use them at all due to the insane…

Always this poor argument pops up. 1st AWS and big 3 is much more reliable. 2nd AWS is not only a VPS provider. Changing vendors is not the answer to all cost related problems.

Re: Designing a scalable API on AWS spot instances

#14

Post by MrPowers »

Great article. Cutting ec2 costs is important, especially for companies with heavy data engineering / data science workflows. Spark service providers make it easy to spin up huge clusters (100+ nodes) to perform ad hoc analyses. The costs can quickly spiral out of control, even if you're getting 3x cost savings on the spot market.

Some tangential thoughts:

* Is there an AWS API that returns the cheapest availability zone in a region for a given instance type? Or is the GUI that's screenshot in the blog the only way to see?

* I have seen the 90%+ cost savings for certain instance types

* Sometimes you lose a spot instance, look at the pricing history graph to confirm the price spike, and don't see any spike that was above your bid price... it can be frustrating

Re: Designing a scalable API on AWS spot instances

#15

Post by csunbird »

jorams wrote: post #4
makkesk8 wrote: post #3

"Our backend system is built on AWS. Today I’m going to tell you how we had cut costs" This is a recurring topic here on HN and it boggles me and makes me wonder if people know that there are other platforms than aws, azure and google cloud out there that are very capable and much much cheaper. Unless any of the big 3 has a feature or certification you need I don't see any reason to use them at all due to the insane…

I can give you the reason we moved from Digital Ocean to Google Cloud: Spaces (object storage) was ridiculously slow and unreliable. There were several incidents per week where Spaces had bad connectivity or simply seemed to have crashed entirely, and that's not something we can deal with for production traffic. We now run a cheap CDN in front of it to cut down on the ridiculous bandwidth costs, but Google Cloud Stor…

is it that bad? we are thinking about bootstrapping a project, which will use spaces for file storage and we will be using them frequently.

Re: Designing a scalable API on AWS spot instances

#16

Post by speedgoose »

saddlerustle wrote: post #10
jorams wrote: post #4

Earlier quoted context omitted.

I can give you the reason we moved from Digital Ocean to Google Cloud: Spaces (object storage) was ridiculously slow and unreliable. There were several incidents per week where Spaces had bad connectivity or simply seemed to have crashed entirely, and that's not something we can deal with for production traffic. We now run a cheap CDN in front of it to cut down on the ridiculous bandwidth costs, but Google Cloud Stor…

This. There is no alternative for high performance object storage outside of AWS S3 and Google Cloud Storage. Even Azure's offering is wonky. Lots of providers claim to offer object storage, but try hitting them from couple thousand cores and they all tend to immediately fall over.

That's not a very common use case though. Most companies don't need to DDOS their cloud provider.

Re: Designing a scalable API on AWS spot instances

#17

Post by secondcoming »

makkesk8 wrote: post #3

"Our backend system is built on AWS. Today I’m going to tell you how we had cut costs" This is a recurring topic here on HN and it boggles me and makes me wonder if people know that there are other platforms than aws, azure and google cloud out there that are very capable and much much cheaper. Unless any of the big 3 has a feature or certification you need I don't see any reason to use them at all due to the insane…

It's probably because when devs jump into the cloud they can be distracted by Shiny New Toys and cost is only considered once the first bill comes in.

Re: Designing a scalable API on AWS spot instances

#18

Post by marcinzm »

dijit wrote: post #8
PetahNZ wrote: post #5

Earlier quoted context omitted.

Tell me who/what I should migrate my Elastic Beanstalk/Aurora Serverless/S3 stack to to save significant costs?

The solution to saving costs is to go for cheaper cloud providers or run k8s on VPS's/colocated physical hardware. I'm not sure where the idea came from that this is hard to do. To answer your questions in order: 1) People warned aggressively about lock-in of cloud providers with proprietary extensions and you must have chosen not to listen; so, I have little sympathy. I'm not saying it's black and white; but I hope…

Now you have five more things you need to research, learn, keep track of, debug, etc. Plus, in my experience, anyone who says kubernetes is easy to run (cloud or native) has never had to run kubernetes in production.

Re: Designing a scalable API on AWS spot instances

#19

Post by marcinzm »

makkesk8 wrote: post #3

"Our backend system is built on AWS. Today I’m going to tell you how we had cut costs" This is a recurring topic here on HN and it boggles me and makes me wonder if people know that there are other platforms than aws, azure and google cloud out there that are very capable and much much cheaper. Unless any of the big 3 has a feature or certification you need I don't see any reason to use them at all due to the insane…

>So why do you or your company who uses any of the big 3 use them if you had to cut cost at some point?

Because it takes X hours of effort to cut costs and using another provider would have required Y hours of effort where Y >> X. Y may be greater due to reliability issues, missing features that you need to build yourself, training costs of new employees, etc.

edit: Also X is paid once you're succeeding, Y has to be paid before you're succeeding which makes Y even more expensive in opportunity cost.

Re: Designing a scalable API on AWS spot instances

#20

Post by dijit »

marcinzm wrote: post #18
dijit wrote: post #8

Earlier quoted context omitted.

The solution to saving costs is to go for cheaper cloud providers or run k8s on VPS's/colocated physical hardware. I'm not sure where the idea came from that this is hard to do. To answer your questions in order: 1) People warned aggressively about lock-in of cloud providers with proprietary extensions and you must have chosen not to listen; so, I have little sympathy. I'm not saying it's black and white; but I hope…

Now you have five more things you need to research, learn, keep track of, debug, etc. Plus, in my experience, anyone who says kubernetes is easy to run (cloud or native) has never had to run kubernetes in production.

Sure, then pay?

I don't understand the argument: "I want to outsource understanding but I want to save costs";

You can think of things as a spider-graph of three points:

Quality -- Low-Cost -- Knowledge-Required.

No solution can score high points on all three reasonably.

Anyway, things are a bit skewed because I'm an infrastructure type, and people in my profession really do think of systems administration tasks as being "very easy" and if done right soak up nearly no time at all, but developers don't like hearing that because sysadmins are "old world".

I don't really care if you're paying someone elses sysadmins or not, the fact remains that you're going to be spending something in that area, and if you balk at the cost of cloud then maybe taking ownership of what they do can help optimise costs.

Obviously they put a premium on their own time in these areas.

Post reply on HN