Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Designing a scalable API on AWS spot instances

blog.adapty.io

Post reply on HN
Print view View original

31–40 of 63 posts

Re: Designing a scalable API on AWS spot instances

#31

Post by kpotehin »

jakozaur wrote: post #30

For service type workloads (e.g API service with 99.99% uptime SLA) we keep comparing on-demand vs. spot. In reality, you would like to compare Reserved Instances as you can get 60% discount. So in us-east-1: - spot costs: 35-43% of on-demand - RI 1 year standard: 60% of on-demand - RI 3 year convertible: 46% of on-demand So if you have some base load that you can commit to running for 3 years, the price gets often a…

Good point, thanks! We're looking at saving plans, probably will use them too. But when you're doing prototypes it's pretty much impossible to commit to 1 year of usage, let alone 3:)

Re: Designing a scalable API on AWS spot instances

#33

Post by duggan »

dijit wrote: post #25
saddlerustle wrote: post #10

Earlier quoted context omitted.

This. There is no alternative for high performance object storage outside of AWS S3 and Google Cloud Storage. Even Azure's offering is wonky. Lots of providers claim to offer object storage, but try hitting them from couple thousand cores and they all tend to immediately fall over.

AWS S3 buckets have a limit of 5gb/s access to a single bucket. Unless that limitation has changed in the last couple of years; I can easily make a system beat that, if that's the requirement.. and ultimately it does come down to understanding requirements. :\ I think generally people forget that cloud is just computers too, it's really not anything special, and amazon/google/microsoft are solving the general case (a…

It's 25Gbps now[1]. I have a rule of thumb: if I haven't refreshed something I think I know about AWS for more than half a year, it's probably out of date.

[1]: https://aws.amazon.com/premiumsupport/knowledge-center/s3-ma...

Re: Designing a scalable API on AWS spot instances

#35

Post by barake »

csunbird wrote: post #15
jorams wrote: post #4

Earlier quoted context omitted.

I can give you the reason we moved from Digital Ocean to Google Cloud: Spaces (object storage) was ridiculously slow and unreliable. There were several incidents per week where Spaces had bad connectivity or simply seemed to have crashed entirely, and that's not something we can deal with for production traffic. We now run a cheap CDN in front of it to cut down on the ridiculous bandwidth costs, but Google Cloud Stor…

is it that bad? we are thinking about bootstrapping a project, which will use spaces for file storage and we will be using them frequently.

Provisioning of new Spaces was recently disabled in some regions while they add capacity. Vaguely concerning IMO and has caused me to seriously reconsider using DO Spaces for a new product I'm building.

https://www.digitalocean.com/docs/release-notes/upcoming/spa...

Re: Designing a scalable API on AWS spot instances

#36

Post by zerubeus »

kpotehin wrote: post #31
jakozaur wrote: post #30

For service type workloads (e.g API service with 99.99% uptime SLA) we keep comparing on-demand vs. spot. In reality, you would like to compare Reserved Instances as you can get 60% discount. So in us-east-1: - spot costs: 35-43% of on-demand - RI 1 year standard: 60% of on-demand - RI 3 year convertible: 46% of on-demand So if you have some base load that you can commit to running for 3 years, the price gets often a…

Good point, thanks! We're looking at saving plans, probably will use them too. But when you're doing prototypes it's pretty much impossible to commit to 1 year of usage, let alone 3:)

You can still reserve classic instance for one year and when you don't use it anymore you can sell it in the market

Re: Designing a scalable API on AWS spot instances

#37

Post by csunbird »

barake wrote: post #35
csunbird wrote: post #15

Earlier quoted context omitted.

is it that bad? we are thinking about bootstrapping a project, which will use spaces for file storage and we will be using them frequently.

Provisioning of new Spaces was recently disabled in some regions while they add capacity. Vaguely concerning IMO and has caused me to seriously reconsider using DO Spaces for a new product I'm building. https://www.digitalocean.com/docs/release-notes/upcoming/spa...

I do know that they have limited the creation and looking at the docs for limitation, it seems like they have a 750 requests per second limit per space, which might not scale well. My idea was to create multiple spaces to distribute a load, e.g. sharding per customer, but if the availability is not that good, I might have to re-consider our choice.

The main reason we wanted to work with DO is that the egress traffic cost is reasonable, unlike AWS or Google Cloud where it is questionably high.

Re: Designing a scalable API on AWS spot instances

#38

Post by jorams »

csunbird wrote: post #15
jorams wrote: post #4

Earlier quoted context omitted.

I can give you the reason we moved from Digital Ocean to Google Cloud: Spaces (object storage) was ridiculously slow and unreliable. There were several incidents per week where Spaces had bad connectivity or simply seemed to have crashed entirely, and that's not something we can deal with for production traffic. We now run a cheap CDN in front of it to cut down on the ridiculous bandwidth costs, but Google Cloud Stor…

is it that bad? we are thinking about bootstrapping a project, which will use spaces for file storage and we will be using them frequently.

It was that bad 2 years ago, in the AMS3 region. I don't know how it is these days, but I expect after 2 years it has matured at least a little bit. They also now include a CDN, which might improve things a bit if it reduces the load on the buckets behind it.

Honestly it soured my opinion on DO a bit. I'd previously had a droplet there for several years, which never seemed to experience any issues at all.

Re: Designing a scalable API on AWS spot instances

#39

Post by jrockway »

speedgoose wrote: post #16
saddlerustle wrote: post #10

Earlier quoted context omitted.

This. There is no alternative for high performance object storage outside of AWS S3 and Google Cloud Storage. Even Azure's offering is wonky. Lots of providers claim to offer object storage, but try hitting them from couple thousand cores and they all tend to immediately fall over.

That's not a very common use case though. Most companies don't need to DDOS their cloud provider.

25Gbps is 250 cable Internet users downloading the release of your new software. What you call a DDOS is actually a woefully underwhelming instantaneous transfer rate.

Re: Designing a scalable API on AWS spot instances

#40

Post by jrockway »

jorams wrote: post #4
makkesk8 wrote: post #3

"Our backend system is built on AWS. Today I’m going to tell you how we had cut costs" This is a recurring topic here on HN and it boggles me and makes me wonder if people know that there are other platforms than aws, azure and google cloud out there that are very capable and much much cheaper. Unless any of the big 3 has a feature or certification you need I don't see any reason to use them at all due to the insane…

I can give you the reason we moved from Digital Ocean to Google Cloud: Spaces (object storage) was ridiculously slow and unreliable. There were several incidents per week where Spaces had bad connectivity or simply seemed to have crashed entirely, and that's not something we can deal with for production traffic. We now run a cheap CDN in front of it to cut down on the ridiculous bandwidth costs, but Google Cloud Stor…

I run my own Docker registry for my personal projects backed by DO Spaces. It was so slow that I couldn't actually believe it, and had to instrument the registry code extensively to account for all the time spent in the storage API, to see what the actual problem was. Turns out Spaces is really, really slow.
Post reply on HN