Lots of people in here mentioning Reserved Instances, so it's worth mentioning Reserved AI (https://www.reserved.ai/).
We're customers and have been very happy with them, it was super quick to get set up and saves us a big chunk.
Ask HN: How can I quickly trim my AWS bill?
61–70 of 134 posts
Re: Ask HN: How can I quickly trim my AWS bill?
#62Disclosure: I work on Google Cloud (but my advice isn’t to come to us). Sorry to hear that. I’m sure it’s super stressful, and I hope you pull through. If you can, I’d suggest giving a little more information about your costs / workload to get more help. But, in case you only see yet another guess, mine is below. If your growth has accelerated yielding massive cost, I assume that means you’re doing inference to serve…
I was in the same boat and this is good advice!
I stopped using gpu's, "Vectorized inference isn’t bad at all!". This soo much, I was blinded with gpu speed, using tensorflow builds with avx optimization is actually pretty fast.
My discovery:
+ Stop expensive GPU's for inference and switch to avx optimized tensorflow builds.
+ Cleaned up the inference pipeline and reduced complexity.
+ Buying compute instance for a year or more provides a discount.
- I never got pruning to work without a significant loss increase.
- Tried spot instances with gpu's that are cheaper. Random kills and spinning up new instances took too long loading my code. The discount is a lot, but I couldn't reliable get it up. Users where getting more timeouts. I bailed and just used cpu inference. The gpu was being underutilized, using cpu only increased the inference to around 2-3 seconds. With the price trade off it was a more simpel,cheaper and easier solution.
Re: Ask HN: How can I quickly trim my AWS bill?
#63Make sure to tag every instance/resource/disk in AWS, with their purpose, team, etc...
Then you can go into the AWS costs explorer and see the costs breakdown per tag.
Usually there will be a few resources standing out. 80% of the costs is 20% of resources. Find out what they are and cut.
Re: Ask HN: How can I quickly trim my AWS bill?
#64Segment's blog posts on cost optimisation have plenty of detail and tips on this topic:
https://segment.com/blog/the-million-dollar-eng-problem/ https://segment.com/blog/spotting-a-million-dollars-in-your-... https://segment.com/blog/the-10m-engineering-problem/
Similarly this Honeycomb writeup is also excellent: https://www.honeycomb.io/blog/treading-in-haunted-graveyards...
By the sounds of it, you need to take drastic action. It sounds like you will not be able to just optimise your AWS spend to get more runway, though you should definitely do some bill optimisation. You will need to optimise your product itself and maybe even getting rid of unprofitable customers.
If you are not sure exactly who or what is driving the AWS cost, take a look at Honeycomb to get the ability to dive deep into what is eating up resources.
Re: Ask HN: How can I quickly trim my AWS bill?
#65If you're running GPU heavy stuff all the time then you're probably better off just buying some GPUs outright and doing that part on-site.
Especially if you can keep the own gear busy 24/7. i.e. run those 24/7 and any excess GPU use above that fall back onto cloud for that.
Re: Ask HN: How can I quickly trim my AWS bill?
#66Disclosure: I work on Google Cloud (but my advice isn’t to come to us). Sorry to hear that. I’m sure it’s super stressful, and I hope you pull through. If you can, I’d suggest giving a little more information about your costs / workload to get more help. But, in case you only see yet another guess, mine is below. If your growth has accelerated yielding massive cost, I assume that means you’re doing inference to serve…
I was in the same boat and this is good advice! I stopped using gpu's, "Vectorized inference isn’t bad at all!". This soo much, I was blinded with gpu speed, using tensorflow builds with avx optimization is actually pretty fast. My discovery: + Stop expensive GPU's for inference and switch to avx optimized tensorflow builds. + Cleaned up the inference pipeline and reduced complexity. + Buying compute instance for a y…
Also, consider physical servers from providers like Hetzner. These can be several times cheaper than EC2.
Re: Ask HN: How can I quickly trim my AWS bill?
#67I was in same situation.
We bough 2 Dell servers via their financing program. Each server is about 19-25K. We paid AWS $60K per month before that. We pay $600 for co-location.
So my advice is try to get hardware via financing of provider Dell had a good program I think.
Re: Ask HN: How can I quickly trim my AWS bill?
#68Real hardware with colocation
Re: Ask HN: How can I quickly trim my AWS bill?
#69I was in same situation. We bough 2 Dell servers via their financing program. Each server is about 19-25K. We paid AWS $60K per month before that. We pay $600 for co-location. So my advice is try to get hardware via financing of provider Dell had a good program I think.
This! We did the exact same, though our payback period was 2 months of AWS costs. Try and put the base load on your own servers, use the cloud to scale up and down when needed.
Cloud servers are a “luxury” that most don’t realise and just take for granted. Having said that, there are obvious overheads with handling your own servers, but when your costs are several salaries it’s probably worth considering.
Re: Ask HN: How can I quickly trim my AWS bill?
#70[DISCLAIMER] I work at AWS, not speaking for my employer. We really need some more details on your infrastructure, but I assume it's EC2 instance cost that skyrocketed? A couple of pointers: - Experiment with different GPU instance types. - Try Inferentia [1], a dedicated ML chip. Most popular ML frameworks are supported by the Neuron compiler. Assuming you manage your instances in an auto scaling group (ASG): - Enab…
It could also be worth it to have a look at SageMaker? IIRC it's cheaper.