Live data from Hacker News

Ask HN: How can I quickly trim my AWS bill?

news.ycombinator.com

1–10 of 134 posts

Ask HN: How can I quickly trim my AWS bill?

#1
Hi HN,

I work with a company that has a few GPU-intensive ML models. Over the past few weeks, growth has accelerated, and with that costs have skyrocketed. AWS cost is about 80% of revenue, and the company is now almost out of runway.

There is likely a lot of low hanging cost-saving-fruit to be reaped, just not enough people to do it. We would love any pointers to anyone who specializes in the area of cost optimization. Blogs, individuals, consultants, or magicians are all welcome.

Thank you!

Re: Ask HN: How can I quickly trim my AWS bill?

#2
Howdy.

I have loud and angry thoughts about this; https://www.lastweekinaws.com/blog/ has a bunch of pieces, some of which may be more relevant than others. The slightly-more-serious corporate side of the house is at https://www.duckbillgroup.com/blog/, if you can stomach a slight decline in platypus.

Re: Ask HN: How can I quickly trim my AWS bill?

#6
Without any idea of what your infrastructure looks like, I can't give you anything actionable, but that might be enough advice in and of itself: go after the low hanging fruit first. What are you spending on? Look at the top two or three services by spend and dig a little deeper.

Are you spending on bandwidth? See if there's compression you can enable. Ec2? Can your reduce instance sizes or autoscale down instances you're not using overnight? Elasticache or elastic search? Tune your usage, store smaller keys or expire things out.

Re: Ask HN: How can I quickly trim my AWS bill?

#8
How about you just purchase some motherboards and GPUs and start running them in your office (assuming you're not bandwidth limited or looking for millisecond response times).. I'm always tempted to do this when we have fairly constant workload. Wasn't GPU instance pricing quite insane on AWS compared to actual GPU costs?

Re: Ask HN: How can I quickly trim my AWS bill?

#9
AWS/clouds aren't always the best solution for a problem. Often they're the worst (just like any other tool).

You don't provide a lot of detail but I imagine at this point you need to get "creative" and move at least some aspect of your operation out of AWS. Some variation of:

- Buy some hardware and host it at home/office/etc.

- Buy some hardware and put it in a colocation facility.

- Buy a lot of hardware and put it in a few places.

Etc.

Cash and accounting is another problem. Hardware manufacturers offer financing (leasing). Third party finance companies offer lines of credit, special leasing, etc. Even paying cash outright can (in certain cases) be beneficial from a tax standpoint. If you're in the US there's even the best of both worlds: a Section 179 deduction on a lease!

https://www.section179.org/section_179_leases/

You don't even need to get dirty. Last I checked it was pretty easy to get financing from Dell, pay next to nothing to get started, and have hardware shipped directly to a co-location facility. Remote hands rack and configure it for you. You get a notification with a system to log into just like an AWS instance. All in at a fraction of the cost. The dreaded (actually very rare) hardware failure? That's what the warranty is for. Dell will dispatch people to the facility and replace XYZ as needed. You never need to physically touch anything.

A little more complicated than creating an AWS account with a credit card number? Of course. More management? Slightly. But at the end of the day it's a fraction of the total cost and probably even advantageous from a taxation standpoint.

AWS and public clouds really shine in some use cases and absolutely suck at others (as in suck the cash right out of your pockets).

Re: Ask HN: How can I quickly trim my AWS bill?

#10
[DISCLAIMER] I work at AWS, not speaking for my employer.

We really need some more details on your infrastructure, but I assume it's EC2 instance cost that skyrocketed?

A couple of pointers:

- Experiment with different GPU instance types.

- Try Inferentia [1], a dedicated ML chip. Most popular ML frameworks are supported by the Neuron compiler.

Assuming you manage your instances in an auto scaling group (ASG):

- Enable a target tracking scaling policy to reactively scale your fleet. The best scaling metric depends on your inference workload.

- If your workload is predictable (e.g. high traffic during the daytime, low traffic during nighttime), enable predictive scaling. [3]

[1] https://aws.amazon.com/machine-learning/inferentia/

[2] https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-sca...

[3] https://docs.aws.amazon.com/autoscaling/plans/userguide/how-...

Post reply on HN