Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How can I quickly trim my AWS bill?

news.ycombinator.com

Post reply on HN
Print view View original

81–90 of 134 posts

Re: Ask HN: How can I quickly trim my AWS bill?

#81

Post by paulcole »

jayzalowitz wrote: post #76

Here's my deck on this @quinnypig is a great resource elsewhere in this thread. https://docs.google.com/presentation/d/1sNtFugQp_Mcq62gf4F1n... Last year I cut 75 million in spend, so you could say I have a track record there. Are you sure you are using the right type for what you need to generate? Can you have your model generator self kill (stop) the instance when it finishes the model? 100% If it doesnt need JIT g…

Can you give a little context to the $75 million in savings? What was the original amount you were spending? I didn’t see this on your deck.

Re: Ask HN: How can I quickly trim my AWS bill?

#83

Post by mlthoughts2018 »

Don’t use GPUs at inference (serving) time unless you prove that you need to.

The only consistent case when I’ve found it’s needed (across a variety of NLP & computer vision services that have latency requirements under 50 milliseconds) is for certain very deep RNNs, especially for long input sequence lengths and large vocabulary embeddings.

I’ve never found any need for it with deep, huge CNNs for image processing.

Also consider a queue system if utilization is a problem switching from GPU. Create batch endpoints that accept small batches, like 8-64 instances, and put a queue system in front to mediate collating and uncollating batch calls from the stream of all incoming requests (this is good for GPU services too).

Re: Ask HN: How can I quickly trim my AWS bill?

#85

Post by jwr »

papaf wrote: post #77
jwr wrote: post #66

Earlier quoted context omitted.

Also, consider physical servers from providers like Hetzner. These can be several times cheaper than EC2.

I use Hetzner for quite a lot for personal projects and can recommend them for reliability and predictable costs. I've done reasonably high CPU tasks like compiling Android images on the larger Cloud instances. However, this morning I was playing around with Scaleway bare metal [1] and General Purpose instances [2] -- I am thinking of making a switch for high CPU tasks. [1] https://www.scaleway.com/en/bare-metal-serv…

Interesting! These look very good indeed. I will have to try them.

The main point is that physical servers are much cheaper than VMs and provide significantly better performance as well (see my benchmarking and comparison: https://jan.rychter.com/enblog/cloud-server-cpu-performance-...).

Re: Ask HN: How can I quickly trim my AWS bill?

#86

Post by pixiemaster »

I‘m a CTO of a compute intensive AI SaaS company, so I can relate.

One advice: speak to your AWS rep immediately. Get credits to redesign your system and keep you running. you can expect up to 7 digits in credits (for real!) and support for a year for free, they really want to help you in avoiding this.

Re: Ask HN: How can I quickly trim my AWS bill?

#87

Post by atsaloli »

beardface wrote: post #30
QuinnyPig wrote: post #2

Howdy. I have loud and angry thoughts about this; https://www.lastweekinaws.com/blog/ has a bunch of pieces, some of which may be more relevant than others. The slightly-more-serious corporate side of the house is at https://www.duckbillgroup.com/blog/ , if you can stomach a slight decline in platypus.

Came here to recommend you! Your newsletter always provides both enlightenment and a giggle.

I came here to recommend QuinnyPig's services as well. He's a pro at reducing AWS costs.

Re: Ask HN: How can I quickly trim my AWS bill?

#89

Post by FridgeSeal »

lmeyerov wrote: post #37
lmeyerov wrote: post #19

Sounds familiar =\ - get devs on GPU laptops - for always-on, where doable, switch to an 8a - 6p policy, and reserved. Call aws for a discount. - use g4dn x spot. Check per workload tho, it assumes single vs double. - consider if can switch to fully on-demand if not already , and hybrid via GCP's attachable GPUs - make $ more visible to devs. Often individuals just don't get it, too easy to be sloppy. More probably d…

ALSO: For all the discussion of on-prem, for ML in particular, consider running training on a dedicated local hw box and run only inference on the cloud (which can be CPU)

I’ve been mulling this idea over in my head recently of investing a $2-3k in building a machine to do exactly that (and use it as a normal dev day to day machine when it’s not training), because it appears the economics of it are surprisingly great.

Have you (or anything else here) had experience doing this? Did it end up being a worthwhile approach? (Even for a while)

Re: Ask HN: How can I quickly trim my AWS bill?

#90

Post by pluc »

jwr wrote: post #66
ParanoidShroom wrote: post #62

Earlier quoted context omitted.

I was in the same boat and this is good advice! I stopped using gpu's, "Vectorized inference isn’t bad at all!". This soo much, I was blinded with gpu speed, using tensorflow builds with avx optimization is actually pretty fast. My discovery: + Stop expensive GPU's for inference and switch to avx optimized tensorflow builds. + Cleaned up the inference pipeline and reduced complexity. + Buying compute instance for a y…

Also, consider physical servers from providers like Hetzner. These can be several times cheaper than EC2.

I worked on an unrelated market study - look at Upcloud and Raptr as well.
Post reply on HN