Live data from Hacker News

A $1k AWS mistake

geocod.io

151–160 of 281 posts

Re: A $1k AWS mistake

#151

Earlier quoted context omitted.

That sounds like an architecture choice? One that would cause less revenue on the AWS side, with a conflicting incentive there.

to be fair, im not sure its a conscious choice, since its not really easy to couple lets say data transfer bytes directly to billing data in real time, and im sure that would also use up a lot of resources. But of course, the incentive to optimize this is not there.

I mean, generally real time isn't needed. Even hourly updates could save a massive amount of headache. 24 hours or more is becoming excessive.

Re: A $1k AWS mistake

#152
post #149

Earlier quoted context omitted.

Yes uploading into AWS is free/cheap. You pay per GB of data downloaded, which is not cheap. You can see why, from a sales perspective: AWS' customers generally charge their customers for data they download - so they are extracting a % off that. And moreover, it makes migrating away from AWS quite expensive in a lot of circumstances.

> And moreover, it makes migrating away from AWS quite expensive in a lot of circumstances. Please get some training...and stop spreading disinformation. And to think on this thread only my posts are getting downvoted.... "Free data transfer out to internet when moving out of AWS" - https://aws.amazon.com/blogs/aws/free-data-transfer-out-to-i...

I don't appreciate your disinformation accusation nor your tone.

People are trying to tell you something with the downvotes. They're right.

Re: A $1k AWS mistake

#153

Earlier quoted context omitted.

Yes, delete things in reverse order of their creation time until the cap is satisfied (the cap should be a rate, not a total)

I would put $100 that within 6 months of that, we'll get a post on here saying that their startup is gone under because AWS deleted their account because they didn't pay their bill and didn't realise their data would be deleted. > (the cap should be a rate, not a total) this is _way_ more complicated than there being a single cap.

> I would put $100 that within 6 months of that, we'll get a post on here saying that their startup is gone under because AWS deleted their account because they didn't pay their bill and didn't realise their data would be deleted.

The cap can be opt-in.

Re: A $1k AWS mistake

#154
post #132

Earlier quoted context omitted.

It works just fine. No reason you couldn't adjust your billing cap on the fly. I work in a medium size org that's part of a large one, and we have to funnel any significant resource requests (e.g. for more EKS nodes) through our SRE teams anyway to approve. Actual spikey traffic that you can't plan for or react to is something I've never heard of, and believe is a marketing myth. If you find yourself actually trying…

I think I am having some misunderstanding about exactly how this cost control works. Suppose that a company in the transportation industry needs 100 CPUs worth of resources most of the day and 10,000 CPUs worth of resources during morning/evening rush hours. How would your reserved cost proposal work? Would it require having a cost cap sufficient for 10,000 CPUs for the entire day? If not, how?

10,000 cores is an insane amount of compute (even 100 cores should already be able to easily deal with millions of events/requests per second), and I have a hard time believing a 100x diurnal difference in needs exists at that level, but yeah, actually I was suggesting that they should have their cap high enough to cover 10,000 cores for the remainder of the billing cycle. If they need that 10,000 for 4 hours a day, that's still only a factor of 6 of extra quota, and the quota itself 1. doesn't cost them anything and 2. is currently infinity.

I also expect that in reality, if you regularly try to provision 10,000 cores of capacity at once, you'll likely run into provisioning failures. Trying to cost optimize your business at that level at the risk of not being able to handle your daily needs is insane, and if you needed to take that kind of risk to cut your compute costs by 6x, you should instead go on-prem with full provisioning.

Having your servers idle 85% of the day does not matter if it's cheaper and less risky than doing burst provisioning. The only one benefiting from you trying to play utilization optimization tricks is Amazon, who will happily charge you more than those idle servers would've cost and sell the unused time to someone else.

Re: A $1k AWS mistake

#155

This happens so often that the S3 VPC endpoint should be setup by default when your VPC is created. AWS engineers on here - make this happen. Also, consider using fck-nat ( https://fck-nat.dev/v1.3.0/ ) instead of NAT gateways unless you have a compelling reason to do otherwise, because you will save on per-Gb traffic charges. (Or, just run your own Debian nano instance that does the masquerading for you, which every…

S3 Gateway endpoints break cross-region S3 operations. Changing defaults will break customers.

Re: A $1k AWS mistake

#156
Imagine a world were Amazon was forced to provide a publicly available report were they disclose how many clients have made this error -and similar ones- and how much money they have made from it. I know nothing like this will ever exist but hey, is free to dream.

Re: A $1k AWS mistake

#157
post #37

These sort of things show up about once a day between the three big cloud subreddit. Often with larger amounts And it’s always the same - clouds refuse to provide anything more than alerts (that are delayed) and your only option is prayer and begging for mercy. Followed by people claiming with absolute certainty that it’s literally technically impossible to provide hard capped accounts to tinkerers despite there bein…

AWS just yesterday launched flat rate pricing for their CDN (including a flat rate allowance for bandwidth and S3 storage), including a guaranteed $0 tier.

https://news.ycombinator.com/item?id=45975411

I agree that it’s likely very technically difficult to find the right balance between capping costs and not breaking things, but this shows that it’s definitely possible, and hopefully this signals that AWS is interested in doing this in other services too.

Re: A $1k AWS mistake

#158

As a bootstrapped dev, reading stories like these gives me so much anxiety. I just can’t bring myself to use AWS even despite its advantages.

What is a bootstrapped dev?

It means you are self funded and do not have a pile of other people's money to burn.

Re: A $1k AWS mistake

#159
post #55
post #37

These sort of things show up about once a day between the three big cloud subreddit. Often with larger amounts And it’s always the same - clouds refuse to provide anything more than alerts (that are delayed) and your only option is prayer and begging for mercy. Followed by people claiming with absolute certainty that it’s literally technically impossible to provide hard capped accounts to tinkerers despite there bein…

It's not that it's technically impossible. The very simple problem is that there is no way of providing hard spend caps without giving you the opportunity to bring down your whole production environment when the cap is met. No cloud provides wants to give their customers that much rope to hang themselves with. You just know too many customers will do it wrong or will forget to update the cap or will not coordinate in…

Orrr AWS could just buffer it for you. Algo.

1) you hit the cap 2) aws sends alert but your stuff still runs at no cost to you for 24h 3) if no response. Aws shuts it down forcefully. 4) aws eats the “cost” because lets face it. It basically cost them 1000th of what they bill you for. 5) you get this buffer 3 times a year. After that. They still do the 24h forced shutdown but you get billed. Everybody wins.

Re: A $1k AWS mistake

#160
It used to be that you could whine to your account rep and they'd waive sudden accidental charges like this. Which we did regularly due to all the sharp edges. These days I gather it's a bit harder.
Post reply on HN