Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A $1k AWS mistake

geocod.io

Post reply on HN
Print view View original

221–230 of 281 posts

Re: A $1k AWS mistake

#221

Post by moduspol »

DenisM wrote: post #197
moduspol wrote: post #102

Earlier quoted context omitted.

AWS would much rather let you accidentally overspend and then forgive it when you complain than see stories about critical infrastructure getting shut off or failing in unexpected ways due to a miscommunication in billing.

They could have given us a choice though. Sign in blood that you want to be shut off in case of over spend.

As long as "shut off" potentially includes irrecoverable data loss, I guess, as it otherwise couldn't conclusively work. Along with a bunch of warnings to prevent someone accidentally (or maliciously) enabling it on an important account.

Still sounds kind of ugly.

Re: A $1k AWS mistake

#223

Post by richwater »

deanCommie wrote: post #179
cowsandmilk wrote: post #155

Earlier quoted context omitted.

S3 Gateway endpoints break cross-region S3 operations. Changing defaults will break customers.

Changing defaults doesn't have to mean changing existing configurations. It can be the new default for newly created VPCs after a certain date, or for newly created accounts after a certain date. And if there are any interoperability concerns, you offer an ability to opt-out with that (instead of opting in). There is precedent for all of this at AWS.

> Changing defaults doesn't have to mean changing existing configurations. It can be the new default for newly created VPCs after a certain date, or for newly created accounts after a certain date.

This is breaking existing IAAC configurations because they rely on the default. You will never see the change you're describing except in security-related scenarios

> There is precedent for all of this at AWS.

Any non-security IAAC default changes you can point to?

Re: A $1k AWS mistake

#224

Post by abigail95 »

jabroni_salad wrote: post #158
abigail95 wrote: post #142

Earlier quoted context omitted.

What is a bootstrapped dev?

It means you are self funded and do not have a pile of other people's money to burn.

I would guess that's most AWS accounts. I have my 5 personal accounts all on one debit card.

I learned AWS the same way most "bootstrapped" people do, with the free tier. Maybe it's more of a minefield than it was a decade ago.

Re: A $1k AWS mistake

#225

Post by tecleandor »

wiether wrote: post #52
tecleandor wrote: post #18

Earlier quoted context omitted.

I think it's been calculated that data transfer is the biggest margin product in all AWS catalog by a huge difference. A 2021 calculation done by Cloudflare [0] estimated almost 8000% price markup in EU and US regions. And I can see how, in very big accounts, small mistakes on your data source when you're doing data crunching, or wrong routing, can put thousands and thousands of dollars on your bill in less than an h…

> can put thousands and thousands of dollars on your bill in less than an hour By default a NGW is limited to 5Gbps https://docs.aws.amazon.com/vpc/latest/userguide/nat-gateway... A GB transferred through a NGW is billed 0.05 USD So, at continuous max transfer speed, it would take almost 9 hours to reach $1000 Assuming a setup in multi-AZ with three AZs, it's still 3 hours if you have messed so much that you can mana…

That's a NAT gateway, but if you're pulling data for analysis from S3 buckets you don't have those limitations.

Let's say they decide to recalculate or test a algorithm: they do parallel data loading from the bucket(s), and they're pulling from the wrong endpoint or region, and off they go.

And maybe they're sending data back, so they double the transfer price. RDS Egress. EC2 Egress. Better keep good track of your cross region data!

Re: A $1k AWS mistake

#226

Post by timando »

maccard wrote: post #71
rwmj wrote: post #45

Earlier quoted context omitted.

Wait, what measures you implemented ? How about AWS implements a hard cap, like everyone has been asking for forever?

What does a hard cap look like for EBS volumes? Or S3? RDS? Do you just delete when the limit is hit?

2 caps: 1 for things that are charged for existing (e.g. S3 storage, RDS, EBS, EC2 instances) and 1 for things that are charged when you use them (e.g. bandwidth, lambda, S3 requests). Fail to create new things (e.g. S3 uploads) when the first cap is met.

Re: A $1k AWS mistake

#227

Post by mgaunard »

hanikesn wrote: post #145
Hikikomori wrote: post #40

Earlier quoted context omitted.

Only works in public subnets, which isn't what you want most of the time.

Yep and have to pay for public IPs, which can become quite costly on it's own. Can't wait for v6 to be here.

An IP costs $50, or $0.50 per month if leasing.

Re: A $1k AWS mistake

#229

Post by icedchai »

unethical_ban wrote: post #185
Spivak wrote: post #126

Earlier quoted context omitted.

The reason to not include the endpoint by default is because VPCs should be secure by default. Everything is denied and unless you explicitly configure access to the Internet, it's unreachable. An attacker who manages to compromise a system in that VPC now has a means of data exfiltration in an otherwise air gapped set up. It's annoying because this is by far the more uncommon case for a VPC, but I think it's the rig…

I don't get your argument. If an ec2 needs access to an s3 resource, doesn't it need that role? Or otherwise, couldn't there be some global s3 URL filter that automagically routes same-region traffic appropriately if it is permitted? My point is that, architecturally, is there ever in the history of AWS an example where a customer wants to pay for the transit of same-region traffic when a check box exists to say "do…

The EC2 needs credentials, but not necessarily a role. If someone is able to compromise an EC2 instance that has unrestricted S3 connectivity (no endpoint policies), they could use their own credentials to exfiltrate data to a bucket not associated with the account.

Re: A $1k AWS mistake

#230

Post by sofixa »

callmeal wrote: post #117
sofixa wrote: post #55

Earlier quoted context omitted.

It's not that it's technically impossible. The very simple problem is that there is no way of providing hard spend caps without giving you the opportunity to bring down your whole production environment when the cap is met. No cloud provides wants to give their customers that much rope to hang themselves with. You just know too many customers will do it wrong or will forget to update the cap or will not coordinate in…

>The very simple problem is that there is no way of providing hard spend caps without giving you the opportunity to bring down your whole production environment when the cap is met. And why is that a problem? And how different is that from "forgetting" to pay your bill and having your production environment brought down?

> And how different is that from "forgetting" to pay your bill and having your production environment brought down?

AWS will remind you for months before they actually stop it.

Post reply on HN