Live data from Hacker News

How an empty S3 bucket can make your AWS bill explode

medium.com

91–100 of 118 posts

Re: How an empty S3 bucket can make your AWS bill explode

#91

Quite terrifying. Anyone know if competitors also bill requests which result in a "permission denied"? GCP, Azure, Backblaze B2, etc.?

This note at https://cloud.google.com/storage/pricing suggests otherwise:

> Generally, you are not charged for operations that return 307, 4xx, or 5xx responses. The exception is 404 responses returned by buckets with Website Configuration enabled and the NotFoundPage property set to a public object in that bucket.

Re: How an empty S3 bucket can make your AWS bill explode

#93
For me, this is not only a cost thing, but a security issue as well. Considering the IAM auth or any kind of auth method, the goal is never to be "impossible to break-in", but "the spending of break-in is higher than the potential value it protected". Yet if the price of break-in is paid by the victim, this will be a whole different story. Someone can spam the S3 API with brute force IAM credentials can try out everything until find the right credentials, and the failure attempts are paid by the victim, which is insane.

Re: How an empty S3 bucket can make your AWS bill explode

#94
post #23

Ok I'm actually going to delete my web site and cloudfront distribution over this. It gets no traffic (at the moment). The bucket is the same name as the domain so I can imagine this could be exploited quietly very easily as PUTs are so expensive. Literally a $5 VPS could cost someone $1k.

A $5 VPS should have a $10 spend limit or something, at which point it should shut down. In a reasonable world.

Re: How an empty S3 bucket can make your AWS bill explode

#95
post #2

I don't use S3, but it's kind of insane that unauthorized requests contribute to your bill... couldn't a bored/malicious actor with a botnet just spam common bucket names?

That's exactly it. I just spent the morning cleaning up my buckets. I had two old buckets left over from when I was using Elasticbeanstalk 5 years ago, and they had predictable names with my account ID in them. And as we heard in February, account IDs are easily leaked and not considered private. Besides that AWS themselves recommend you to use your domain name as bucket name when using S3 static site hosting. Thankf…

You cannot shut down an S3 bucket with a lambda tho, the mentioned issue above is only solvable by deleting the bucket.

Re: How an empty S3 bucket can make your AWS bill explode

#96

Earlier quoted context omitted.

What's crazier is that turning on Requester Pays does not actually mean requester pays when the request is a 403 [1][2]. Essentially every S3 bucket, public or private, whose name is discovered can be DDoSed creating an insane bill... That's a platform level security issue for AWS to resolve. We do not need another Bucketgate. [1]: https://twitter.com/Lauramaywendel/status/178507498639629151... [2]: https://docs.aws.…

For me it looks like AWS has mainly different scenarios to handle. First, unauthenticated users are a big security pain for AWS and S3 because S3 is used for everything. But S3 was not designed for "secure" public access. As a result billing attacks are possible (e.g. [1]) and will not be fixed (by design). Second, authenticated AWS users are kind of trustworthy and these are never assumed to do any DDoS stuff or sim…

private in the sense that the connection will never make it to the S3 bucket, is that correct?

private in the sense that it is locked down to allowed source IP's or IAM-- then the connection still gets there and AWS incurs a charge for the Access Denied response, etc

Re: How an empty S3 bucket can make your AWS bill explode

#97
post #63

Quite terrifying. Anyone know if competitors also bill requests which result in a "permission denied"? GCP, Azure, Backblaze B2, etc.?

90% sure GCP charges for rate limited API gateway. i.e. doesn't matter if it is above rate limit. Never got 100% confirmation either way though - their documentation on pricing is too vague

Yea they do, it says always charge when listing buckets.

Any way to mitigate this on AWS by restricting access to it via VPC only?

Re: How an empty S3 bucket can make your AWS bill explode

#99
Many are speculating that this will not be fixed by AWS (by design) however now that this has been discovered AWS will "need" to repair this flaw or they will start incurring customer flight to more secure or cheaper services.

The question is more about how long AWS are going to take to fix this issue and how many DDoS bills will they forgive.

Re: How an empty S3 bucket can make your AWS bill explode

#100
post #84

Unfortunately you can't always keep your bucket name a secret, if you use presigned URLs to let your users directly download objects...

It’s a good idea to use a reverse proxy like Nginx to handle the transfer through an internal redirect, which will hide the bucket name and authentication credentials.

Introducing complexity, failure point, scalability issue just to work around broken billing. You are probably right but I feel it’s wrong architecturally. I always liked S3 precisely because I could offload file serving to them.
Post reply on HN