Live data from Hacker News

How an empty S3 bucket can make your AWS bill explode

medium.com

41–50 of 118 posts

Re: How an empty S3 bucket can make your AWS bill explode

#41
post #40

I wonder why does not Amazon be responsible and lock these accounts clearly attempting abuse? Logically either they are compromised or malicious. At least after certain number of failed request. And thus should be treated as such.

Anybody can do this.

Re: How an empty S3 bucket can make your AWS bill explode

#42
post #22

Earlier quoted context omitted.

The right way is for unauthorized requests to not cost anything.

That makes no sense, though. Some network and compute are running for this to happen. Someone needs to pay the bill. It's not obvious it should be the cloud service, if that's not the agreement.

This is part of doing business for the cloud provider.

The cost of running compute to deal with unauthorized requests for an arbitrary extant S3 bucket, is the same as the cost of running compute to deal with unauthorized requests for nonexistent S3 buckets.

If I generated a billion requests to an S3 bucket that did not exist, Amazon would have to decline that traffic in the same way. Since the recipient did not exist, there would be no one to bill.

I as the attacker should not be able to add on a sticky note saying "btw you can charge X user for this malicious traffic" and have Amazon actually honor that.

EDIT: Here's an analogy. Say a business in a city has a front lawn that they must pay to maintain. Sometimes people walk on the grass as a shortcut to the business. Some subset of those people also enter the business as a customer. Would it be fair for the business to charge those people more to account for the extra landscaping bill they cause?

But the answer doesn't matter, because that's not what happened. What happened here is more like, someone left a note on the front door of the business saying "My name is X, and I walked on your grass last night", and so that day the business charges more to any customer whose credit card says their name is X.

Re: How an empty S3 bucket can make your AWS bill explode

#43
post #30
post #2

I don't use S3, but it's kind of insane that unauthorized requests contribute to your bill... couldn't a bored/malicious actor with a botnet just spam common bucket names?

This kinda happened with Netlify recently ( https://www.reddit.com/r/webdev/comments/1b14bty/netlify_jus... , https://news.ycombinator.com/item?id=39520776 ). They got a bill over $100,000 via spam requests. Unauthorized requests are a weird thing. They still do cost money for a web service to handle and if you (the customer) have screwed things up that's kinda on you. I've run SaaS where a customer forgot to send th…

> Unauthorized requests are a weird thing. They still do cost money for a web service to handle and if you (the customer) have screwed things up that's kinda on you.

Who pays when the unauthorized requests contains truly random data, i.e. they point at an org and bucket that do not exist? Amazon does. Part of their cost of doing business is building the infrastructure that routes authorized requests correctly, and if a request is for invalid data, it's dropped. That should be par the course.

Large numbers of unauthorized requests can be effectively considered a DDOS on the service itself. That the attacker can add a note onto their attack that says "btw bill X customer for all this" and have that honored is ridiculous.

Re: How an empty S3 bucket can make your AWS bill explode

#44
post #39
post #2

I don't use S3, but it's kind of insane that unauthorized requests contribute to your bill... couldn't a bored/malicious actor with a botnet just spam common bucket names?

First of all, I agree this is all bananas:-) but it seems is the same with GCP and Azure. For Backblaze B2 it seems its unclear in the documentation. Willing to be corrected on this....

I'm less familiar with AWS. But on Azure, you can have private storage accounts which aren't internet routable so they wouldn't even be able to hit the endpoints to rack up charges. It's a bit ridiculous that you'd have to go that far to avoid being charged for unauthorized access, but I'd be surprised if AWS and GCP didn't have similar capabilities.

Re: How an empty S3 bucket can make your AWS bill explode

#45
post #23

Ok I'm actually going to delete my web site and cloudfront distribution over this. It gets no traffic (at the moment). The bucket is the same name as the domain so I can imagine this could be exploited quietly very easily as PUTs are so expensive. Literally a $5 VPS could cost someone $1k.

i doubt it. i would imagine any single ip address would get rate-limited/throttle before doing much. the situation the article is describing is much more like a botnet — there were hundreds or thousands (or more?) of unique instances making requests.

> i would imagine any single ip address would get rate-limited/throttle before doing much.

Does AWS have default per-IP rate limits?

Re: How an empty S3 bucket can make your AWS bill explode

#46
post #37

Crazy, I wonder what AWS's margin is on these unauthorized requests? Questionable incentives.

Guess what? It seems is the same with GCP and Azure. I am willing to be corrected if it's not...

Yes? AIUI all 3 are regularly accused of wildly overcharging on egress; what's your point?

Re: How an empty S3 bucket can make your AWS bill explode

#48
However, such logs can be enabled using AWS CloudTrail or S3 Server Access Logging. After enabling CloudTrail logs--

Aaaah! CloudTrail data events are priced in such a way that each S3 request gets logged--at a cost of 20x that of the S3 request itself. I really hope OP got THAT part of the bill waived as well.

Re: How an empty S3 bucket can make your AWS bill explode

#49
post #15
post #2

I don't use S3, but it's kind of insane that unauthorized requests contribute to your bill... couldn't a bored/malicious actor with a botnet just spam common bucket names?

A bored or malicious actor can also just DDoS your CloudFront endpoint and run up an effectively unlimited bill on your end. That exposure is par for the course with AWS, they don't have a spending hard-stop and at this point they probably never will, you just have to hope that they have mercy and waive the bill if you get a surprise extra few zeroes on the end.

Shield Advanced claims cost protection, although as always, the devil is in the details... - https://aws.amazon.com/shield/features/

"AWS Shield Advanced comes with DDoS cost protection to safeguard against scaling charges resulting from DDoS-related usage spikes on protected EC2, ELB, CloudFront, Global Accelerator, and Route 53 resources. If any of these protected resources scale up in response to a DDoS attack, you can request Shield Advanced service credits through your regular AWS Support channel."

Re: How an empty S3 bucket can make your AWS bill explode

#50
post #2

I don't use S3, but it's kind of insane that unauthorized requests contribute to your bill... couldn't a bored/malicious actor with a botnet just spam common bucket names?

That's why, even though during the day I work on AWS for people who pay me to do so, in the evening I work on my startups hosted on Hetzner where I know in advance what my bill will be not only this month but also this year.

Same here, it might be slightly less convenient but boy how well I sleep at night.
Post reply on HN