Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How an empty S3 bucket can make your AWS bill explode

medium.com

Post reply on HN
Print view View original

71–80 of 118 posts

Re: How an empty S3 bucket can make your AWS bill explode

#71

Post by jpambrun »

tedmiston wrote: post #67
jpambrun wrote: post #60

Today I was hit by a surprise 500$ bill by using byte range request within zip files without an upper bound, but not consuming the whole stream. I tested for about 45 min at about 1gbps. My ISP meter says I downloaded 300gb which lines up with physics, but AWS says it was 6000gb. My first thought is that can be abused to multiply the damage if you want to engage in some cost based denial of service. About every other…

To preface this comment, I'm not super familiar with range GETs or the library you're using. Are you saying that the library was performing range GET requests but instead of sending a byte range, it was only sending a starting byte number? > Today I was hit by a surprise 500$ bill by using byte range request within zip files without an upper bound, but not consuming the whole stream. I tested for about 45 min at abou…

Yes. It made many request with Range star- (i.e. without end).

500$ is not making a huge dent in the monthly bill for this. I just got a few laughs at my expense from colleagues..

I don't know how AWS meters, but, if your assumption is right, I can abuse this and generate a huge bill to anyone just by starting streams and not consuming then. I could generate 10s of gbps of "apparent" billable traffic with a puny VPS.

Re: How an empty S3 bucket can make your AWS bill explode

#72

Post by nicolas_17 »

belter wrote: post #39
tazu wrote: post #2

I don't use S3, but it's kind of insane that unauthorized requests contribute to your bill... couldn't a bored/malicious actor with a botnet just spam common bucket names?

First of all, I agree this is all bananas:-) but it seems is the same with GCP and Azure. For Backblaze B2 it seems its unclear in the documentation. Willing to be corrected on this....

I don't know if Backblaze charges for failed requests, but it lets you set spending limits, so the worst case is that the attacker drains the daily quota that you set and later requests fail (DoS), not that you get a surprise $1000 bill.

Re: How an empty S3 bucket can make your AWS bill explode

#73

Post by nicolas_17 »

jpambrun wrote: post #71
tedmiston wrote: post #67

Earlier quoted context omitted.

To preface this comment, I'm not super familiar with range GETs or the library you're using. Are you saying that the library was performing range GET requests but instead of sending a byte range, it was only sending a starting byte number? > Today I was hit by a surprise 500$ bill by using byte range request within zip files without an upper bound, but not consuming the whole stream. I tested for about 45 min at abou…

Yes. It made many request with Range star- (i.e. without end). 500$ is not making a huge dent in the monthly bill for this. I just got a few laughs at my expense from colleagues.. I don't know how AWS meters, but, if your assumption is right, I can abuse this and generate a huge bill to anyone just by starting streams and not consuming then. I could generate 10s of gbps of "apparent" billable traffic with a puny VPS.

You could also generate 10s of gbps of actual billable traffic with a puny VPS by forging TCP ACKs and getting AWS to send you data faster than you can actually receive it, but that's closer to an actual DoS attack and more likely to get you in trouble...

Re: How an empty S3 bucket can make your AWS bill explode

#74

Post by thenickdude »

jftuga wrote: post #66

Since you need to create a unique bucket name any way, I like to include my AWS Account ID in my bucket names. Be advised that bucket names need to be 63 characters or less. Another reason I do this is that CloudFormation supports the use of: ${AWS::AccountId}

This does not save you because AWS Account IDs are not private, e.g. if you are using S3 pre-signed URLs anywhere, those URLs include your account ID as part of the X-AMZ-Credential URL field:

https://medium.com/@TalBeerySec/a-short-note-on-aws-key-id-f...

Re: How an empty S3 bucket can make your AWS bill explode

#77

Post by INTPenis »

tazu wrote: post #2

I don't use S3, but it's kind of insane that unauthorized requests contribute to your bill... couldn't a bored/malicious actor with a botnet just spam common bucket names?

That's exactly it. I just spent the morning cleaning up my buckets.

I had two old buckets left over from when I was using Elasticbeanstalk 5 years ago, and they had predictable names with my account ID in them. And as we heard in February, account IDs are easily leaked and not considered private.

Besides that AWS themselves recommend you to use your domain name as bucket name when using S3 static site hosting. Thankfully I've been using Cloudfront infront of randomized bucket names for a few years already.

So now I'm forced to increase the random string I use in all my bucket names from 8 to 16 characters. Every bucket is now a password for a malicious actor who wants to attack your finances.

Obviously all accounts should have budget alerts, but there is no budget ceiling feature.

Technically AWS does offer you all the tools to create a budget ceiling feature using budget alerts and lambdas that will shutdown the affected service. I just wish someone would do it because I don't have the time.

Re: How an empty S3 bucket can make your AWS bill explode

#78

Post by robertlagrant »

plorkyeran wrote: post #59
robertlagrant wrote: post #26

Earlier quoted context omitted.

That makes no sense, though. Some network and compute are running for this to happen. Someone needs to pay the bill. It's not obvious it should be the cloud service, if that's not the agreement.

They charge ten times as much for an unauthorized PUT as they do for an unauthorized GET, and I feel fairly confident in saying that it does not cost them ten times as much.

That seems like a different issue to "who should pay at all".

Re: How an empty S3 bucket can make your AWS bill explode

#79

Post by robertlagrant »

kolanos wrote: post #31
robertlagrant wrote: post #14

Earlier quoted context omitted.

How is that possible? AWS can't bill anyone on the planet.

This is an unauthorized S3 API request. If there is a valid access id and secret attached, bill them for the request.

I doubt that anyone's agreed to pay that. Also that there's often a valid access ID and secret. Also that the cost of scanning through all the client IDs across all secrets is worth the money.

Re: How an empty S3 bucket can make your AWS bill explode

#80

Post by minhuw »

For a long time, bucket name is believed to be non-sensitive so they are sometimes placed in public just for convenience (like in CI scripts of public Git repositories). It's easy to collect examples by searching "s3://" on sourcegraph. I believe it's much easier to find a bucket charging a company/open source project/person than we expect.
Post reply on HN