Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How an empty S3 bucket can make your AWS bill explode

medium.com

Post reply on HN
Print view View original

51–60 of 118 posts

Re: How an empty S3 bucket can make your AWS bill explode

#51

Post by jsheard »

belter wrote: post #49
jsheard wrote: post #15

Earlier quoted context omitted.

A bored or malicious actor can also just DDoS your CloudFront endpoint and run up an effectively unlimited bill on your end. That exposure is par for the course with AWS, they don't have a spending hard-stop and at this point they probably never will, you just have to hope that they have mercy and waive the bill if you get a surprise extra few zeroes on the end.

Shield Advanced claims cost protection, although as always, the devil is in the details... - https://aws.amazon.com/shield/features/ "AWS Shield Advanced comes with DDoS cost protection to safeguard against scaling charges resulting from DDoS-related usage spikes on protected EC2, ELB, CloudFront, Global Accelerator, and Route 53 resources. If any of these protected resources scale up in response to a DDoS attack, yo…

Shield Advanced is $3000 per month with a 1 year commitment, plus extra bandwidth fees on top of your existing bandwidth fees, I somehow doubt that many of the users who would be impacted the most by an unexpectedly large bill (individuals and smaller businesses) are proactively paying that much for DDoS insurance just in case.

Re: How an empty S3 bucket can make your AWS bill explode

#52

Post by belter »

yjftsjthsd-h wrote: post #46
belter wrote: post #37

Earlier quoted context omitted.

Guess what? It seems is the same with GCP and Azure. I am willing to be corrected if it's not...

Yes? AIUI all 3 are regularly accused of wildly overcharging on egress; what's your point?

My point is that we need to talk about all three.

Re: How an empty S3 bucket can make your AWS bill explode

#53

Post by belter »

jsiepkes wrote: post #27

Quite terrifying. Anyone know if competitors also bill requests which result in a "permission denied"? GCP, Azure, Backblaze B2, etc.?

It seems nobody replied to you, so let me try.

Yes for GCP and Azure, also for GPT...For Backblaze I am still investigating...

Two quick examples in different contexts:

https://learn.microsoft.com/en-us/answers/questions/1064419/...

https://community.openai.com/t/cost-for-failed-gpt-requests/...

Edit: It seems for Backblaze that would be a no, see below:

https://www.reddit.com/r/backblaze/comments/16o4bed/charge_b...

Re: How an empty S3 bucket can make your AWS bill explode

#55

Post by scottlamb »

Looking at AWS's price page for us-east-1 [1]

* PUT, COPY, POST, LIST requests are $0.005 per 1,000 requests

* GET, SELECT, and all other requests are $0.0004 per 1,000 requests

It sounds like these are getting charged at the PUT request pricing. And IIUC, those prices are set because successful write operations seek on multiple HDDs (physically moving the head) to commit the data to multiple machines, or S3 would be unable meet durability goals. Successful read operations are an order of magnitude cheaper because they likely move only one of several heads, sometimes none at all (because they're cacheable).

Authorization failures should never hit an HDD.

Other folks have mentioned that these auth failures are costing AWS something. Sure...but very very little. Far less than the cost of a successful GET, much less a successful PUT. They really shouldn't be charging at all for this, or if they do, it should be like $0.00001 per 1,000 requests (another order of magnitude cheaper than GETs). Not even that really; the actual cost for them to process this request is likely less than they're (over)charging for the bandwidth alone.

[1] https://aws.amazon.com/s3/pricing/

Re: How an empty S3 bucket can make your AWS bill explode

#56

Post by threatofrain »

Ekaros wrote: post #40

I wonder why does not Amazon be responsible and lock these accounts clearly attempting abuse? Logically either they are compromised or malicious. At least after certain number of failed request. And thus should be treated as such.

This case is especially problematic because this person was being hit from many distinct and legitimate accountants.

Re: How an empty S3 bucket can make your AWS bill explode

#57

Post by belter »

jsheard wrote: post #51
belter wrote: post #49

Earlier quoted context omitted.

Shield Advanced claims cost protection, although as always, the devil is in the details... - https://aws.amazon.com/shield/features/ "AWS Shield Advanced comes with DDoS cost protection to safeguard against scaling charges resulting from DDoS-related usage spikes on protected EC2, ELB, CloudFront, Global Accelerator, and Route 53 resources. If any of these protected resources scale up in response to a DDoS attack, yo…

Shield Advanced is $3000 per month with a 1 year commitment, plus extra bandwidth fees on top of your existing bandwidth fees, I somehow doubt that many of the users who would be impacted the most by an unexpectedly large bill (individuals and smaller businesses) are proactively paying that much for DDoS insurance just in case.

Agree on that. Was not claiming that Shield Advanced is the solution for this nutty request Rodeo...or maybe we should call it 404 festival...

Re: How an empty S3 bucket can make your AWS bill explode

#58

Post by utensil4778 »

silisili wrote: post #4
tazu wrote: post #2

I don't use S3, but it's kind of insane that unauthorized requests contribute to your bill... couldn't a bored/malicious actor with a botnet just spam common bucket names?

This is exactly where my mind went. I assume most people, myself included, didn't realize this. This is absolutely going to turn into the economic equivalent of a ddos.

I'm far too lazy to do the math, but I do wonder how many typical residential connections it would take to make a noticeable dent on a big company's bottom line with this method? I'd wager it's a surprisingly low number.

Re: How an empty S3 bucket can make your AWS bill explode

#59

Post by plorkyeran »

robertlagrant wrote: post #26
immibis wrote: post #22

Earlier quoted context omitted.

The right way is for unauthorized requests to not cost anything.

That makes no sense, though. Some network and compute are running for this to happen. Someone needs to pay the bill. It's not obvious it should be the cloud service, if that's not the agreement.

They charge ten times as much for an unauthorized PUT as they do for an unauthorized GET, and I feel fairly confident in saying that it does not cost them ten times as much.

Re: How an empty S3 bucket can make your AWS bill explode

#60

Post by jpambrun »

Today I was hit by a surprise 500$ bill by using byte range request within zip files without an upper bound, but not consuming the whole stream. I tested for about 45 min at about 1gbps. My ISP meter says I downloaded 300gb which lines up with physics, but AWS says it was 6000gb.

My first thought is that can be abused to multiply the damage if you want to engage in some cost based denial of service.

About every other week AWS gets out of their way to make us remember that they are the worst cloud around.

For reference https://github.com/ZJONSSON/node-unzipper/issues/308

Post reply on HN