Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The AWS S3 Denial of Wallet Amplification Attack

blog.limbus-medtec.com

Post reply on HN
Print view View original

31–40 of 97 posts

Re: The AWS S3 Denial of Wallet Amplification Attack

#31

Post by nijave »

__roland__ wrote: post #27
fabian2k wrote: post #9

Earlier quoted context omitted.

That sounds egregious enough that I have trouble believing this can be correct. My understanding is that AWS bills for egress for every service, parts of the file that aren't transferred are not part of this so can't be billed. There could certainly be S3-specific charges that affect cases like this, no idea. But if AWS bills the full egress traffic costs for a range request I'd consider that essentially fraud.

Sorry, I think that part of our write-up is misleading (I was involved in analyzing the issue described here). To our best understanding, what happens is the following: - A client sends range requests and cancels them quickly. - The full range request data will be billed (NOT the whole file), so I think this should read that the entire requested range gets billed, even if it never gets transferred (the explanation we…

This is a problem with lots of services. Blocking large quantities of legitimate looking requests is a hard problem. Request cancellation is also tricky and not supported well in a lot of frameworks/programming languages.

Re: The AWS S3 Denial of Wallet Amplification Attack

#34

Post by anonymousDan »

jsheard wrote: post #29
Havoc wrote: post #24

It's almost like the combination of public accessible + charged per use + big cloud refusing to allow hardcaps on spend is a terrible idea...

Azure is probably the most egregious example of this, AWS and GCP can at least claim they have architectural barriers to implementing a hard spending cap, but Azure already has one and arbitrarily only allows certain subscription types to use it. If you have a student account then you get a certain amount of credit each month and if you over-spend it then most services are automatically suspended until the next month…

AWS Educate has the same ability to impose a hard cap I believe...

Re: The AWS S3 Denial of Wallet Amplification Attack

#35

Post by nijave »

akira2501 wrote: post #22

We use CloudFront and we deny public users the ability to access S3 directly. You can even use Signed URLs with CloudFront if you like. I'm not sure I'd evere feel comfortable letting the public at large hit my S3 endpoints.

Direct S3 is pretty common for file distribution where latency is less of a concern.

e.x. build an installer and distribute it, generate a report and generate a signed url

Re: The AWS S3 Denial of Wallet Amplification Attack

#36

Post by paulddraper »

fabian2k wrote: post #16
paulddraper wrote: post #13

Earlier quoted context omitted.

tl;dr AWS user believes that testing on a 1Gbps connection for 45 min can't be more than $10 of egress. Gets a $500 bill instead. Note: This user specified a lower range but not an upper range on the request (and closed the connection prematurely). Essentially read() with an offset, for a ZIP tool. See also: https://news.ycombinator.com/item?id=40205213

There is probably a small area where it's difficult to measure, so I would not expect billing to be exact to the byte here. But billing for the requested range if not the entire range was actually transferred is just not correct and not acceptable.

Certainly if you are charging for internet egress.

Like, charging for requests or internal data processing, sure okay.

But this is a charge specifically for the data transferred from AWS to the internet. So if you're not transferring data to the internet....

Re: The AWS S3 Denial of Wallet Amplification Attack

#37

Post by nijave »

INTPenis wrote: post #26
akira2501 wrote: post #22

We use CloudFront and we deny public users the ability to access S3 directly. You can even use Signed URLs with CloudFront if you like. I'm not sure I'd evere feel comfortable letting the public at large hit my S3 endpoints.

As it should be, but recently on HN it was posted that AWS will charge you for any unauthorized PUT request to your S3 buckets. Meaning even 4xx errors will rack up a charge. So your S3 bucket names must be hidden passphrases now that stand between an attacker and your budget.

In all fairness, systems administrators have always had to pay for unauthorized requests and systems to mitigate the risk

The new thing is hyperscalers have so much capacity you can get flooded by these long before the service degrades or goes offline

Re: The AWS S3 Denial of Wallet Amplification Attack

#38

Post by mikepurvis »

nielsole wrote: post #28
nicklecompte wrote: post #21

Earlier quoted context omitted.

This must be a regression bug in AWS's internal system. At a past job (2020) we used S3 to store a large amount of genomic data, and a web application read range requests to visualize tiny segments of the genetic sequence in relevant genes - like 5kb out of 50GB. If AWS had billed the cost of an entire genome/exome every time we did that, we would have noticed. I monitored costs pretty closely, S3 was never a problem…

S3 egress costs are free if the traffic stays within AWS. Sounds like your clients were EC2 instances so this wouldn't apply to you, would it?

If it was a web application as stated in the GP, then it would indeed be egress as the request would be coming from a browser.

Re: The AWS S3 Denial of Wallet Amplification Attack

#40

Post by fabian2k »

paulddraper wrote: post #36
fabian2k wrote: post #16

Earlier quoted context omitted.

There is probably a small area where it's difficult to measure, so I would not expect billing to be exact to the byte here. But billing for the requested range if not the entire range was actually transferred is just not correct and not acceptable.

Certainly if you are charging for internet egress . Like, charging for requests or internal data processing, sure okay. But this is a charge specifically for the data transferred from AWS to the internet . So if you're not transferring data to the internet....

The part where I think there is some flexibility is about the difference between "bytes attempted to transfer" and "bytes actually transferred". I think it is pretty fair to bill for the former, as long as you abort requests in a reasonable way. So I don't expect it to be billed exactly by the transferred byte, but I do expect it to not go above that higher than whatever the chunk size for transferring is.
Post reply on HN