Many are speculating that this will not be fixed by AWS (by design) however now that this has been discovered AWS will "need" to repair this flaw or they will start incurring customer flight to more secure or cheaper services. The question is more about how long AWS are going to take to fix this issue and how many DDoS bills will they forgive.
How an empty S3 bucket can make your AWS bill explode
101–110 of 118 posts
Re: How an empty S3 bucket can make your AWS bill explode
#102Earlier quoted context omitted.
That makes no sense, though. Some network and compute are running for this to happen. Someone needs to pay the bill. It's not obvious it should be the cloud service, if that's not the agreement.
AWS rakes in multi billion dollars in profit every year. They can definitely eat the costs of unauthorized requests to s3 buckets.
At this point I believe that convoluted pricing is their business model. I got hit by their “tiny font price” twice, paying extraordinary money for trivial things. And I barely used AWS.
Re: How an empty S3 bucket can make your AWS bill explode
#103Looking at AWS's price page for us-east-1 [1] * PUT, COPY, POST, LIST requests are $0.005 per 1,000 requests * GET, SELECT, and all other requests are $0.0004 per 1,000 requests It sounds like these are getting charged at the PUT request pricing. And IIUC, those prices are set because successful write operations seek on multiple HDDs (physically moving the head) to commit the data to multiple machines, or S3 would be…
> Other folks have mentioned that these auth failures are costing AWS something.
This should be entirely irrelevant, as those same failures were happening even before the bucket existed, they were just hitting a different failure path.
Re: How an empty S3 bucket can make your AWS bill explode
#104Looking at AWS's price page for us-east-1 [1] * PUT, COPY, POST, LIST requests are $0.005 per 1,000 requests * GET, SELECT, and all other requests are $0.0004 per 1,000 requests It sounds like these are getting charged at the PUT request pricing. And IIUC, those prices are set because successful write operations seek on multiple HDDs (physically moving the head) to commit the data to multiple machines, or S3 would be…
What's crazier is that turning on Requester Pays does not actually mean requester pays when the request is a 403 [1][2]. Essentially every S3 bucket, public or private, whose name is discovered can be DDoSed creating an insane bill... That's a platform level security issue for AWS to resolve. We do not need another Bucketgate. [1]: https://twitter.com/Lauramaywendel/status/178507498639629151... [2]: https://docs.aws.…
What is bucketgate? Your comment comes up when searching...
Re: How an empty S3 bucket can make your AWS bill explode
#105Ok I'm actually going to delete my web site and cloudfront distribution over this. It gets no traffic (at the moment). The bucket is the same name as the domain so I can imagine this could be exploited quietly very easily as PUTs are so expensive. Literally a $5 VPS could cost someone $1k.
A $5 VPS should have a $10 spend limit or something, at which point it should shut down. In a reasonable world.
I think they meant the other way around, someone pays $5 for a VPS to spam unauthorized requests to your S3 bucket to get Amazon to bill you $1000.
Re: How an empty S3 bucket can make your AWS bill explode
#106Earlier quoted context omitted.
A $5 VPS should have a $10 spend limit or something, at which point it should shut down. In a reasonable world.
I think they meant the other way around, someone pays $5 for a VPS to spam unauthorized requests to your S3 bucket to get Amazon to bill you $1000.
Thanks. Damn, I've misunderstood some things recently so I'll take some rest and slow down I think.
Re: How an empty S3 bucket can make your AWS bill explode
#107Just waiting for a botnet group to abuse this, buy puts on the expected shortfall in earnings and profit.
Why would there be a shortfall in earnings? Failed (due to being unauthorized) HTTP requests cost AWS ~nothing.
Re: How an empty S3 bucket can make your AWS bill explode
#108Earlier quoted context omitted.
To preface this comment, I'm not super familiar with range GETs or the library you're using. Are you saying that the library was performing range GET requests but instead of sending a byte range, it was only sending a starting byte number? > Today I was hit by a surprise 500$ bill by using byte range request within zip files without an upper bound, but not consuming the whole stream. I tested for about 45 min at abou…
Yes. It made many request with Range star- (i.e. without end). 500$ is not making a huge dent in the monthly bill for this. I just got a few laughs at my expense from colleagues.. I don't know how AWS meters, but, if your assumption is right, I can abuse this and generate a huge bill to anyone just by starting streams and not consuming then. I could generate 10s of gbps of "apparent" billable traffic with a puny VPS.
> I don't know how AWS meters, but, if your assumption is right, I can abuse this and generate a huge bill to anyone just by starting streams and not consuming then.
I suppose that would be easy enough to test.
I was curious how video streaming services handle the case where a stream is started or partially consumed. Mux meters it in terms of [seconds of] video segments delivered:
> Cost is per minute of video delivered. To calculate video delivered, we measure the number of seconds of video delivered to a video player. Note that if a segment of video is delivered, it is charged, even if the viewer doesn’t actually watch the video.
Re: How an empty S3 bucket can make your AWS bill explode
#109I don't use S3, but it's kind of insane that unauthorized requests contribute to your bill... couldn't a bored/malicious actor with a botnet just spam common bucket names?
This kinda happened with Netlify recently ( https://www.reddit.com/r/webdev/comments/1b14bty/netlify_jus... , https://news.ycombinator.com/item?id=39520776 ). They got a bill over $100,000 via spam requests. Unauthorized requests are a weird thing. They still do cost money for a web service to handle and if you (the customer) have screwed things up that's kinda on you. I've run SaaS where a customer forgot to send th…
> They still do cost money for a web service to handle and if you (the customer) have screwed things up that's kinda on you.
They cost a lot less than a multiple disk write (PUT), a disk read (GET).
They cost the same to AWS whether the target bucket belongs to me, Amazon themselves, or does not exist.
And in this case, the customer definitely didn't "screw things up".
Charging for unauthorized requests is just weird and customer-hostile. Like, how am I supposed to prevent that?
I get that a service provider might want to "punish" a customer who misconfigures their systems and DoSes themselves with e.g. no backoff on 403 error, but I find it hard to believe that that's the common case -- and how is typo in authentication supposed to be so different from typo in bucket name?
Re: How an empty S3 bucket can make your AWS bill explode
#110Earlier quoted context omitted.
First of all, I agree this is all bananas:-) but it seems is the same with GCP and Azure. For Backblaze B2 it seems its unclear in the documentation. Willing to be corrected on this....
Not in GCP if https://cloud.google.com/storage/pricing is to be believed: > Generally, you are not charged for operations that return 307, 4xx, or 5xx responses. The exception is 404 responses returned by buckets with Website Configuration enabled and the NotFoundPage property set to a public object in that bucket.
Heh, charging for 5xx would be even more insane than charging for 403. "Our system is not working correctly at this moment. That'll be $0.25."