Live data from Hacker News

Amazon S3 will no longer charge for several HTTP error codes

aws.amazon.com

31–40 of 77 posts

Re: Amazon S3 will no longer charge for several HTTP error codes

#31
post #13
post #3

The system works! Just raise your concerns and they'll get around to it in [checks notes] 18 years https://twitter.com/cperciva/status/1785402732976992417

In the same timespan Microsoft released Windows 1 all the way up to XP

Why have things stagnated?

Re: Amazon S3 will no longer charge for several HTTP error codes

#32

We've done it. Now let's re-engineer our apps to use error codes for 200 responses and get free S3 usage.

If I understand TFA, you'd need to find a way to get S3 (which offers no server-side script execution, only basic file delivery) to emit an error code (403 specifically) alongside a response of useful data. Good luck...

Well, you can probably send out one bit a time by updating your ACLs on a clock (with which your clients are also roughly synchronized) and distinguishing between 403 and 404.

take an awful lot of time to get that data out, though.

Re: Amazon S3 will no longer charge for several HTTP error codes

#33
post #12

Can't imagine a change like this would be made without some analysis.. would love an internal view into a decision like this, I wonder if they already have log data to compute financial loss from the change, or if they have sampling instrumentation fancy enough to write/deploy custom reports like this quickly. In any case 2 weeks seems like an impressive turnaround for such a large service, unless they'd been interna…

> 2 weeks seems like an impressive turnaround for such a large service

I assume they were lucky in that whatever system counts billable requests also has access to the response code, and therefore it's pretty easy to just say "if response == 403: return 0".

The fact that is the case suggests they may do the work to fulfill the request before knowing the response code and doing billing, so there might be some loophole to get them to do lots of useful work for free...

Re: Amazon S3 will no longer charge for several HTTP error codes

#34

We've done it. Now let's re-engineer our apps to use error codes for 200 responses and get free S3 usage.

If I understand TFA, you'd need to find a way to get S3 (which offers no server-side script execution, only basic file delivery) to emit an error code (403 specifically) alongside a response of useful data. Good luck...

Simple. Just encode all of your app's data and logic as a massive lookup table, each bit of which is represented by an object that either doesn't exist (a zero) or is unauthorized to access (a one).

When you read a sequential series of keys (404 403 403 404 = 0110) it will either tell you the data you were looking for or the next key name to begin reading from.

Re: Amazon S3 will no longer charge for several HTTP error codes

#35
post #12

Can't imagine a change like this would be made without some analysis.. would love an internal view into a decision like this, I wonder if they already have log data to compute financial loss from the change, or if they have sampling instrumentation fancy enough to write/deploy custom reports like this quickly. In any case 2 weeks seems like an impressive turnaround for such a large service, unless they'd been interna…

> 2 weeks seems like an impressive turnaround for such a large service I assume they were lucky in that whatever system counts billable requests also has access to the response code, and therefore it's pretty easy to just say "if response == 403: return 0". The fact that is the case suggests they may do the work to fulfill the request before knowing the response code and doing billing, so there might be some loophole…

> do lots of useful work for free

Have often wondered about this in terms of some of their control plane APIs, a read-only IAM key used as part of C&C infrastructure for a botnet might be interesting, you get DNS/ClientHello signature to a legitimate and reputable service for free, while stuffing "DDoS this blog" e.g. in some tags of a free resource. Even better if the AWS account belonged to someone else.

But certainly, ability to serve an unlimited URL space from an account with only positive hits being billed seems ripe for abuse. Would guess there's already some ticket for a "top 404ers" internal report or similar

Re: Amazon S3 will no longer charge for several HTTP error codes

#36
post #32

Earlier quoted context omitted.

If I understand TFA, you'd need to find a way to get S3 (which offers no server-side script execution, only basic file delivery) to emit an error code (403 specifically) alongside a response of useful data. Good luck...

Well, you can probably send out one bit a time by updating your ACLs on a clock (with which your clients are also roughly synchronized) and distinguishing between 403 and 404. take an awful lot of time to get that data out, though.

> take an awful lot of time to get that data out, though.

That’s what glacier is for!

Re: Amazon S3 will no longer charge for several HTTP error codes

#38
post #19

Earlier quoted context omitted.

It has the same cost as a successful response which can quickly add up to a few hundred dollars per month with a couple of DNS enumeration scans. Google Cloud and Azure also bill DNS like this. Unless you need some of the advanced features you really shouldn't host your DNS in the big cloud providers.

Cloudflare don't bill like this, part of why I moved off route 53.

Neither does Digital Ocean, for that matter.

Re: Amazon S3 will no longer charge for several HTTP error codes

#39
post #20

AWS is full of dark patterns. You can sign up for the so-called "free" tier and then too easily, unwittingly enable something that suddenly charges you hundreds of dollars before you know it (by getting a bill at the end of the month), even if you're not doing anything with the service except looking around. AWS doesn't give any warning to free tier members that a configuration change is going to cost you, and their…

> AWS doesn't give any warning

It does if you ask it to. You can get billing alerts if current costs are projected to go over a threshold.

Post reply on HN