Live data from Hacker News

Amazon S3 will no longer charge for several HTTP error codes

aws.amazon.com

71–77 of 77 posts

Re: Amazon S3 will no longer charge for several HTTP error codes

#71
post #29

Earlier quoted context omitted.

Are you for real? Legitimately baffled by your comment. How about the financial losses of customers that could be DDoS-ed into bankruptcy through no fault of their own? Keeping S3 bucket names secret is not always easy.

I prefer your version: Barr replies to a tweet before gatecrashing the next S3 planning session. "A customer is hurting , folks!". The call immediately falls silent with only occasional gasps heard from stunned engineers, and the gentle weeping of a PM. I wonder if Amazon offers free therapy following an incident like this

Not billing you because a script kiddie ran a script on your S3 bucket is a good start of a therapy, I'd say. :)

Re: Amazon S3 will no longer charge for several HTTP error codes

#72
post #19

Earlier quoted context omitted.

I just searched for this and this documentation entry came up: https://docs.aws.amazon.com/whitepapers/latest/aws-best-prac... I can't believe that their 'fix' is to set a wildcard dns entry, this feels somewhat like a joke. Does this mean that a NXDOMAIN response costs more than a successful response?

It has the same cost as a successful response which can quickly add up to a few hundred dollars per month with a couple of DNS enumeration scans. Google Cloud and Azure also bill DNS like this. Unless you need some of the advanced features you really shouldn't host your DNS in the big cloud providers.

That’s not entirely true - aliases to AWS resources are free (their suggested “workaround”).

Re: Amazon S3 will no longer charge for several HTTP error codes

#73
post #68
post #67

There needs to be a law that says any user needs to set any limit on any service or subscription, and then the costs can not surpass this until the budget is upped by the user. At the same time, there should be real-time cost analysis, breakdown per service and predicted costs per day.

A law in which country? Well, GDPR showed a bit that rather global impact is possible. If you offer an open service on the internet you need to be prepared that users and misusers will cause costs. However, if you block it for public access you as a customer are not offering a public service. It's the cloud provider offering a public service so it seems just a basic legal principle that it's the cloud provider who pa…

Agreed. Don’t know what made them wake up, but I did file a complaint about their free tier dark patterns and the Luxembourg EU GDPR office got involved after my countries GDPR office tried it first, and apparently is busy with some bigger investigation, so that investigation might’ve spooked them (not my own application I don’t think)

Re: Amazon S3 will no longer charge for several HTTP error codes

#74
post #32

Earlier quoted context omitted.

If I understand TFA, you'd need to find a way to get S3 (which offers no server-side script execution, only basic file delivery) to emit an error code (403 specifically) alongside a response of useful data. Good luck...

Well, you can probably send out one bit a time by updating your ACLs on a clock (with which your clients are also roughly synchronized) and distinguishing between 403 and 404. take an awful lot of time to get that data out, though.

It seems to me you could just use static ACLs and create (or not) object names to cause this 403 vs 404 distinction? The drawback is that you'll be paying for the minimum retention of minimum-sized objects, not to mention all the other bucket management traffic you are using.

So you're going to have a lot of consumers of the same bit stream before you've somehow made the covert, "free" egress a net positive value versus a regular object. I imagine AWS can trivially put in place some throttling of error responses to make this impractical.

Ignoring these economic issues, imagine a content-addressing scheme like /stream-identifier/bitnumber which you can then poll to fetch one bit per request. Populate an object (which will return 403) for 1 bits and omit an object (which will return 404) for 0 bits.

You also need to know some stream length or "end of stream" limit. Otherwise you can't tell if you've read past the end or are really fetching 0 bits of a longer stream.

One strategy might be to use an 8b/10b encoding so you can detect when you're not getting a valid symbol anymore. You could treat that as end of stream if it is supposed to be static, or go into some polling mode to wait for more symbols to be posted.

Hybrid strategies might use regular objects or recursive use of these streams to publish metadata streams that tell you about the available stream names, lengths, and encoding schemes.

Re: Amazon S3 will no longer charge for several HTTP error codes

#75
post #53

Earlier quoted context omitted.

> Can't imagine a change like this would be made without some analysis.. would love an internal view into a decision like this Sure, here you go: There was some buzz and negative press so it got picked up by the social media managers who forwarded it to executive escalations who loops in legal. Legal realizes that what they are doing is borderline fraud and sends it to the VP that oversees billing as a P0. It then ge…

As someone that works at AWS (but not on S3), that's wrong in like eight different ways. But the only way that matters is the core one - analysis, data, and instrumentation. AWS does not make these kinds of decisions without a look at the metrics.

As someone who has been involved in high level crisis management issues like this multiple times across various companies I can tell you that in a competent organization it looks nothing like your day-to-day decision making as an engineer or PM. Better yet, as few "rank and file" employees are involved as possible to avoid dangerous situations like you just described.

I don't want to debate the merits of what happened, but a prosecutor is going to open with "AWS billed people for things they never asked for or consented to." You're already fighting an uphill battle that it is not fraud.

Now what is going to save you is intent. If your defense is "yeah we identified the problem and corrected it" you're good to go. If on the other hand, someone decides to run a fucking metrics report of how much you could lose by stopping doing fraud and god forbid it is ever seen or mentioned in front of anyone in the decision making path - you now have to deal with mens rea.

If you have material knowledge that someone took "a look at the metrics", shoot me an email. I can help put you in touch with programs that offer financial rewards for whistleblowers.

Re: Amazon S3 will no longer charge for several HTTP error codes

#77
post #52

Now please do this for NXDOMAIN on Route53. This can be a big problem with acquired domains.

You should never actually use Route53 for your domains. Delegate a subdomain like cloud.yourcompany.net to R53 and use that.

Why?
Post reply on HN