&tldr; A legacy behavior is to treat + as a space. When you've been around you need to keep backwards compatibility. URLs and URIs have separate standards from HTTP and they have changed over time (been replaced by newer ones). Many years ago it was common to encode a space as a + sign. For example, the PHP function urlencode[1] does the same thing with a + sign. If you're a PHP user, don't use this function unless y…
> If you're a PHP user, don't use this function unless you know you need to. There are better functions now.
Don't leave me hanging! What are the better functions now?
In response to the reported RFC violation, elving@AWS writes: "I agree that's unconventional and unfortunate." My corporate bullshit detector is off the scale. In earlier times, we would have both the ability and the balls to treat that unwillingness to uphold the rules we all set out with as damage to the Internet, and route around it. But sadly, AWS has become too big to fail, so the engineers introduce special cas…
Eh, URL/URI escaping is an interesting example, because people have been doing inconsistent and sometimes standards-problematic things with it pretty much as long as they existed. And indeed it's been a perpetual pain and problem. (just one example read up on `&` and `;`, and whether `&` can/should/must be escaped in what contexts; that's not the only one, `+` is another long-standing one). So not a great example of how everyone used to always be consistently standards-compliant in "earlier times", more like a counter-example. I don't think it's unique, my experience is not that everything used to be more consistent and standards compliant in "earlier times" than it is now, when it comes to the web, if anything the reverse.
&tldr; A legacy behavior is to treat + as a space. When you've been around you need to keep backwards compatibility. URLs and URIs have separate standards from HTTP and they have changed over time (been replaced by newer ones). Many years ago it was common to encode a space as a + sign. For example, the PHP function urlencode[1] does the same thing with a + sign. If you're a PHP user, don't use this function unless y…
> If you're a PHP user, don't use this function unless you know you need to. There are better functions now. Don't leave me hanging! What are the better functions now?
Anyone that's dealt with S3 in any capacity should be aware of this, it's literally one of the first encoding problems to come up when dealing with signing requests.
Amazon has a difficult time with the HTTP standard sometimes. Last time I had to touch an AWS project we discovered a bug[1] in the C++ code backing a Java library (sic). They had implemented their own HTTP client, but forgot to add the "Host" header to requests which is required by HTTP 1.1. Interestingly this client sent requests only to their own services, which means that they either released that without testing…
It's common for HTTP servers to accept requests without a host header. It's not usually needed by the server unless you're hardening it (I don't class it as a security issue but some security audits will flag it up if you don't force the server to reject invalid host headers) or running named virtual hosts (which is more common than it used to be thanks to SNI but you still often see a 1:1 relationship between (virtu…
> So Amazon could easily have tested their client on 3rd party servers and still not spotted the problem
This would still be a red flag, as the service in question is their instance metadata service that provides authentication tokens.
Something that important should be integration-tested with the actual service.
But can they even change it without risking to break tens of thousands of websites?
They could, by versioning the API (e.g. add a /v2/ to all paths), but that would benefit no-one and should only be done alongside any number of much more important changes.
That opens a whole new can of worms. They have not deprecated a single part of the s3 API ever, in history (okay I'm partially lying, the SOAP API is now officially deprecated in the sense that they won't add new features to it).
They're not going to change it just for the sake of some minor path issues that have a workaround. (Side note: They tend to uses headers rather than paths for API declarations). I have personally been bit by this same issue, but I would never recommend they change pieces of the service to accommodate it. It's handled easily in client code. What they could do is make an obvious "gotchas" section of documentation
AWS has done a tremendous job of getting things right-enough the first time. They have never killed an AWS service. They'd need a much bigger reason to version an API.
Should this really be considered to be a spec violation? It's a restriction, sure, but S3 is to be considered a specific application with specific constraints.
Does S3 use HTTP ? If so, it's a violation of the specification of S3 by way of incorporation of the HTTP specification.
Otherwise, if S3 does not use HTTP, we would need to see the S3 specification to determine if it (the implementation Amazon uses) is in violation
It's common for HTTP servers to accept requests without a host header. It's not usually needed by the server unless you're hardening it (I don't class it as a security issue but some security audits will flag it up if you don't force the server to reject invalid host headers) or running named virtual hosts (which is more common than it used to be thanks to SNI but you still often see a 1:1 relationship between (virtu…
> So Amazon could easily have tested their client on 3rd party servers and still not spotted the problem This would still be a red flag, as the service in question is their instance metadata service that provides authentication tokens. Something that important should be integration-tested with the actual service.
> This would still be a red flag,
Perhaps I don't understand the issue you're discussing but how would the client working on 3rd party services be a red flag when that is the desired behavior?
They could compromise by adding a few more lines of code and having '+' resolve to ' ' if and only if the file can't be found with '+', or vice versa. Immidiatly mark this behaviour as deprecated and switch over to proper '+' == '+' behaviour later. edit: LiquidFire's idea is better.
That would require synchronisation, potentially between multiple servers. Doing this efficiently, without race conditions could be very tricky at their size.
It also makes lookups ambiguous if you have both. :)