Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

S3: Plus sign is interpreted as space in the path part of URLs

forums.aws.amazon.com

Post reply on HN
Print view View original

21–30 of 49 posts

Re: S3: Plus sign is interpreted as space in the path part of URLs

#21

Post by jchw »

jamix wrote: post #18
ryanbrunner wrote: post #16

I don't necessarily think this is even breaking the HTTP standard. While '+' should not be interpreted as spaces as part of a URL while it's being treated as a URL , the HTTP spec doesn't specify / care what file that may map to on a server. Edit: As mentioned below, this isn't correct since URLs should be able to be escaped and return the same resource, and an escaped + differs from an unescaped + on S3.

Exactly! The OP's point is summarized in this sentence: > My point is that the spec requires + to be escaped only inside the querystring. So what? What the standard mandates for query strings is irrelevant here. It's up to the server how to interpret and map the URLs. "Unconventional and unfortunate" - yes, but breaking the HTTP spec? No.

It breaks the HTTP spec because it internally is decoding the URL wrong. This is important because things that speak HTTP are free to choose to percent encode, or not, the plus sign in a path, and the canonical URL should not differ. If it mapped even an escaped plus to a space, it'd be consistent, though still questionable, behavior.

Re: S3: Plus sign is interpreted as space in the path part of URLs

#22

Post by mfer »

&tldr; A legacy behavior is to treat + as a space. When you've been around you need to keep backwards compatibility.

URLs and URIs have separate standards from HTTP and they have changed over time (been replaced by newer ones).

Many years ago it was common to encode a space as a + sign. For example, the PHP function urlencode[1] does the same thing with a + sign. If you're a PHP user, don't use this function unless you know you need to. There are better functions now.

[1] http://php.net/manual/en/function.urlencode.php

Re: S3: Plus sign is interpreted as space in the path part of URLs

#24

Post by ryanbrunner »

asdfaoeu wrote: post #19
ryanbrunner wrote: post #16

I don't necessarily think this is even breaking the HTTP standard. While '+' should not be interpreted as spaces as part of a URL while it's being treated as a URL , the HTTP spec doesn't specify / care what file that may map to on a server. Edit: As mentioned below, this isn't correct since URLs should be able to be escaped and return the same resource, and an escaped + differs from an unescaped + on S3.

Sure but /%2B should resolve to the same thing as /+

Ah, fair enough, that's a good point.

Re: S3: Plus sign is interpreted as space in the path part of URLs

#25

Post by mmahemoff »

cm2187 wrote: post #20
mmahemoff wrote: post #7

Earlier quoted context omitted.

To the contrary, I think it's actually a refreshingly honest response. A "corporate bullshit" response would be to ignore it altogether, try to argue it's a feature not a bug, or give a canned statement about how we respect the environment and want the world to be a better place. The AWS support is explicitly acknowledging it's an issue, while giving a rational reason why it probably won't be fixed (even if you disag…

But can they even change it without risking to break tens of thousands of websites?

They could, by versioning the API (e.g. add a /v2/ to all paths), but that would benefit no-one and should only be done alongside any number of much more important changes.

Re: S3: Plus sign is interpreted as space in the path part of URLs

#26

Post by zAy0LfpBZLC8mAC »

jamix wrote: post #18
ryanbrunner wrote: post #16

I don't necessarily think this is even breaking the HTTP standard. While '+' should not be interpreted as spaces as part of a URL while it's being treated as a URL , the HTTP spec doesn't specify / care what file that may map to on a server. Edit: As mentioned below, this isn't correct since URLs should be able to be escaped and return the same resource, and an escaped + differs from an unescaped + on S3.

Exactly! The OP's point is summarized in this sentence: > My point is that the spec requires + to be escaped only inside the querystring. So what? What the standard mandates for query strings is irrelevant here. It's up to the server how to interpret and map the URLs. "Unconventional and unfortunate" - yes, but breaking the HTTP spec? No.

Please read the actual spec before telling poeple whether something is conforming to it or not. Just making stuff up is exactly how this mess is created. The relevant section in this case:

https://tools.ietf.org/html/rfc3986#section-6.2.2.2

Re: S3: Plus sign is interpreted as space in the path part of URLs

#27

Post by Aissen »

mmahemoff wrote: post #25
cm2187 wrote: post #20

Earlier quoted context omitted.

But can they even change it without risking to break tens of thousands of websites?

They could, by versioning the API (e.g. add a /v2/ to all paths), but that would benefit no-one and should only be done alongside any number of much more important changes.

Isn't the API so huge that you could just configure a bucket (default-off) to behave properly ?

Re: S3: Plus sign is interpreted as space in the path part of URLs

#28

Post by majewsky »

acranox wrote: post #13
majewsky wrote: post #9

Earlier quoted context omitted.

I'm not familiar with how S3 works in detail, but I imagine this could require additional API calls in the backend which increases the latency and resource usage of API requests. In the worst case, such a change could easily require Amazon to purchase dozens, if not hundreds of additional servers.

With the rate of AWS growth, they probably bought dozens more servers in the time it took you to write out your response. :)

Likely, but Amazon didn't get where they are by ignoring small costs.

Re: S3: Plus sign is interpreted as space in the path part of URLs

#29

Post by brlewis »

mfer wrote: post #22

&tldr; A legacy behavior is to treat + as a space. When you've been around you need to keep backwards compatibility. URLs and URIs have separate standards from HTTP and they have changed over time (been replaced by newer ones). Many years ago it was common to encode a space as a + sign. For example, the PHP function urlencode[1] does the same thing with a + sign. If you're a PHP user, don't use this function unless y…

When was + treated as space in the path part of the URL? Sure it's been treated as space in the query part, but that would be a weird breaking change if early web treated path and query the same way, and then later standards made them different.

Re: S3: Plus sign is interpreted as space in the path part of URLs

#30

Post by hnlmorg »

tazjin wrote: post #11

Amazon has a difficult time with the HTTP standard sometimes. Last time I had to touch an AWS project we discovered a bug[1] in the C++ code backing a Java library (sic). They had implemented their own HTTP client, but forgot to add the "Host" header to requests which is required by HTTP 1.1. Interestingly this client sent requests only to their own services, which means that they either released that without testing…

It's common for HTTP servers to accept requests without a host header. It's not usually needed by the server unless you're hardening it (I don't class it as a security issue but some security audits will flag it up if you don't force the server to reject invalid host headers) or running named virtual hosts (which is more common than it used to be thanks to SNI but you still often see a 1:1 relationship between (virtual) hosts and IPs). So Amazon could easily have tested their client on 3rd party servers and still not spotted the problem.

As an anecdote, about 15/20ish years ago I wrote my own webbrowser. Obviously something highly rudimentary albeit browsers were much easier to implement back then anyway. I was too lazy to read the HTTP spec (it was a hobby project and I was young and impatient) so a lot of what I did was trial and error. I too wasn't sending a host header but it took long while before I ran into any sites that rejected my HTTP requests. The web landscape was very different back then though and IPs were plentiful but it just goes to show how servers have coded around bad clients for years.

Post reply on HN