Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

S3: Plus sign is interpreted as space in the path part of URLs

forums.aws.amazon.com

Post reply on HN
Print view View original

41–49 of 49 posts

Re: S3: Plus sign is interpreted as space in the path part of URLs

#41

Post by tazjin »

hnlmorg wrote: post #38
tazjin wrote: post #35

Earlier quoted context omitted.

> So Amazon could easily have tested their client on 3rd party servers and still not spotted the problem This would still be a red flag, as the service in question is their instance metadata service that provides authentication tokens. Something that important should be integration-tested with the actual service.

> This would still be a red flag, Perhaps I don't understand the issue you're discussing but how would the client working on 3rd party services be a red flag when that is the desired behavior?

Sorry if this was unclear: It's a client that they specifically wrote to talk to their own services, and they're releasing it to their customers as an official way to talk to their own services. It could not talk to their own service.

Their own documentation refers to that library (or did at that point in time, not sure about now).

Re: S3: Plus sign is interpreted as space in the path part of URLs

#42

Post by hnlmorg »

tazjin wrote: post #41
hnlmorg wrote: post #38

Earlier quoted context omitted.

> This would still be a red flag, Perhaps I don't understand the issue you're discussing but how would the client working on 3rd party services be a red flag when that is the desired behavior?

Sorry if this was unclear: It's a client that they specifically wrote to talk to their own services, and they're releasing it to their customers as an official way to talk to their own services. It could not talk to their own service. Their own documentation refers to that library (or did at that point in time, not sure about now).

Ahh I did misunderstand you then. Sorry. Yeah that does sound bad.

Re: S3: Plus sign is interpreted as space in the path part of URLs

#43

Post by [deleted] »

jamix wrote: post #18
ryanbrunner wrote: post #16

I don't necessarily think this is even breaking the HTTP standard. While '+' should not be interpreted as spaces as part of a URL while it's being treated as a URL , the HTTP spec doesn't specify / care what file that may map to on a server. Edit: As mentioned below, this isn't correct since URLs should be able to be escaped and return the same resource, and an escaped + differs from an unescaped + on S3.

Exactly! The OP's point is summarized in this sentence: > My point is that the spec requires + to be escaped only inside the querystring. So what? What the standard mandates for query strings is irrelevant here. It's up to the server how to interpret and map the URLs. "Unconventional and unfortunate" - yes, but breaking the HTTP spec? No.

[deleted]

Re: S3: Plus sign is interpreted as space in the path part of URLs

#45

Post by godDLL »

jordanlev wrote: post #31
mfer wrote: post #22

&tldr; A legacy behavior is to treat + as a space. When you've been around you need to keep backwards compatibility. URLs and URIs have separate standards from HTTP and they have changed over time (been replaced by newer ones). Many years ago it was common to encode a space as a + sign. For example, the PHP function urlencode[1] does the same thing with a + sign. If you're a PHP user, don't use this function unless y…

> If you're a PHP user, don't use this function unless you know you need to. There are better functions now. Don't leave me hanging! What are the better functions now?

`rawurlencode()` is what you're after.

And here is where you'd ask that question, a coding forum https://stackoverflow.com/questions/996139/urlencode-vs-rawu...

Re: S3: Plus sign is interpreted as space in the path part of URLs

#46

Post by Crespyl »

jordanlev wrote: post #31
mfer wrote: post #22

&tldr; A legacy behavior is to treat + as a space. When you've been around you need to keep backwards compatibility. URLs and URIs have separate standards from HTTP and they have changed over time (been replaced by newer ones). Many years ago it was common to encode a space as a + sign. For example, the PHP function urlencode[1] does the same thing with a + sign. If you're a PHP user, don't use this function unless y…

> If you're a PHP user, don't use this function unless you know you need to. There are better functions now. Don't leave me hanging! What are the better functions now?

Knowing PHPs standard library, probably something like "urlencode_safe_for_real_this_time".

Kidding aside, IIRC "rawurlencode" is the RFC compliant one.

Re: S3: Plus sign is interpreted as space in the path part of URLs

#47

Post by mfer »

brlewis wrote: post #29
mfer wrote: post #22

&tldr; A legacy behavior is to treat + as a space. When you've been around you need to keep backwards compatibility. URLs and URIs have separate standards from HTTP and they have changed over time (been replaced by newer ones). Many years ago it was common to encode a space as a + sign. For example, the PHP function urlencode[1] does the same thing with a + sign. If you're a PHP user, don't use this function unless y…

When was + treated as space in the path part of the URL? Sure it's been treated as space in the query part, but that would be a weird breaking change if early web treated path and query the same way, and then later standards made them different.

At the time S3 launched the URL spec was RFC 1738 and we had HTML 4.01[2]. And, the URI syntax (all the way back in 1998) noted to use %20 for a space[3].

As far as I can tell, this traces its history back to encoding for forms[4]. It's been used far beyond the encoding for forms and maybe someone can explain why.

It's also not just PHP whose function is that way. In Python urlencode encodes as a + (at least in 2.7.x).

I remember working on the web many years ago where "+" is what was used. This may have been a spec misinterpretation or something else. In any case, it was common enough.

Note, I'm not saying it was right. Just not uncommon.

[1] https://www.ietf.org/rfc/rfc1738.txt [2] https://www.w3.org/TR/html401/ [3] https://www.ietf.org/rfc/rfc2396.txt [4] https://www.w3.org/TR/html4/interact/forms.html#h-17.13.4.1

Re: S3: Plus sign is interpreted as space in the path part of URLs

#48

Post by jordanlev »

godDLL wrote: post #45
jordanlev wrote: post #31

Earlier quoted context omitted.

> If you're a PHP user, don't use this function unless you know you need to. There are better functions now. Don't leave me hanging! What are the better functions now?

`rawurlencode()` is what you're after. And here is where you'd ask that question, a coding forum https://stackoverflow.com/questions/996139/urlencode-vs-rawu...

Thanks!

And here is where you'd answer that question, a coding forum https://stackoverflow.com/questions/996139/urlencode-vs-rawu...

;)

Re: S3: Plus sign is interpreted as space in the path part of URLs

#49

Post by mike503 »

This burned me and because of it I can't host a specific static site on S3 because it requires plus signs. Can't change the files being uploaded due to the system generating them... tried to rig up some sort of Akamai rewrite rule to change it at the CDN level but couldn't get it to work.
Post reply on HN