Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

S3: Plus sign is interpreted as space in the path part of URLs

forums.aws.amazon.com

Post reply on HN
Print view View original

11–20 of 49 posts

Re: S3: Plus sign is interpreted as space in the path part of URLs

#11

Post by tazjin »

Amazon has a difficult time with the HTTP standard sometimes. Last time I had to touch an AWS project we discovered a bug[1] in the C++ code backing a Java library (sic).

They had implemented their own HTTP client, but forgot to add the "Host" header to requests which is required by HTTP 1.1.

Interestingly this client sent requests only to their own services, which means that they either released that without testing it or the backend once accepted faulty requests.

[1]: https://github.com/awslabs/amazon-kinesis-producer/issues/61

Re: S3: Plus sign is interpreted as space in the path part of URLs

#12

Post by viraptor »

bmn__ wrote: post #3

In response to the reported RFC violation, elving@AWS writes: "I agree that's unconventional and unfortunate." My corporate bullshit detector is off the scale. In earlier times, we would have both the ability and the balls to treat that unwillingness to uphold the rules we all set out with as damage to the Internet, and route around it. But sadly, AWS has become too big to fail, so the engineers introduce special cas…

When were these "earlier times" for the web? Not during browser wars, that's for sure. Not when web2.0 started with crazy ideas about rest. Not during flash-everywhere era. Etc...

Re: S3: Plus sign is interpreted as space in the path part of URLs

#13

Post by acranox »

majewsky wrote: post #9
Liquid_Fire wrote: post #5

Earlier quoted context omitted.

They could make it configurable on a per bucket basis (perhaps defaulting to the old behaviour if necessary; ideally you would make the conformant behaviour the default, of course). That way you could opt in to the standard conformant behaviour if you require it, but they can still keep backward compatibility.

I'm not familiar with how S3 works in detail, but I imagine this could require additional API calls in the backend which increases the latency and resource usage of API requests. In the worst case, such a change could easily require Amazon to purchase dozens, if not hundreds of additional servers.

With the rate of AWS growth, they probably bought dozens more servers in the time it took you to write out your response. :)

Re: S3: Plus sign is interpreted as space in the path part of URLs

#14

Post by viraptor »

Coding_Cat wrote: post #8
marindez wrote: post #2

It's reasonable they don't want to fix it because it will break existing URLs. Welcome to the ugly world of back compatibility.

They could compromise by adding a few more lines of code and having '+' resolve to ' ' if and only if the file can't be found with '+', or vice versa. Immidiatly mark this behaviour as deprecated and switch over to proper '+' == '+' behaviour later. edit: LiquidFire's idea is better.

That would require synchronisation, potentially between multiple servers. Doing this efficiently, without race conditions could be very tricky at their size.

Re: S3: Plus sign is interpreted as space in the path part of URLs

#15

Post by lallysingh »

Liquid_Fire wrote: post #5
marindez wrote: post #2

It's reasonable they don't want to fix it because it will break existing URLs. Welcome to the ugly world of back compatibility.

They could make it configurable on a per bucket basis (perhaps defaulting to the old behaviour if necessary; ideally you would make the conformant behaviour the default, of course). That way you could opt in to the standard conformant behaviour if you require it, but they can still keep backward compatibility.

Or use a different domain for the same buckets, and resolve the name correctly on this new domain.

Re: S3: Plus sign is interpreted as space in the path part of URLs

#16

Post by ryanbrunner »

I don't necessarily think this is even breaking the HTTP standard. While '+' should not be interpreted as spaces as part of a URL while it's being treated as a URL, the HTTP spec doesn't specify / care what file that may map to on a server.

Edit: As mentioned below, this isn't correct since URLs should be able to be escaped and return the same resource, and an escaped + differs from an unescaped + on S3.

Re: S3: Plus sign is interpreted as space in the path part of URLs

#17

Post by kinkrtyavimoodh »

bmn__ wrote: post #3

In response to the reported RFC violation, elving@AWS writes: "I agree that's unconventional and unfortunate." My corporate bullshit detector is off the scale. In earlier times, we would have both the ability and the balls to treat that unwillingness to uphold the rules we all set out with as damage to the Internet, and route around it. But sadly, AWS has become too big to fail, so the engineers introduce special cas…

How is this 'corporate bullshit'? Corporate BS is about giving vague circumlocutionary responses that try to just press all the right PR buttons.

This is the opposite of that.

Re: S3: Plus sign is interpreted as space in the path part of URLs

#18

Post by jamix »

ryanbrunner wrote: post #16

I don't necessarily think this is even breaking the HTTP standard. While '+' should not be interpreted as spaces as part of a URL while it's being treated as a URL , the HTTP spec doesn't specify / care what file that may map to on a server. Edit: As mentioned below, this isn't correct since URLs should be able to be escaped and return the same resource, and an escaped + differs from an unescaped + on S3.

Exactly! The OP's point is summarized in this sentence:

> My point is that the spec requires + to be escaped only inside the querystring.

So what? What the standard mandates for query strings is irrelevant here. It's up to the server how to interpret and map the URLs. "Unconventional and unfortunate" - yes, but breaking the HTTP spec? No.

Re: S3: Plus sign is interpreted as space in the path part of URLs

#19

Post by asdfaoeu »

ryanbrunner wrote: post #16

I don't necessarily think this is even breaking the HTTP standard. While '+' should not be interpreted as spaces as part of a URL while it's being treated as a URL , the HTTP spec doesn't specify / care what file that may map to on a server. Edit: As mentioned below, this isn't correct since URLs should be able to be escaped and return the same resource, and an escaped + differs from an unescaped + on S3.

Sure but /%2B should resolve to the same thing as /+

Re: S3: Plus sign is interpreted as space in the path part of URLs

#20

Post by cm2187 »

mmahemoff wrote: post #7
bmn__ wrote: post #3

In response to the reported RFC violation, elving@AWS writes: "I agree that's unconventional and unfortunate." My corporate bullshit detector is off the scale. In earlier times, we would have both the ability and the balls to treat that unwillingness to uphold the rules we all set out with as damage to the Internet, and route around it. But sadly, AWS has become too big to fail, so the engineers introduce special cas…

To the contrary, I think it's actually a refreshingly honest response. A "corporate bullshit" response would be to ignore it altogether, try to argue it's a feature not a bug, or give a canned statement about how we respect the environment and want the world to be a better place. The AWS support is explicitly acknowledging it's an issue, while giving a rational reason why it probably won't be fixed (even if you disag…

But can they even change it without risking to break tens of thousands of websites?
Post reply on HN