Live data from Hacker News

RFC 10008: The new HTTP Query Method

rfc-editor.org

61–70 of 189 posts

Re: RFC 10008: The new HTTP Query Method

#61

Earlier quoted context omitted.

Idempotency is an important attribute for correctness. Yep, you can document that POSTing to $ENDPOINT is idempotent, but you can't communicate that to caching layers throughout the network. QUERY, by definition, is idempotent and cacheable.

[flagged]

Larger scales like what? I expect that everywhere you currently cache GETs you can cache QUERYs. But does caching GETs work at scale?

Re: RFC 10008: The new HTTP Query Method

#62
post #51

> GET request with a body was heavily considered by the IETF working group, but it was ultimately rejected in favor of creating the new QUERY method. The decision to create a distinct method came down to historical interoperability issues and strict compliance with the core architectural definitions of HTTP. I've been sending request body along GET method for years now

Apparently some load balancers drop the body.

I expect all sorts of intermediaries may drop the body, since having a body is forbidden by the standard.

When it's your client talking to your server you can obviously do whatever you want - it doesn't cause problems until you want to involve third-party code, such as a reverse proxy (such as nginx) or a CDN. This includes proxies your customers may be using.

Re: RFC 10008: The new HTTP Query Method

#63

As "just a guy that programs" (ok, now guides agents to program) and tries to follow the rules (with a big dose of pragmatism), this totally makes sense to me. This is also the first time I've seen or heard about this coming. I like that we now have a way to not being forced to define Resources when we want to query. It always felt like I was missing something that there could be an infinite, defined-on-the-fly numbe…

The standards have always been a bit more abstract than you use in practice. Common practice prior to this would be /catsearch?sleep=20&sunbeams=yes&breakfast=0300 where the "resource" is /catsearch and the rest are query parameters, but you could also use /catsearch/sleep/20/sunbeams/yes/breakfast/0300 which looks like a "resource", but nobody is actually enforcing that it is a "resource".

Re: RFC 10008: The new HTTP Query Method

#64
post #33

Earlier quoted context omitted.

QUERY is GET with a request body. So it must be safe , not just idempotent . Where safe means it has no significant side-effects. Typically servers will not keep any state for QUERY requests. There is one interesting variant though, which uses state: The client sends a QUERY containing the full query, and the server returns a url usable with GET with which this query can be triggered in the future. Similar to prepare…

Thanks for the explanation! I still don’t get how idempotency can typically be ensured without state. It very much depends on data model and application design. Even side effects like using a user’s lookup quota need to be handled at a higher layer than HTTP (I think?).

Yes it varies. Using the QUERY method doesn't automatically mean your app is idempotent - it means the browser, and any intermediaries, can assume it's idempotent. So when you go forward and then back they're free to reissue the request and you won't get the "this may repeat whatever you just did" popup.

If it's not actually idempotent but you're telling the browser it is, of course you may cause bugs. Same as GET.

Re: RFC 10008: The new HTTP Query Method

#65
post #10

Including a strong motivating example might have helped sell this, using an example that could trivially be expressed as a GET is extremely distracting. Even imagining a QUERY with a large JSON filtering structure, or say an image input as request body, it feels extremely odd to include the request body as part of the cache key. It also implies an unbounded and user-controlled cache key, with the only really meaningf…

The browser can simply store a collision resistant hash (e.g. SHA-256) of the body, if it wants a smaller cache key. I can't really think of any caching related attacks that don't equally apply to a query parameter. Generating a unique 30 character query parameter is just as easy as generating a 30 MB request body, if you want to flood the cache.

Not necessarily that simple, as you'd have sort all the input parameters to maintain a useable cache key. Not especially difficult, but if the data is large and so re-allocation and sorting is required, then you're starting to open up the attack surface where bugs might have been introduced.

Re: RFC 10008: The new HTTP Query Method

#66
post #33

Earlier quoted context omitted.

Thanks for the explanation! I still don’t get how idempotency can typically be ensured without state. It very much depends on data model and application design. Even side effects like using a user’s lookup quota need to be handled at a higher layer than HTTP (I think?).

Minor side-effects like quotas or request logging are generally ignored when considering the semantics of http methods. I don't see any complications for QUERY that don't already apply to GET. It just allows you to bypass the url length limit by putting the data in the body instead of the url itself.

Exactly. "GET / is idempotent" ... "But I just launched a DDOS against / and now it returns 502...

Re: RFC 10008: The new HTTP Query Method

#67
post #60

Just in case anyone wants to pretend it's still that other century: https://www.rfc-editor.org/rfc/rfc10008.txt

I'll forever love a long, totally plain text document like this. So many good times with video game FAQs as a kid. It really is a superior form of information in a lot of ways (not all).

Re: RFC 10008: The new HTTP Query Method

#68

> GET request with a body was heavily considered by the IETF working group, but it was ultimately rejected in favor of creating the new QUERY method. The decision to create a distinct method came down to historical interoperability issues and strict compliance with the core architectural definitions of HTTP. I've been sending request body along GET method for years now

> I've been sending request body along GET method for years now

Generally not a great idea. With some http implementations this is not even possible (for example, fetch)

https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/U...

> You cannot include a body with GET requests

And transparent caching might result in weird issues.

Re: RFC 10008: The new HTTP Query Method

#69
post #59
post #10

Including a strong motivating example might have helped sell this, using an example that could trivially be expressed as a GET is extremely distracting. Even imagining a QUERY with a large JSON filtering structure, or say an image input as request body, it feels extremely odd to include the request body as part of the cache key. It also implies an unbounded and user-controlled cache key, with the only really meaningf…

Not all usage scenarios are the public internet, and something doesn't have to be useful on the public internet to be standardized. Realistically, systems for the public internet will use a secure hash as the cache key so it'll always be the same size. The cache key already includes a URL that can be very long, and an arbitrary set of header values.

Except that by definition, in a URL the data has no implicit meaning so for a cache hit you need an exact match, including order and case, but for a list of POST parameters, they could legitimately be in any order and so you can't just hash it all as a blob, you need to sort the keys, possibly copy data around (unless using keys plus hash), probably allocating more memory, etc. I'm pretty certain we'll see at least one CVE out of the first few implementations of this!

Re: RFC 10008: The new HTTP Query Method

#70
post #45

Earlier quoted context omitted.

I'm guessing you never hit this issue then, but it's a real issue. Whether or not it's in the RFC as a hard limit it doesn't matter, no HTTP server will allow unlimited sized URIs. You simply can't base64 large payloads and you're stuck with workarounds.

You are guessing wrong. Thanks, I know specific implementation will come with their limits. This will equally apply to QUERY body size and caching strategy. Are we seriously ok with linking the RFC as source while providing a statement that doesn't match? RFC does matter.

The RFC does say "It is RECOMMENDED that all senders and recipients support, at a minimum, URIs with lengths of 8000 octets in protocol elements."

One can infer from the RFC that you can reasonably expect many implementations to fail beyond 8000 characters, and that there are no guarantees up to that either.

True, the RFC doesn't specify a limit, but it does clearly indicate that it's not unbounded, nor should you expect it to be.

Post reply on HN