Earlier quoted context omitted.
Very timely as I just recently ended up with a URL query string so big that CloudFront rejected the request before it even hit my server.. Ended up switching that endpoint to POST. Would've liked QUERY for that!
I have come across systems that use GET but with a payload like POST. This allows the GET to bypass the 4k URL limit. It's not a common pattern, and QUERY is a nice way to differentiate it (and, I suspect will be more compatible with Middleware). I have a suspicion that quite a few servers support this pattern (as does my own) but not many programmers are aware of it, so it's very infrequently used.
The HTTP Query Method
111–120 of 136 posts
Re: The HTTP Query Method
#112Earlier quoted context omitted.
Cache in web middleware like Apache or nginx by default ignores GET request body, which may lead to bugs and security vulnerabilities.
But as I said, you control that infra. I don’t think it’s unreasonable to expect your sysadmins, devops, platform engineers, or whatever title you choose to give them, to set up these services correctly, given it’s their job to do so and there’s a plethora of security risks involved. If you can’t trust them to do that little, then you’re fuck regardless of whether you decide to send payloads as GET bodies. And there…
Exactly, and the correct way to setup GET requests is to ignore their bodies for caching purposes because they aren't expected to exist: "content received in a GET request has no generally defined semantics, cannot alter the meaning or target of the request, and might lead some implementations to reject the request and close the connection because of its potential as a request smuggling attack" (RFC 9110)
> And there isn’t any good reason not to contract pen testers to check over everything afterwards.
I am pretty sure our SecOps and Infra Ops and code standards committee will check it and declare that GET bodies is a hard no.
Re: The HTTP Query Method
#113Earlier quoted context omitted.
> unless I'm missing something I don't think a QUERY verb will change all that much here? The semantics are important. GET APIs are expected to be safe, idempotent, and cache-friendly. When you are unable to use GET for technical reasons and move to POST, suddenly none of the infrastructure (like routers, gateways, or generic http libs) can make these assumptions about your API. For example, many tools will not attem…
You can just use body with GET. QUERY is redundant.
> Although request message framing is independent of the method used, content received in a GET request has no generally defined semantics, cannot alter the meaning or target of the request, and might lead some implementations to reject the request and close the connection because of its potential as a request smuggling attack (Section 11.2 of [HTTP/1.1]). A client SHOULD NOT generate content in a GET request unless it is made directly to an origin server that has previously indicated, in or out of band, that such a request has a purpose and will be adequately supported. An origin server SHOULD NOT rely on private agreements to receive content, since participants in HTTP communication are often unaware of intermediaries along the request chain.
QUERY is a new option to help avoid some of those downsides.
Re: The HTTP Query Method
#114Earlier quoted context omitted.
But as I said, you control that infra. I don’t think it’s unreasonable to expect your sysadmins, devops, platform engineers, or whatever title you choose to give them, to set up these services correctly, given it’s their job to do so and there’s a plethora of security risks involved. If you can’t trust them to do that little, then you’re fuck regardless of whether you decide to send payloads as GET bodies. And there…
> I don’t think it’s unreasonable to expect your sysadmins, devops, platform engineers, or whatever title you choose to give them, to set up these services correctly, given it’s their job to do so and there’s a plethora of security risks involved. Exactly, and the correct way to setup GET requests is to ignore their bodies for caching purposes because they aren't expected to exist: "content received in a GET request…
No. The correct way to set up this infra is the way that works for a particular problem while still being secure.
If you’re so inflexible as an engineer that you cannot set up caching correctly for a specific edge case because it breaks you’re preferred assumptions, then you’re not a very good engineer.
> and might lead some implementations to reject the request and close the connection because of its potential as a request smuggling attack"
Once again, you have control over the implementations you use in your infra.
Also It’s not a RSA if the request is supposed to contain a payload in the body.
> I am pretty sure our SecOps and Infra Ops and code standards committee will check it and declare that GET bodies is a hard no.
I wouldn’t be so sure. I’ve worked with a plethora of different infosec folk from those who will mandate that PostgreSQL needs to use non-standard ports because of mandating strict compliance with NIST, even for low risk reports. To others that have been fine with some pretty massive deviations from traditionally recommended best practices.
The good infosec guys, and good platform engineers too, don’t look at things in black and white like you are. They build up a risk assessment and judge each deviation on its own merit. Thus GET body payloads might make sense in some specific scenarios.
This doesn’t mean that everyone should do it nor that it’s a good idea outside of those niche circumstances. But it does mean that you shouldn’t hold on to these rigid rules like gospel truths. Sometimes the most pragmatic solution is the unconditional one.
That all said, I can’t think of any specific circumstance where you’d want to do this kind of hack. But that doesn’t mean that reasonable circumstances would never exist.
Re: The HTTP Query Method
#115Earlier quoted context omitted.
It would be pretty impossible to actually ‘enforce’ that GETs don’t modify state. I am not sure if I would call the lack of enforcement a problem when it is more a simple fact about distributed systems; no specification can enforce what a service does outside of the what is returned in a response.
That is exactly my point. There is no reason to syntactically distinguish what is semantically non-distinguishable. The interpretation of a request is up to the server. There is no reason for the client to syntactically distinguish that the request body is for a POST vs QUERY; the request parameters and response have the same shape with the same serialization format. However, on the other side, a server does control…
The point is to have a standard, so you can more easily learn what an API is doing instead of having to start from scratch every time.
Re: The HTTP Query Method
#116Earlier quoted context omitted.
> I don’t think it’s unreasonable to expect your sysadmins, devops, platform engineers, or whatever title you choose to give them, to set up these services correctly, given it’s their job to do so and there’s a plethora of security risks involved. Exactly, and the correct way to setup GET requests is to ignore their bodies for caching purposes because they aren't expected to exist: "content received in a GET request…
> Exactly, and the correct way to setup GET requests is to ignore their bodies for caching purposes because they aren't expected to exist No. The correct way to set up this infra is the way that works for a particular problem while still being secure. If you’re so inflexible as an engineer that you cannot set up caching correctly for a specific edge case because it breaks you’re preferred assumptions, then you’re not…
Running PostgreSQL on any non-privileged port is a feature, not a deviation from a standard. If you want to run PostgreSQL on a port under 1024 now that would be security vulnerability as it requires root access.
There is no reason to "build up a risk assessment and judge each deviation on its own merit" unless it is an unavoidable technical limitation. Just don't make your life harder for no reason.
Re: The HTTP Query Method
#117Earlier quoted context omitted.
> I don’t think it’s unreasonable to expect your sysadmins, devops, platform engineers, or whatever title you choose to give them, to set up these services correctly, given it’s their job to do so and there’s a plethora of security risks involved. Exactly, and the correct way to setup GET requests is to ignore their bodies for caching purposes because they aren't expected to exist: "content received in a GET request…
> Exactly, and the correct way to setup GET requests is to ignore their bodies for caching purposes because they aren't expected to exist No. The correct way to set up this infra is the way that works for a particular problem while still being secure. If you’re so inflexible as an engineer that you cannot set up caching correctly for a specific edge case because it breaks you’re preferred assumptions, then you’re not…
For one, you’re wrong about TLS meaning only your infra and the client matter. Some big corps install a root CA on everyone’s laptop and MITM all HTTP/S traffic. The one I saw was Bluecoat, no idea if it follows your expected out-of-spec behavior or not.
For two, this is likely to become incredibly limiting at some point and require a bunch of work to re-validate or re-implement. If you move to AWS, do ELBs support this? If security wants you to use Envoy for a service mesh, is it going to support this? I don’t pick all the tools we use, so there’s a good chance corporate mandates something incompatible with this.
You would need very good answers to why this is the only solution and is a mandatory feature. Why can’t we cache server side, or implement our own caching in the front end for POST requests? I can’t think of any situations where I would rather maintain what is practically a very similar fork of HTTP than implement my own caching.
Re: The HTTP Query Method
#118Earlier quoted context omitted.
> POST requests by design are neither safe not idempotent. Outright wrong. You are allowed to handle POST requests in a safe and idempotent way. In fact, the existing usage of POST to query, the literal impetus for this proposal, has exactly that behavior. What you are not allowed to do is assume that any arbitrary POST request is safe and idempotent. Only a endpoint that is documented to support POST and that is doc…
> Outright wrong. You are allowed to handle POST requests in a safe and idempotent way. You clearly have no idea about what you are discussing. The definition of HTTP verb semantics clearly defines the concept of safe and idempotent methods. You should seriously learn about the concept before commenting on discussions on semantics. If that's too much work, RFC 9110 clearly and unambiguously states that only the GET,…
The behavior and constraints of a generic interface is a lower bound on the behavior and upper bound on the assumable constraints by all direct participants when handled in general. This is POST in general. This is the weakest usage.
A specialized interface may enhance behavior or tighten constraints further. This is actually using POST to achieve some website-specific end. You can not actually use a website/reason about usage in anything other than a read-only fashion without knowing what the endpoints actually do and when to issue requests at which endpoint.
Indirect or intermediary participants should maintain behavior and constraints at the lower of the level of specialization they are aware of and the participants they are between. If you are designing a generic intermediary, then you do not have access to or knowledge about the specialized interfaces, so are less able to leverage specialized behavior or constraints. In HTTP, various headers serve the purpose of communicating various classes of common interface specializations so someone can write more powerful generic intermediarys. However, you could always write a specific intermediary or handler if you control portions of the chain, which people do all the time.
This applies across all classes of software interfaces and protocols not just websites using HTTP. But I guess HTTP is just such a special snowflake that it is beyond learning good practice from other fields.
Re: The HTTP Query Method
#119PSA: When posting an RFC or (especially) an RFC-draft, please use the IETF Datatracker URL. For example, this one is: https://datatracker.ietf.org/doc/draft-ietf-httpbis-safe-met... The best way to view an RFC, IMHO, is to use the "htmlized" format: you can view and compare different versions, view errata for a formal RFC, and go back to Datatracker at any time. Also, the Datatracker URL is version-insensitive, so un…
From the Datatracker, I can see that it was called SEARCH until draft-2 (Nov 2021), and then changed to QUERY. Also, the previous SEARCH method was proposed in Apr 2015 (!!), but nobody took it seriously, and it never gained traction back then. A lot of software was developed/updated during the last decade, and a lot of opportunities were missed. Even if the QUERY method is turned into a formal RFC right now, expect…
See the history of the PATCH method, and of the whole WebDAV thing.
Re: The HTTP Query Method
#120Earlier quoted context omitted.
There's nothing holding you back implementing the QUERY method right now - HTTP methods are standardized, but not limited to the standard. Obsviously it depends how proxies/servers/etc. might handle uncommon methods.
> There's nothing holding you back implementing the QUERY method right now - HTTP methods are standardized, but not limited to the standard. I think this comment is terribly naive. Technically you can write your own service to support verbs like LOL or FUBAR, but in practice your service is not only expected to be broken when passing requests through other participants you do not control but also it requires far more…