It's possible that the slogan "There's one obvious way to do it" as a riposte to "There's more than one way to do it" was more responsible for Python's first wave of success than anything else.
Insofar as I'm concerned, a GET request with a body is an attack-shaped aberration. E.g. Somebody who's trying to get me to mix up validating query string parameters and request body parameters.
I'd say it's the framework doing the hacky thing. It should be optional. AFAIK, the HTTP spec allows for it, under certain conditions. "A client SHOULD NOT generate content in a GET request unless it is made directly to an origin server that has previously indicated, in or out of band, that such a request has a purpose and will be adequately supported."
Using GET with a Body doesn't work if you try using it in the browser with JS fetch for example[1]. Additionally, a lot of existing web servers by default ignore GET requests with a body.
The use case of QUERY is because POST conveys non-safe, non-idempotent requests which can potentially modify stuff according to the REST spec. GET requests on the other hand convey retrieval of a resource. However, due to GET requests not having a body, there's a limit to the amount of data you can put in the URL and you also cannot put sensitive data in it.
Additionally, GET requests are meant to be highly cacheable by default while a lot of the QUERY type requests are usually meant more for one-shot access.
There's no such thing as REST spec. The closes mechanism to actual REST is to create a resource using POST and then query it using GET. You have the added benefit of the resource being cacheable.
Not exactly. OPTIONS lets servers tell you that QUERY is supported if the client requests it and Status Code 405 lets servers tell you that QUERY is not supported. That's very different from the current landscape where several proxy servers and web servers simply ignore the Body of a GET request.
Yeah I really don't understand the anti-GET-body argument.
"Using GET with a body isn't in the spec, WAFs and webservers that haven't been updated might reject it!"
Ok, QUERY wasn't in the spec when those were written either. What do you expect those appliances to do with a totally unknown verb?
It's a welcome addition but the new method is pure marketing. There's no reason the update couldn't have been to expand GET instead of add support for QUERY.
It's exactly the opposite. The HTTP spec does cover GET with bodies. However, what you fail to account is that the spec specifies they are invalid and a GET with body is meaningless, and represents a potential attack.
Sure but 405 Method Not Allowed is a response you can fallback from, whereas "body was silently stripped by a middlebox" is not as easy to know when it happens, much less deal with.
Actually you are quite wrong. HTTP already accommodaties nonstandard methods, so HTTP-compliant servers do support whatever string you put together as the method.
What you are failing to understand is that this proposal defines both a method and its semantics. This means the expected behavior regarding idempotency, safety, cacheabiliry. Nonstandard methods by design are interpreted as being unsafe.
What you are also failing to understand is that GET is explicitly designed to not have a request body. This means that a GET with a body is interpreted as something that violates specifications and is potentially an attack such as request smuggling. Again, some API gateways strip them as a security precaution.
> Most of the systems that are blocking GET/body could be easily tweaked to allow it. Today. As is.
Utter nonsense. You are talking about things like home routers and old phones.
> QUERY will likely need firmware updates, core engine updates, etc.
Not really, only if those devices do not comply with HTTP.
The real value proposition is the semantics of a QUERY operarion regarding safety and caching. It's not a coincidence that this proposal is backed by the likes of Cloudflare. All HTTP compliant requests are just forwarded through all internet boxes, and the likes of Cloudflare sits at the edge safely caching them.
Yep. We had to change our app when we took on a client with a strictly configured WAF which rejected GET with body. I know I have come across multiple points where I have used POST when I know it is wrong, or GET with a body, when I know it is wrong. So I welcome QUERY!
1) Changes to how GET works likely require a new HTTP version to assure maximum breaking change awareness. We're already in a Postel's Law state where we have HTTP/1.1, HTTP/2, and HTTP/3 all running side-by-side for reasons that all three are very different under the hood at even the transport layers. Do we really want to add HTTP/1.2, HTTP/2.1, and HTTP/3.1 to that list? An entirely new method is easier to apply horizontally to all three versions, because the HTTP standard already allows that as an extension mechanism. (There's an IANA registry for HTTP methods and HTTP methods such as WebDAV's have always been their own standards outside of HTTP RFCs.)
2) A plain separation between GET should only accept query string parameters and QUERY should only accept body parameters potentially reduces attack surface of attackers trying to mix and match the two to find potential order of precedence attacks.
You're missing the point. Using GET with a body is currently unspecified, so of course you're not supposed to do it (though you're not forbidden to either).
But specifying this behavior would get you in the same situation as adding a new method: everything not up to date with the spec will keep behaving poorly but newer system would work.
The only benefit of adding a new method is for marketing/awareness: it may end up getting support faster than the alternative because it sounds as a sexy new thing to implement. This kind of benefit should not be overlooked, but we should also acknowledge its limits: most of enterprise stuff (WAF, frameworks, etc.) are not going to work overnight just because it's a new method instead of a spec change.
Slightly off topic Funfact: you can buy a several thousand dollars expensive ssl intercepting proxy appliance which doesn’t support anything beyond http/1.1.
Will be fun when those see a whole new http verb, I bet that leads to at least DoS by the track record of that company.
Historically there have been vulnerabilities in various applications due to HTTP method tampering, and in the days of people accidentally leaving WebDAV enabled then methods like PUT and DELETE could be very damaging. Plus the issues with TRACK and TRACE.
Given that most websites only ever use a handful of methods (even once you account for REST APIs using PUT, PATCH and DELETE now), and that list very rarely changes, the WAF developers tend to look at this question from the opposite angle: when you know there are only half a dozen widely used methods, why would you allow anything else by default?
> using HTTP GET with a request body is a bad idea, as for example users behind a corporate firewall or a different browser may be unable to use your website.
So is using QUERY requests for quite some time from now.
> The difference is the method. Query you're saying I can use body. GET you should never use body.
The biggest win is how intermediary boxes now have concrete guidance that a specific HTTP request is both safe, idempotent, and carries a request body. Up until now none of this existed, and at best developers could use unsafe methods to carry request bodies (see GraphQL and how it uses POST for queries)
Then all bets are off, and I guess we just can’t HTTP ever again because a proxy can misbehave.
One should adhere to Best Practices since one cannot control every device between the app and the user. Best Practice says “GET has no body. QUERY can have a body. If QUERY fails (405), use POST with the body.” And eventually, enough proxies will behave well enough that at least the HTTP bit of the app has a chance of working.
Do you test with a proxy? I know I don't, and if I would it'd be the latest version of squid.
Some of those non-conforming middle boxes could be unattended black boxes in the back of a closet somewhere on the 3rd floor. Who knows. Some customers would consider alternatives to your offering because it doesn't work, others would file a support ticket and there becomes an incentive not to use the new and shiny. The broken middle black box vendor can't be bothered and keeps selling broken boxes. Shit like this is why we can't have nice things.
The fact that some infrastructure is poorly maintained is not a reason against evolving protocols, it's a reason to maintain infrastructure better. It's really not that difficult to do.
I wonder what the drawbacks of standardizing a GET body would have been. CoAP already has it (which creates friction in building CoAP<->HTTP proxies).
All in all, I dislike the overall focus on the HTTP method when designing "RESTful" interfaces. If all we're building is, effectively, an RPC, why would the cacheability meta-information be the first thing we specify?
A absolute swats of middle boxes that will not get addressed ever. As industry, it's preferable to create something that is a hard break and makes players upgrade and give people a feature to argue for said upgrade
It's interesting to see additions to HTTP methods as it much feels like the existing ones are set in stone. At least for the time that I have been a developer.
I'm curious to see how fast the adoption/support for HTTP QUERY will be. I've had my fair share of situations where I wished for something like HTTP QUERY.
This assumes that all of the infrastructure surrounding your application also plays nicely with it and supports it, as stated in the article. That's why I expressed my interest in speed of adoption in the first place.
I think the article summarizes pretty well what the drawbacks of POST are: unclear idempotency (well it's actually pretty damned clear: they are not cacheable). That complicates caching logic, and that's not just for the application server itself, but any reverse proxies in front of it as well as the user agent itself.
I'm not sure QUERY is a great solution, because in the context of a web application absolutely no one enjoys using a page that does not keep its state on refresh, so that really limits where QUERY makes sense, but if you have a case that is not driven by navigation, great.
> zero. Many libs will/can just request method as a string so you can start coding now
Not so fast, champ. It matters nothing what your library let's you run away with. What matters is that every single box in the internet between you and the origin server will tolerate, and your pet library doesn't have a say in that.
> Using POST instead comes with no drawbacks
There's a hefty share of ignorance in your comment. Between POST being classified as an unsafe method and the absence of support for cashing, there are plenty of downsides of abusing POST for query requests.
As the RFC was initially proposed by someone from Cloudflare, were you aware that not even Cloudflare support caching POST requests? Their unofficial support for caching POST requests is to create a fake GET request to serve as cache key and use that to cache the response. This is the kind of hacks everyone is forced to go through instead of using something like QUERY
This is why there's another method. It's easier to communicate 'QUERY method support' rather than get-with-a-body-no-not-the-one-that-is-unspecified-it-is-accepted-with-slightly-different-semantics-now-EOF.
Would this be a defensible decision if the spec were designed today, an additional read method that takes the same argument, entirely for the purpose of not ignoring a specific property? It seems like just the path of least resistance considering all the controversy and legacy tools. That is not a good way to maintain the functionality and long-term relevance of a spec. But if there is a good reason to design it this way from the beginning, I'm curious to know more.
My guess is that if you were building all of this from scratch, you would start with
- request-with-a-body
- idempotent-request-with-a-body
- safe-request-with-a-body
because the additional constraints induce properties that are extremely useful to general purpose clients ("I didn't get a reply to my idempotent-request-with-a-body, can I resend it without risking loss of property?")
Would someone then come along an introduce safe-request-without-a-body method? After all, we can already meet that "need" with safe-request-with-a-body and content-length: 0.
Think rfc-5789::PATCH - mechanically, it's just another request-with-a-body, but with more tightly constrained semantics. But general purpose components can take advantage of the additional properties, and so we introduce a "niche" method with tighter constraints.
Document resource manipulation is a common case, so we probably end up with a family of specialized methods, in much the same way that we have a bunch of WebDAV methods.
There are good arguments that if you were designing this from scratch it would still make sense to separate GET and QUERY. GET are things addressable purely by URI. QUERY are things that need HTML forms or JS to initiate (but optionally may return GET addressable URLs for future requests).
Similarly there are good arguments that if you were designing this from scratch it would make sense to still separate QUERY and POST. To some extent they mean very different things: "search" versus "create"/"do". Some of that is a modern expectation from years of mapping the common "CRUD" concepts to "REST": POST ~= Create; GET ~= Read; PUT ~= Update; DELETE == Delete. But that's a lens that's still useful in designing the thing from scratch even if it wasn't necessarily in mind when HTTP was first designed (especially given the different verbs in HTTP terminology).
Three different definitions of the hypertext transfer protocol (HTTP). Choose one
1. What the most popular HTTP servers actually accept
2. What the most popular HTTP clients actually send
3. What the RFC "standard" says
Personally I choose #1. I don't use the most popular HTTP clients
For example, under definition #1, I can do HTTP/1.1 pipelining with POST
The RFC "standard" often comes after the software implementation(s); most RFCs document what's already in use on the internet. As the selection of software expands, some competent software authors trying their best still often struggle to conform to RFCs. Go figure
To me, the source code of the most popular servers in use is the standard
The other issue with adding a separate supported way to do what people did with GET+body is that we will probably see servers slowly drop support for the GET+body approach when QUERY gets widespread support/usage, and then a ton of other stuff will break.
Unless you're really going to improve things or the existing practices are really too painful, standards should follow convention. Even though GET+body is not handled the same everywhere, it's easier to make that the standard than it is to make a new syntax the standard.
With standards it helps to reconcile existing behaviour, rather than create new unproven syntax.
Likewise creating a new syntax for something that already exists means you are just adding to the heap of stuff that needs support on mainstream servers, and as I already said it will probably create compatibility issues as the old deprecated/illegal syntax is removed. This is unnecessary friction.
And really what is the advantage of a new syntax? That needs explaining.
But it's not existing behaviour. It does not already exist. Breaking the semantics of something that exists means that things will work inconsistently. Breaking the semantics of get will also create compatibility issues.
Does the article provide supporting evidence for your claim that "Even though GET+body is not handled the same everywhere, it's easier to make that the standard than it is to make a new syntax the standard."? Because if it does I don't see it. To me it just seems like a baseless claim.
This is awesome and very much needed. Sending massive get requests always felt like shit and support for body parsing of GET was all over the place. I hope it will be adopted quickly.
Nice, not having bodies on GET has been a pet peeve of mine for a long time. It would be nice to allow bodies on DELETE as well, but that is less of a problem in most cases.
If you're doing anything complicated enough to need so much data that it'd be better to send the data in a body, it's probably not a DELETE and so POST would be more appropriate anyway.
DELETE is intended to delete one specific object, pointed to by a unique URL, not to delete arbitrary objects matching some criteria.
It blows my mind that people are invested in this. HTTP is a 35+ year old text-based protocol. Its becoming the COBOL of digital transmission.
Just as one example, among many, you could try WebSockets (or some other similar protocol) and then push anything over it. Your message could be plain text, JSON, binary, whatever. Web Sockets and protobufs are bidirectional (full duplex) too.
The internet is complex, and you have tons of protocols that are not well supported. TCP/HTTP are well supported by proxies and have well-defined, stable specs, which also help with caching, throttling, etc.
Just because it's old doesn't mean it is worse than alternatives, most likely it is quite the contrary.
Its worse than alternatives because it imposes greater complexity on everybody that uses it. COBOL is also old. That also does not mean COBOL is bad, but nobody wants to write COBOL any more when there are so many better options available.
I wish there was an HTTP method that directly signals async intent of the server. I know we have 202 Accepted status, but these days there are so many APIs that use the async patterns and each one differs a bit. Having a standard for accepting async jobs with idempotency and notification about results via webhook
I still don't get the need for QUERY. One can create a search or filter resource with a POST request and then query it using GET. As a bonus, creating a resource allows it to be shared and cached.
Obviously the approach you mentioned has the downside of two server round-trips being necessary while the QUERY request only requires a single round-trip. Not to mention the two-request approach adds more complexity to both clients and servers, as it mandates that both the client and server have to physically create and manage those resources.
Because HTTP is stateless by definition, you now need to support persistence (state) on the server side whenever you want to run a slightly different query, which contradicts the preamble.
I understand the confusion around GraphQL's cached/persisted queries, but this is not the intention of HTTP.
Right, that process works today just fine, but that's application code that the browser (and middleboxes) can't assume. The idea is that QUERY directly represents that flow to the browser as a direct participant (and to caching middleboxes as well). A QUERY can (optionally) create a search or filter resource and return a Location: response header that points to a GET resource to refresh the next time the same QUERY is asked. The browser can directly cache that Location and associate it with that QUERY body. (So can middleboxes.)
A POST can return Content-Location: to the GET resource, but the browser can't assume that the same POST body contents create the same result Content-Location, whereas the QUERY Location represents the QUERY itself as a repeatable object. (Also, QUERY can return Content-Location instead of or in addition to Location for subtly different caching implications.)
What do you think people will make the Query request body? Most everything will use this for JSON but it could be anything so what other interesting things do you think will go in there? Query 1 + 1 and get 2?
I'm curious too. Unless the developer is really passionate about this I don't think a dev will risk (potential) compatibility issues or unexpected footguns to use this when the workarounds do seem to work quite well already. I just dont see the benefit but maybe it's because I am just not aware of a real world use case; happy to be corrected.
Elastic/Opensearch uses GET requests with a body for search, which is complicated or forbidden (not exactly sure) with the HTTP spec. Not all HTTP clients are willing to submit a body with a GET.
So opensearch also allows you to POST search requests, but those are uncacheable
QUERY would fit here perfectly - it's probably trivial for opensearch to add but it will take some time for clients to catch up.
I disagree. I think the adoption (or dismissal) of QUERY will show.
First thing that comes to mind is that the idempotency of GET resources are easy to handle. URL's have a fixed size, they can be efficiently hashed, cached and are unambiguous about how they serve this purpose.
It is unclear how the ecosystem will deal with the QUERY requirements. It's easy for apps, but browsers, http caches and servers will take some time to figure out solutions.
Fixing GET would have the same amount of uncertainty in addition to the need to keep current expectations valid. It's not easier, it's harder.
We might start using QUERY for the search requests from our web app to our server, if nothing in the stack in between the app and our server side code does not drop the body. A JSON body beats the spaghetti arguments of most filters.
Unironically QUERY seems to me a "quality of life" feature for the SPA world and the ultimate legitimation to kill link sharing. It is too hard to keep links and app state consistent, so lets just drop it.
It won't break GraphQL, as it uses POST. It can very much improve it if adopted:
- use QUERY method when querying resources
- use POST method for mutations
The only valid argument against HTTP GET with a body is that it has privacy/security risks.
Exist stuff (caches, CDN, etc.) could serve private information because the HTTP GET is cached without checking the request contents. The new standard can avoid this because old stuff does not know about HTTP QUERY.
The article also addresses why this is not the chosen solution. It's pretty much the first one you'd think of: all kinds of existing software (that can be between client and server and out of their control) already handle GET bodies in all kinds of incompatible ways, because the existing standard says they're meaningless and "shouldn't" be included. The idea is to not break people's stuff, so they don't rugpull the established standard.
There's usually a reason why the simplest solution that pops into one's head is not "just" used by the people who put a lot more thought into it. Not always, but it can be useful to try to come up with it.
Arguably the only explanation you need is that "QUERY is the same as GET plus a body". The article just explains what GET is and isn't, but that can be implied.
Yeah I always disliked that there's this idea that you can't put a body on a GET request.
Iirc openapi generators goes out of its way to not support that which has lead to me writing a small rant into an API specification before to explain why the get_xyz uses POST...
Even HTTP 1.0 RFC[11] is fairly clear on this, although it doesn't explicitly spell it out like RFC 9110. GET requests should only consider the Request-URI and request bodies should only be included if the method calls for it.
> ... unless it is made directly to an origin server that has previously indicated, in or out of band, that such a request has a purpose and will be adequately supported.
A lot of the RFCs are flavored by the lack of https and prevalence of forward proxies run by the ISPs to improve perceived speed and reduce their network loads.. Back in the day.
By my estimation, that's why they explicitly call out only the client and origin can know what a GET body means; proxies should forward as-is and ignore.
Those days of ISP forward http proxies are gone, but those semantics are still fine; the body means what the origin and client agree it means.
I have a weird feeling. Query body is encrypted by https. So CDN will not be able to cache results.
In order to make it work right - whole topology of the internet should be redone. Caching on the backend server will not give any real gains for large scale apps.
The whole connection is encrypted by https, the request body is treated the same as the url, the headers or the response. The only unencrypted parts are the IP addresses/ports and the domain name (if SNI without ECH is used).
CDNs already terminate TLS connections so they can cache GET requests.
Even past the TLS point (CDNs terminate TLS, so they can read the body) there's a harder problem nobody's solved: to cache a QUERY the cache has to fold the body into the cache key, and there's no standard way to canonicalize a request body. {"a":1,"b":2} and {"b":2,"a":1} are the same query and two different cache entries; whitespace, float formatting, unordered keys all fork the key. GET gets this for free because the URL is already a normalized string. So "cacheable in principle" is real, but "actually cached" needs every layer to agree on a canonical form first - the same coordination problem that killed GET-with-body. I want QUERY for the honest semantics; I just wouldn't budget for cache hits yet
Can you elaborate? Do you think the noted issues are non-issues that should be fixed in applications? Which would require a new standard to create stable JSON outputs, which is just one use case that can fail right now.
> {"a":1,"b":2} and {"b":2,"a":1} are the same query and two different cache entries
These two payloads are actually different. You're talking about semantics, which is determined by the payload format; in the case of JSON, these two are semantically similar.
> GET gets this for free because the URL is already a normalized string
It's the same principle; the order of properties matters too.
"QUERY is just GET"
"Using GET with a Body works"
Seems like this is going everyone's head. You're not supposed to use GET with a Body, this is a hack, therefore having an explicit method makes sense.
Just because it works, doesn't mean its the right way
Tell that to anybody in the business long enough to decipher someone else's Perl!
It's possible that the slogan "There's one obvious way to do it" as a riposte to "There's more than one way to do it" was more responsible for Python's first wave of success than anything else.
Hey, remember what Larry said, there's more than one way to do things ;P
I’ve seen a framework strip body content off GET requests, so doing hacky things doesn’t even always work. The QUERY method is a welcome addition
Insofar as I'm concerned, a GET request with a body is an attack-shaped aberration. E.g. Somebody who's trying to get me to mix up validating query string parameters and request body parameters.
Hacky things not working is a feature, not a bug.
Is the stripper service in question already implementing it?
I'd say it's the framework doing the hacky thing. It should be optional. AFAIK, the HTTP spec allows for it, under certain conditions. "A client SHOULD NOT generate content in a GET request unless it is made directly to an origin server that has previously indicated, in or out of band, that such a request has a purpose and will be adequately supported."
The whole stack is a pile of badly designed hacks. Not much point in fixing it now. I mean they can’t even spell referrer correctly.
Using GET with a Body doesn't work if you try using it in the browser with JS fetch for example[1]. Additionally, a lot of existing web servers by default ignore GET requests with a body.
The use case of QUERY is because POST conveys non-safe, non-idempotent requests which can potentially modify stuff according to the REST spec. GET requests on the other hand convey retrieval of a resource. However, due to GET requests not having a body, there's a limit to the amount of data you can put in the URL and you also cannot put sensitive data in it.
Additionally, GET requests are meant to be highly cacheable by default while a lot of the QUERY type requests are usually meant more for one-shot access.
QUERY is meant to address these limitations.
[1]: https://github.com/whatwg/fetch/issues/551
There's no such thing as REST spec. The closes mechanism to actual REST is to create a resource using POST and then query it using GET. You have the added benefit of the resource being cacheable.
To clarify, by REST spec I meant the state of access defined by https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/....
This is just HTTP. The definitive guide to REST is "Architectural Styles and the Design of Network-based Software Architectures" by Roy Fielding.
>Additionally, a lot of existing web servers by default ignore GET requests with a body.
I think the point made is that _all_ existing web servers have no idea what a "QUERY" is anyway, so changes need to be made anyhow.
Not exactly. OPTIONS lets servers tell you that QUERY is supported if the client requests it and Status Code 405 lets servers tell you that QUERY is not supported. That's very different from the current landscape where several proxy servers and web servers simply ignore the Body of a GET request.
Some security/ API gateway block requests when it's a GET with a body.
> "Using GET with a Body works"
Except it doesn't. Some API gateways outright strip request bodies from GET requests to prevent them from being forwarded.
It sounds like most people with the "just use GET" nonsense are far from having any experience in cloud computing.
QUERY won't be supported by them either.
So, change is required. Just change GET to allow for body and move on.
Most of the systems that are blocking GET/body could be easily tweaked to allow it. Today. As is.
QUERY will likely need firmware updates, core engine updates, etc.
Meanwhile, tweaking GET is a rule change.
Yeah I really don't understand the anti-GET-body argument.
"Using GET with a body isn't in the spec, WAFs and webservers that haven't been updated might reject it!"
Ok, QUERY wasn't in the spec when those were written either. What do you expect those appliances to do with a totally unknown verb?
It's a welcome addition but the new method is pure marketing. There's no reason the update couldn't have been to expand GET instead of add support for QUERY.
> What do you expect those appliances to do with a totally unknown verb?
405 Method Not Allowed
We have existing standards for unsupported methods.
> Using GET with a body isn't in the spec,
It's exactly the opposite. The HTTP spec does cover GET with bodies. However, what you fail to account is that the spec specifies they are invalid and a GET with body is meaningless, and represents a potential attack.
Sure but 405 Method Not Allowed is a response you can fallback from, whereas "body was silently stripped by a middlebox" is not as easy to know when it happens, much less deal with.
> QUERY won't be supported by them either.
Actually you are quite wrong. HTTP already accommodaties nonstandard methods, so HTTP-compliant servers do support whatever string you put together as the method.
What you are failing to understand is that this proposal defines both a method and its semantics. This means the expected behavior regarding idempotency, safety, cacheabiliry. Nonstandard methods by design are interpreted as being unsafe.
What you are also failing to understand is that GET is explicitly designed to not have a request body. This means that a GET with a body is interpreted as something that violates specifications and is potentially an attack such as request smuggling. Again, some API gateways strip them as a security precaution.
> Most of the systems that are blocking GET/body could be easily tweaked to allow it. Today. As is.
Utter nonsense. You are talking about things like home routers and old phones.
> QUERY will likely need firmware updates, core engine updates, etc.
Not really, only if those devices do not comply with HTTP.
The real value proposition is the semantics of a QUERY operarion regarding safety and caching. It's not a coincidence that this proposal is backed by the likes of Cloudflare. All HTTP compliant requests are just forwarded through all internet boxes, and the likes of Cloudflare sits at the edge safely caching them.
Yep. We had to change our app when we took on a client with a strictly configured WAF which rejected GET with body. I know I have come across multiple points where I have used POST when I know it is wrong, or GET with a body, when I know it is wrong. So I welcome QUERY!
AWS CloudFront blocks GET requests with a body, so it doesn't even have to be a particularly strict setup or an explicit WAF.
It sounds like GET with a body is just undefined behavior.
Why not just standardize it? It seems to be a better way than adding a new method.
A lot of un-updateable software out there that strip the body. Especially when the companies behind it doesn't provide support anymore.
will those support QUERY?
1) Changes to how GET works likely require a new HTTP version to assure maximum breaking change awareness. We're already in a Postel's Law state where we have HTTP/1.1, HTTP/2, and HTTP/3 all running side-by-side for reasons that all three are very different under the hood at even the transport layers. Do we really want to add HTTP/1.2, HTTP/2.1, and HTTP/3.1 to that list? An entirely new method is easier to apply horizontally to all three versions, because the HTTP standard already allows that as an extension mechanism. (There's an IANA registry for HTTP methods and HTTP methods such as WebDAV's have always been their own standards outside of HTTP RFCs.)
2) A plain separation between GET should only accept query string parameters and QUERY should only accept body parameters potentially reduces attack surface of attackers trying to mix and match the two to find potential order of precedence attacks.
You're missing the point. Using GET with a body is currently unspecified, so of course you're not supposed to do it (though you're not forbidden to either).
But specifying this behavior would get you in the same situation as adding a new method: everything not up to date with the spec will keep behaving poorly but newer system would work.
The only benefit of adding a new method is for marketing/awareness: it may end up getting support faster than the alternative because it sounds as a sexy new thing to implement. This kind of benefit should not be overlooked, but we should also acknowledge its limits: most of enterprise stuff (WAF, frameworks, etc.) are not going to work overnight just because it's a new method instead of a spec change.
So a POST with explicit caching semantics?
no, a POST that is idempotent. caching is an optional side benefit.
Why not just add an optional body to the spec for GET
HTTP QUERY was discussed many times in the past here:
https://news.ycombinator.com/item?id=48568502 (4d ago, 173 comments)
https://news.ycombinator.com/item?id=29794838 (4y ago, 125 comments)
Yes but the author wants to promote his postman replacement
Slightly off topic Funfact: you can buy a several thousand dollars expensive ssl intercepting proxy appliance which doesn’t support anything beyond http/1.1.
Will be fun when those see a whole new http verb, I bet that leads to at least DoS by the track record of that company.
Not mentioning the vendor… means your comment is true for every vendor :)
That would be lancom. Good routers, the rest not so much.
What is the use-case for a WAF/proxy/etc. to block unknown HTTP verbs? It feels like a pathway for obsolescence with no actual security benefit?
Not block, straight out crash the process is my guess will happen here.
If it crashes it's a security issue.
It should adept reject what it does not expect.
Historically there have been vulnerabilities in various applications due to HTTP method tampering, and in the days of people accidentally leaving WebDAV enabled then methods like PUT and DELETE could be very damaging. Plus the issues with TRACK and TRACE.
Given that most websites only ever use a handful of methods (even once you account for REST APIs using PUT, PATCH and DELETE now), and that list very rarely changes, the WAF developers tend to look at this question from the opposite angle: when you know there are only half a dozen widely used methods, why would you allow anything else by default?
> using HTTP GET with a request body is a bad idea, as for example users behind a corporate firewall or a different browser may be unable to use your website.
So is using QUERY requests for quite some time from now.
Yeah, query seems just GET with a body. No difference in protocol nor behavior
Except compatibility. If you're using classic GET and it's enough for you, you aren't affected.
What is compatible with a QUERY but not with a GET ?
Intermediate proxies, caches, CDNs, firewalls, and load balancers.
That is only in the case of GET with a body though.
Yes. That is the issue under discussion, e.g. not "classic GET".
The difference is the method. Query you're saying I can use body. GET you should never use body.
> The difference is the method. Query you're saying I can use body. GET you should never use body.
The biggest win is how intermediary boxes now have concrete guidance that a specific HTTP request is both safe, idempotent, and carries a request body. Up until now none of this existed, and at best developers could use unsafe methods to carry request bodies (see GraphQL and how it uses POST for queries)
There is the Accept-Query header https://www.rfc-editor.org/info/rfc10008/#appendix-A.3 that tells you can use QUERY. That's a bit different.
405 Method Not Allowed is trivial to fall back to POST. How do you know the GET request behaved incorrectly?
That's assuming the corporate proxy is well-behaved.
Then all bets are off, and I guess we just can’t HTTP ever again because a proxy can misbehave.
One should adhere to Best Practices since one cannot control every device between the app and the user. Best Practice says “GET has no body. QUERY can have a body. If QUERY fails (405), use POST with the body.” And eventually, enough proxies will behave well enough that at least the HTTP bit of the app has a chance of working.
We can HTTP and all bets are still off, probably.
Do you test with a proxy? I know I don't, and if I would it'd be the latest version of squid.
Some of those non-conforming middle boxes could be unattended black boxes in the back of a closet somewhere on the 3rd floor. Who knows. Some customers would consider alternatives to your offering because it doesn't work, others would file a support ticket and there becomes an incentive not to use the new and shiny. The broken middle black box vendor can't be bothered and keeps selling broken boxes. Shit like this is why we can't have nice things.
The fact that some infrastructure is poorly maintained is not a reason against evolving protocols, it's a reason to maintain infrastructure better. It's really not that difficult to do.
I wonder what the drawbacks of standardizing a GET body would have been. CoAP already has it (which creates friction in building CoAP<->HTTP proxies).
All in all, I dislike the overall focus on the HTTP method when designing "RESTful" interfaces. If all we're building is, effectively, an RPC, why would the cacheability meta-information be the first thing we specify?
A absolute swats of middle boxes that will not get addressed ever. As industry, it's preferable to create something that is a hard break and makes players upgrade and give people a feature to argue for said upgrade
It's interesting to see additions to HTTP methods as it much feels like the existing ones are set in stone. At least for the time that I have been a developer. I'm curious to see how fast the adoption/support for HTTP QUERY will be. I've had my fair share of situations where I wished for something like HTTP QUERY.
I can implement it in about 10 minutes. Not even kidding.
In what role? As a user writing client code or when implementing the caching middleware or the Webserver?
In my CRUD controller that I already have.
This assumes that all of the infrastructure surrounding your application also plays nicely with it and supports it, as stated in the article. That's why I expressed my interest in speed of adoption in the first place.
zero. Many libs will/can just request method as a string so you can start coding now
> I've had my fair share of situations where I wished for something like HTTP QUERY.
Using POST instead comes with no drawbacks
I think the article summarizes pretty well what the drawbacks of POST are: unclear idempotency (well it's actually pretty damned clear: they are not cacheable). That complicates caching logic, and that's not just for the application server itself, but any reverse proxies in front of it as well as the user agent itself.
I'm not sure QUERY is a great solution, because in the context of a web application absolutely no one enjoys using a page that does not keep its state on refresh, so that really limits where QUERY makes sense, but if you have a case that is not driven by navigation, great.
Much harder to get CDN:s/proxies/etc to cache a post request vs this new one (assuming it’s actually becomes used)
> zero. Many libs will/can just request method as a string so you can start coding now
Not so fast, champ. It matters nothing what your library let's you run away with. What matters is that every single box in the internet between you and the origin server will tolerate, and your pet library doesn't have a say in that.
> Using POST instead comes with no drawbacks
There's a hefty share of ignorance in your comment. Between POST being classified as an unsafe method and the absence of support for cashing, there are plenty of downsides of abusing POST for query requests.
As the RFC was initially proposed by someone from Cloudflare, were you aware that not even Cloudflare support caching POST requests? Their unofficial support for caching POST requests is to create a fake GET request to serve as cache key and use that to cache the response. This is the kind of hacks everyone is forced to go through instead of using something like QUERY
This is why there's another method. It's easier to communicate 'QUERY method support' rather than get-with-a-body-no-not-the-one-that-is-unspecified-it-is-accepted-with-slightly-different-semantics-now-EOF.
OK, but stop trying to make fetch happen.
Would this be a defensible decision if the spec were designed today, an additional read method that takes the same argument, entirely for the purpose of not ignoring a specific property? It seems like just the path of least resistance considering all the controversy and legacy tools. That is not a good way to maintain the functionality and long-term relevance of a spec. But if there is a good reason to design it this way from the beginning, I'm curious to know more.
My guess is that if you were building all of this from scratch, you would start with
- request-with-a-body
- idempotent-request-with-a-body
- safe-request-with-a-body
because the additional constraints induce properties that are extremely useful to general purpose clients ("I didn't get a reply to my idempotent-request-with-a-body, can I resend it without risking loss of property?")
Would someone then come along an introduce safe-request-without-a-body method? After all, we can already meet that "need" with safe-request-with-a-body and content-length: 0.
Think rfc-5789::PATCH - mechanically, it's just another request-with-a-body, but with more tightly constrained semantics. But general purpose components can take advantage of the additional properties, and so we introduce a "niche" method with tighter constraints.
Document resource manipulation is a common case, so we probably end up with a family of specialized methods, in much the same way that we have a bunch of WebDAV methods.
There are good arguments that if you were designing this from scratch it would still make sense to separate GET and QUERY. GET are things addressable purely by URI. QUERY are things that need HTML forms or JS to initiate (but optionally may return GET addressable URLs for future requests).
Similarly there are good arguments that if you were designing this from scratch it would make sense to still separate QUERY and POST. To some extent they mean very different things: "search" versus "create"/"do". Some of that is a modern expectation from years of mapping the common "CRUD" concepts to "REST": POST ~= Create; GET ~= Read; PUT ~= Update; DELETE == Delete. But that's a lens that's still useful in designing the thing from scratch even if it wasn't necessarily in mind when HTTP was first designed (especially given the different verbs in HTTP terminology).
Three different definitions of the hypertext transfer protocol (HTTP). Choose one
1. What the most popular HTTP servers actually accept
2. What the most popular HTTP clients actually send
3. What the RFC "standard" says
Personally I choose #1. I don't use the most popular HTTP clients
For example, under definition #1, I can do HTTP/1.1 pipelining with POST
The RFC "standard" often comes after the software implementation(s); most RFCs document what's already in use on the internet. As the selection of software expands, some competent software authors trying their best still often struggle to conform to RFCs. Go figure
To me, the source code of the most popular servers in use is the standard
The other issue with adding a separate supported way to do what people did with GET+body is that we will probably see servers slowly drop support for the GET+body approach when QUERY gets widespread support/usage, and then a ton of other stuff will break.
Unless you're really going to improve things or the existing practices are really too painful, standards should follow convention. Even though GET+body is not handled the same everywhere, it's easier to make that the standard than it is to make a new syntax the standard.
> Even though GET+body is not handled the same everywhere, it's easier to make that the standard than it is to make a new syntax the standard.
Is there some research or study that you base this claim on? What is the reasoning? Can you elaborate?
With standards it helps to reconcile existing behaviour, rather than create new unproven syntax.
Likewise creating a new syntax for something that already exists means you are just adding to the heap of stuff that needs support on mainstream servers, and as I already said it will probably create compatibility issues as the old deprecated/illegal syntax is removed. This is unnecessary friction.
And really what is the advantage of a new syntax? That needs explaining.
But it's not existing behaviour. It does not already exist. Breaking the semantics of something that exists means that things will work inconsistently. Breaking the semantics of get will also create compatibility issues.
You might want to read the article.
Does the article provide supporting evidence for your claim that "Even though GET+body is not handled the same everywhere, it's easier to make that the standard than it is to make a new syntax the standard."? Because if it does I don't see it. To me it just seems like a baseless claim.
This is awesome and very much needed. Sending massive get requests always felt like shit and support for body parsing of GET was all over the place. I hope it will be adopted quickly.
Nice, not having bodies on GET has been a pet peeve of mine for a long time. It would be nice to allow bodies on DELETE as well, but that is less of a problem in most cases.
If you're doing anything complicated enough to need so much data that it'd be better to send the data in a body, it's probably not a DELETE and so POST would be more appropriate anyway.
DELETE is intended to delete one specific object, pointed to by a unique URL, not to delete arbitrary objects matching some criteria.
DELETE is idempotent, so I am not sure what the body would do?
It blows my mind that people are invested in this. HTTP is a 35+ year old text-based protocol. Its becoming the COBOL of digital transmission.
Just as one example, among many, you could try WebSockets (or some other similar protocol) and then push anything over it. Your message could be plain text, JSON, binary, whatever. Web Sockets and protobufs are bidirectional (full duplex) too.
WONIBDFI (whether or not it's broke, don't fix it)
The internet is complex, and you have tons of protocols that are not well supported. TCP/HTTP are well supported by proxies and have well-defined, stable specs, which also help with caching, throttling, etc.
Just because it's old doesn't mean it is worse than alternatives, most likely it is quite the contrary.
Its worse than alternatives because it imposes greater complexity on everybody that uses it. COBOL is also old. That also does not mean COBOL is bad, but nobody wants to write COBOL any more when there are so many better options available.
a 35+ year old text-based protocol with 3 major variants (two of them supporting binary frames), and which is used to communicate most of the internet
yeah, I also don't get the hype, let's go back to everyone declaring their own communication protocol
Its like saying nobody should bother with IPv6 when IPv4 is more common when the real reason is because people don't want to migrate.
I wish there was an HTTP method that directly signals async intent of the server. I know we have 202 Accepted status, but these days there are so many APIs that use the async patterns and each one differs a bit. Having a standard for accepting async jobs with idempotency and notification about results via webhook
I still don't get the need for QUERY. One can create a search or filter resource with a POST request and then query it using GET. As a bonus, creating a resource allows it to be shared and cached.
Obviously the approach you mentioned has the downside of two server round-trips being necessary while the QUERY request only requires a single round-trip. Not to mention the two-request approach adds more complexity to both clients and servers, as it mandates that both the client and server have to physically create and manage those resources.
Obviously, yes, but is it simpler to add a new HTTP method and add support for it everywhere?
Because HTTP is stateless by definition, you now need to support persistence (state) on the server side whenever you want to run a slightly different query, which contradicts the preamble.
I understand the confusion around GraphQL's cached/persisted queries, but this is not the intention of HTTP.
Right, that process works today just fine, but that's application code that the browser (and middleboxes) can't assume. The idea is that QUERY directly represents that flow to the browser as a direct participant (and to caching middleboxes as well). A QUERY can (optionally) create a search or filter resource and return a Location: response header that points to a GET resource to refresh the next time the same QUERY is asked. The browser can directly cache that Location and associate it with that QUERY body. (So can middleboxes.)
A POST can return Content-Location: to the GET resource, but the browser can't assume that the same POST body contents create the same result Content-Location, whereas the QUERY Location represents the QUERY itself as a repeatable object. (Also, QUERY can return Content-Location instead of or in addition to Location for subtly different caching implications.)
What do you think people will make the Query request body? Most everything will use this for JSON but it could be anything so what other interesting things do you think will go in there? Query 1 + 1 and get 2?
I'm curious too. Unless the developer is really passionate about this I don't think a dev will risk (potential) compatibility issues or unexpected footguns to use this when the workarounds do seem to work quite well already. I just dont see the benefit but maybe it's because I am just not aware of a real world use case; happy to be corrected.
Elastic/Opensearch uses GET requests with a body for search, which is complicated or forbidden (not exactly sure) with the HTTP spec. Not all HTTP clients are willing to submit a body with a GET.
So opensearch also allows you to POST search requests, but those are uncacheable
QUERY would fit here perfectly - it's probably trivial for opensearch to add but it will take some time for clients to catch up.
Fixing GET would be easier.
And yes,it would be fixing a flawed interpretation of what should be implemented.you are, by definition GETting something.
Tools dropping body from GET by default are violating the spec today.
Rules configured to drop it are just that, temporarily configured constraints readily modified.
Adding QUERY will make it unpredictable in effectively the same manner as GET/body. It'll take even longer to resolve it though.
> Fixing GET would be easier.
I disagree. I think the adoption (or dismissal) of QUERY will show.
First thing that comes to mind is that the idempotency of GET resources are easy to handle. URL's have a fixed size, they can be efficiently hashed, cached and are unambiguous about how they serve this purpose.
It is unclear how the ecosystem will deal with the QUERY requirements. It's easy for apps, but browsers, http caches and servers will take some time to figure out solutions.
Fixing GET would have the same amount of uncertainty in addition to the need to keep current expectations valid. It's not easier, it's harder.
Browsers won't.
No, really, they simply won't implement it.
Okay, great.
Did they say why? I kinda thought that was the main point.
that's a legit use case. thanks for the info!
Considering the guideline that QUERY should be idempotent and cacheable, file querying requests come to mind. Reverse image search, for example.
GraphQL implementations could switch to this
We might start using QUERY for the search requests from our web app to our server, if nothing in the stack in between the app and our server side code does not drop the body. A JSON body beats the spaghetti arguments of most filters.
Graphql?
Good use case for graphQL requests which use POST for all queries? But then again what about mutations
Presumably good GraphQL libraries would support requiring POST for mutation and QUERY as acceptable for all queries that don't involve mutation.
What are the chances sites start using this to prevent sharing links...
With some post websites (banking) and spa already present.
Unironically QUERY seems to me a "quality of life" feature for the SPA world and the ultimate legitimation to kill link sharing. It is too hard to keep links and app state consistent, so lets just drop it.
Will this be compatible with graphql?
It won't break GraphQL, as it uses POST. It can very much improve it if adopted: - use QUERY method when querying resources - use POST method for mutations
We should have just added optional body support for GET requests.
So much simpler...
More complex actually
The only valid argument against HTTP GET with a body is that it has privacy/security risks.
Exist stuff (caches, CDN, etc.) could serve private information because the HTTP GET is cached without checking the request contents. The new standard can avoid this because old stuff does not know about HTTP QUERY.
This looks like a effort to inject some planned obsolescence in HTTP more than anything else.
This breaks rest/crud.
Is it not just READ? As for rest, why?
How so?
It can actually improve because the semantics which currently are weird, GET is used for Search (Query), listing and also Get resources.
So what?
This is about HTTP.
And it does not break REST: None of the HTTP constructs that REST is built on change due to the introduction of QUERY.
Yes: If you're doing QUERY, you're (potentially) not doing CRUD.
But this enables a clean way to do CQRS over HTTP.
How does this break rest?
Don’t add new stuff (query). Instead fix the broken shit that’s already added (get). Sigh. Xkcd standards.
nice man
If it needs so much explanation and discussion, maybe it is not a great idea after all?
The article describes the current situation first. The whole explanation is quite simple: QUERY requests are the same as GET, but they have a body.
So just add an optional body to get
The article also addresses why this is not the chosen solution. It's pretty much the first one you'd think of: all kinds of existing software (that can be between client and server and out of their control) already handle GET bodies in all kinds of incompatible ways, because the existing standard says they're meaningless and "shouldn't" be included. The idea is to not break people's stuff, so they don't rugpull the established standard.
There's usually a reason why the simplest solution that pops into one's head is not "just" used by the people who put a lot more thought into it. Not always, but it can be useful to try to come up with it.
So, I need to update all the tools to support QUERY, or I need to update all the tools to support GET/body.
So, either way, I need to update all the tools.
Just fix GET.
The point is that if you do that, you end up with lots of undefined behaviour in existing software that has not been patched yet.
If you make it a whole new request method, existing unpatched software should just respond with "Method not allowed".
Which means you have to structure everything around multiple scenarios anyway.
Arguably the only explanation you need is that "QUERY is the same as GET plus a body". The article just explains what GET is and isn't, but that can be implied.
It doesn't really need that much explanation though. TL;DR: It's GET but with bodies officially supported.
I don't think this requires so much discussion; it is a very obvious and simple addition, IMO.
Body is already optional with GET. Proxies aren't supposed to touch it or assign meaning to it; it's between the client and the end server.
A whole new method whose semantics don't really fit with the others is.. An odd way forward.
Yeah I always disliked that there's this idea that you can't put a body on a GET request. Iirc openapi generators goes out of its way to not support that which has lead to me writing a small rant into an API specification before to explain why the get_xyz uses POST...
Proxies are allowed to drop bodies of HTTP GET requests.
RFC 9110 states:
> [..] content received in a GET request has no generally defined semantics, cannot alter the meaning or target of the request [..]
> A client SHOULD NOT generate content in a GET request [..]
Even HTTP 1.0 RFC[11] is fairly clear on this, although it doesn't explicitly spell it out like RFC 9110. GET requests should only consider the Request-URI and request bodies should only be included if the method calls for it.
[1]: https://www.rfc-editor.org/info/rfc1945/
> ... unless it is made directly to an origin server that has previously indicated, in or out of band, that such a request has a purpose and will be adequately supported.
You left out the important part.
How does the origin server indicate that?
In or out of band. ie an API.
semantics become extremely relevant when "proxies" start caching.
A lot of the RFCs are flavored by the lack of https and prevalence of forward proxies run by the ISPs to improve perceived speed and reduce their network loads.. Back in the day.
By my estimation, that's why they explicitly call out only the client and origin can know what a GET body means; proxies should forward as-is and ignore.
Those days of ISP forward http proxies are gone, but those semantics are still fine; the body means what the origin and client agree it means.
>> QUERY request can be cached
I have a weird feeling. Query body is encrypted by https. So CDN will not be able to cache results. In order to make it work right - whole topology of the internet should be redone. Caching on the backend server will not give any real gains for large scale apps.
The whole connection is encrypted by https, the request body is treated the same as the url, the headers or the response. The only unencrypted parts are the IP addresses/ports and the domain name (if SNI without ECH is used).
CDNs already terminate TLS connections so they can cache GET requests.
I think a lot of people don’t know the http/1.x protocol from url (header) to body is a stream of text* separated by \r\n.
* the body may be compressed.
Even past the TLS point (CDNs terminate TLS, so they can read the body) there's a harder problem nobody's solved: to cache a QUERY the cache has to fold the body into the cache key, and there's no standard way to canonicalize a request body. {"a":1,"b":2} and {"b":2,"a":1} are the same query and two different cache entries; whitespace, float formatting, unordered keys all fork the key. GET gets this for free because the URL is already a normalized string. So "cacheable in principle" is real, but "actually cached" needs every layer to agree on a canonical form first - the same coordination problem that killed GET-with-body. I want QUERY for the honest semantics; I just wouldn't budget for cache hits yet
I think the simple approach of using a bitwise comparison will result in satisfactory caching for most applications.
Can you elaborate? Do you think the noted issues are non-issues that should be fixed in applications? Which would require a new standard to create stable JSON outputs, which is just one use case that can fail right now.
> {"a":1,"b":2} and {"b":2,"a":1} are the same query and two different cache entries
These two payloads are actually different. You're talking about semantics, which is determined by the payload format; in the case of JSON, these two are semantically similar.
> GET gets this for free because the URL is already a normalized string
It's the same principle; the order of properties matters too.