Earlier quoted context omitted.
Out of the ecosystems I’m familiar with, Python application servers have terrible http2 support: neither gunicorn nor uwsgi supports it, and even new hotness like uvicorn is pretty far from it. I don’t think Ruby is doing much better? Correct me if I’m wrong.
But why would you need HTTP/2 perfect support in real world application server? They are never going to terminate the client traffic, they will speak with a load balancer which can speak HTTP/1.1 with them. Sure, if you are at webscale or even less you want everything on HTTP/2 for optimization sake. But in the rest of cases, even if you are in a solo project, you can easily enough put an nginx before it, or a cloud…
HTTP/2-exclusive threats caused by implementation flaws and RFC imperfections
71–80 of 116 posts
Re: HTTP/2-exclusive threats caused by implementation flaws and RFC imperfections
#72Just a quick question about the H2.CL case study. Did Netflix have a separate vulnerability where their server wrongly trusted Host headers with netflix.com suffix and returned a 302 response which can redirected users to an arbitrary host? I'm just trying to see whether I interpret the case study correctly.
Re: HTTP/2-exclusive threats caused by implementation flaws and RFC imperfections
#73The title seem very much anti-HTTP/2. However, unless I missed something huge, the vast majority, if not all, of the issues found relate to how hard it is to handle HTTP/1. And it seems kinda odd to blame HTTP/2 for HTTP/1 being difficult to implement. I'm certainly not trying to downplay the seriousness of these issues. But it seems like a equally (it not more) valid title might be something like "HTTP/1: Continues…
As far as I can tell, the article doesn't attack HTTP2, that seems work fine. The article clearly demonstrates the problems of HTTP1. But the real problem is sloppy HTTP2 forntends that generate broken HTTP1.
Re: HTTP/2-exclusive threats caused by implementation flaws and RFC imperfections
#74Really good explanations here. As a Cloudflare Enterprise customer I've opened a case with them to see if any of these vulnerabilities apply to their WAF. I was a bit surprised to see the article mention that Imperva's WAF was vulnerable. edit: Cloudflare has responded that "[they] are asking your question internally."
Client → (HTTP/2) → Cloudflare ingress → (???) → Cloudflare egress → (???) → Your endpoint → (???) → Your web app
On your endpoint (LB or webserver) logs you can check if the requests coming in from Cloudbees are using HTTP/2 or not. If they do use HTTP/2, they internally might still do translation down to HTTP/1.1 and back.
But more importantly, you should investigate how your LB or webserver talks to the web app. In many cases that part won't be TLS encrypted and therefore on an HTTP/1.1 channel.
The exception is if your outward facing, TLS terminating, webserver interacts with the web app via CGI or FastCGI. While that part of the call stack would not be affected by this particular attack, (Fast)CGI comes with it's own set of risks.
In the end, the only part that you can't easily validate yourself of being susceptible to this attack (aka having to translate between HTTP/2 and HTTP/1.1) is the bit that occurs internally to Cloudflare.
Re: HTTP/2-exclusive threats caused by implementation flaws and RFC imperfections
#75Thanks for this @albinowax_ - super interesting! Just a quick question about the H2.CL case study. Did Netflix have a separate vulnerability where their server wrongly trusted Host headers with netflix.com suffix and returned a 302 response which can redirected users to an arbitrary host? I'm just trying to see whether I interpret the case study correctly.
I wouldn't exactly class this as a vulnerability - more of a useful gadget. The front-end would refuse to forward such a request so it's impossible to hit this code path without request smuggling. Even if you could hit it without request smuggling it would still be useless.
Re: HTTP/2-exclusive threats caused by implementation flaws and RFC imperfections
#76Earlier quoted context omitted.
I think the main problem is indeed missing HTTP2 support in backend servers. This is often just a case of people not being willing to upgrade for various reasons, even if the technology they are using would support HTTP2 in newer versions.
The problem is that HTTP/2 pretty much forces encryption. Most people don't want to deal with certificate management/rotation on every single microservice's application server.
Terminating HTTP/2 over TLS on a web frontend and then HTTP/2 over plaintext to the application servers sounds like a viable model.
Re: HTTP/2-exclusive threats caused by implementation flaws and RFC imperfections
#77Earlier quoted context omitted.
The whole point of this article is that proxies speaking HTTP2 with clients and HTTP1.1 with servers introduce new vulnerabilities. The author found such vulnerabilities in AWS ALB, several WAF solutions, F5 BigIP, and others.
Yeah but serving traffic from an application server directly is probably even worse in plethora of other failure modes. EDIT: and yes I understand that you should use http/2 on the LB and http/2 on the backend to get the best of both worlds. EDIT2: anyway my opinion is that the general reaction to a security discovery like this one shouldn't be "let's stop using this tech immediately" but "let's get this patched ASAP…
Re: HTTP/2-exclusive threats caused by implementation flaws and RFC imperfections
#78Burying the lede... Unfortunately, at the time of this whitepaper being published - 86 days after Apache was notified of the vulnerability - 2.4.49 not yet come out, so although there's a patch on master, this is effectively a zero-day.
I'm not seeing anything about this in nginx changelogs (or matching CVE's) either. Disabling http2 for now.
Re: HTTP/2-exclusive threats caused by implementation flaws and RFC imperfections
#79The title seem very much anti-HTTP/2. However, unless I missed something huge, the vast majority, if not all, of the issues found relate to how hard it is to handle HTTP/1. And it seems kinda odd to blame HTTP/2 for HTTP/1 being difficult to implement. I'm certainly not trying to downplay the seriousness of these issues. But it seems like a equally (it not more) valid title might be something like "HTTP/1: Continues…
Then, the \r\n in headers can easily trip an application that was naively ported to http/2 supporting network library. That's too pretty ripe for blame shifting :)
Re: HTTP/2-exclusive threats caused by implementation flaws and RFC imperfections
#80The title seem very much anti-HTTP/2. However, unless I missed something huge, the vast majority, if not all, of the issues found relate to how hard it is to handle HTTP/1. And it seems kinda odd to blame HTTP/2 for HTTP/1 being difficult to implement. I'm certainly not trying to downplay the seriousness of these issues. But it seems like a equally (it not more) valid title might be something like "HTTP/1: Continues…
Yes, it is weird that a broken protocol translator results in blame for the newer protocol. As far as I can tell, the article doesn't attack HTTP2, that seems work fine. The article clearly demonstrates the problems of HTTP1. But the real problem is sloppy HTTP2 forntends that generate broken HTTP1.
From a look at the HTTP/2 spec. this protocol translation is an expected use case and explicitly results in several restrictions on the contents of HTTP/2 headers. So going by the HTTP/2 spec. at least some of those headers should have been rejected by a conformant front end and never made it into HTTP/1.