Confusion Attacks: Exploiting Hidden Semantic Ambiguity in Apache HTTP Server

Also slides: https://i.blackhat.com/BH-US-24/Presentations/US24-Orange-Co...

In big letters, near the beginning:

> Why Apache HTTP Server Smells Bad?

In the end:

> The entire development process must be carefully built upon a pile of legacy code and technical debt. So, if any Apache HTTP Server developers are reading this: Thank you for your hard work and contributions!

Quite the tone shift, a bit amusing, but definitely an attention catching choice and immensely cool that people are doing research like this.

I'm fairly sure that most large systems have a bunch of potential vulnerabilities, for example you can most likely do more with Apache than you can with Nginx due to its dynamic nature and module system, but at the same time that comes with dangers like this.

> Apache has probably more unofficial modules, but that's irrelevant to the core developpers.

In a way it's irrelevant, but in a way not. The myth was that they called it Apache because it's "A patchy" web server.[1] IOW it was designed to use modules to do most of its work.

[1] https://dan.hersam.com/2004/07/29/a-patchy-web-server/

> that's irrelevant to the core developers

That's relevant to me! If I want Apache to act as an OpenID connect relying party, for example, I can just use mod_auth_openidc https://github.com/OpenIDC/mod_auth_openidc installing which is pretty much trivial and requires little work on my part:

  sudo apt install libapache2-mod-auth-openidc

If I want something similar for Nginx, I can take a look here https://openid.net/developers/certified-openid-connect-imple... and realize that I'll need to use OpenResty with LuaRocks, which does add a bit more complexity: https://github.com/zmartzone/lua-resty-openidc

   luarocks install lua-resty-openidc

If I want a basic WAF with something that runs ModSecurity rules, on Apache I can just do:

  libapache2-mod-security2

and can use the OWASP CRS with no issues: https://owasp.org/www-project-modsecurity-core-rule-set/

With Nginx that is no longer properly supported and needed their Plus version anyways: https://docs.nginx.com/nginx-waf/admin-guide/nginx-plus-mods...

  sudo apt-get install nginx-plus-module-modsecurity

If I want to do that myself with the regular Nginx, I suddenly have to compile a bunch of stuff myself, which becomes far less pleasant and a bit more long winded: https://www.linode.com/docs/guides/securing-nginx-with-modse...

  sudo apt-get install bison build-essential ca-certificates curl dh-autoreconf doxygen \
  flex gawk git iputils-ping libcurl4-gnutls-dev libexpat1-dev libgeoip-dev liblmdb-dev \
  libpcre3-dev libpcre++-dev libssl-dev libtool libxml2 libxml2-dev libyajl-dev locales \
  lua5.3-dev pkg-config wget zlib1g-dev zlibc libxslt libgd-dev
  sudo apt install git
  cd /opt && sudo git clone https://github.com/SpiderLabs/ModSecurity
  cd ModSecurity
  sudo git submodule init
  sudo git submodule update
  sudo ./build.sh
  sudo ./configure
  sudo make
  sudo make install
  cd /opt && sudo git clone --depth 1 https://github.com/SpiderLabs/ModSecurity-nginx.git
  cd /opt && sudo wget http://nginx.org/download/nginx-1.14.0.tar.gz
  sudo tar -xvzmf nginx-1.14.0.tar.gz
  cd nginx-1.14.0
  sudo ./configure --add-dynamic-module=../ModSecurity-nginx 
  sudo make modules
  sudo mkdir /etc/nginx/modules
  sudo cp objs/ngx_http_modsecurity_module.so /etc/nginx/modules

(old versions in the example, but same principle)

At that point I might as well look in the direction of OWASP Coraza: https://coraza.io/

When it comes to ACME and Let's Encrypt, I can use mod_md out of the box with Apache: https://httpd.apache.org/docs/2.4/mod/mod_md.html

It's experimental but works nicely in my experience, whereas with Nginx, I have to go with Certbot separately: https://certbot.eff.org/

If I want everything in a single package, I can just use Caddy instead (which is a lovely web server).

And that's before I get into weird Nginx behavior when I try to use it with containers, where the whole web server crashes, just because it cannot find the DNS record for a container that hasn't been created yet, along the lines of:

  nginx: [emerg] host not found in upstream "my-app"

where trying to use the suggested fix leads to:

  nginx: [emerg] "proxy_redirect default" cannot be used with "proxy_pass" directive with variables

and using other values often breaks app redirects.

Due to these experiences, I'll absolutely say that you can do more with Apache, in ways that will be less of a pain, except for the whole "getting your entire web server hacked" or "brought down by any sizeable amount of traffic" bit. Nginx is really good for hosting static content or acting as a basic reverse proxy, but will be harder to get various bits of the greater ecosystem into it, as opposed to how Apache works.

Then again, I fully acknowledge that someone will come along and say "ModSecurity isn't very useful, you should just put your site behind Cloudflare if you need a WAF." or maybe even "Don't handle OpenID Connect in your ingress, even if that means that in the case of 5 APIs you'll need OIDC libraries and configuration for each of them, which will look quite different in various languages." but that doesn't change the fact that some things are just more doable in one option instead of another.

OpenResty or Caddy increasingly feel like better choices for a lot of stuff, but not bare Nginx. Use Nginx when you want to handle simple use cases (reverse proxy, serving static files) with good performance, security and a sane configuration format.