Myths About CGI Scalability

With a

  ''

it's no earlier than about 1999, when HTML 4.01 came out.

The last big CGI-based web app I did started in 2001. By 2005 (when TurboGears came out) we knew the design was outdated.

Wait, does anyone really use CGI instead of FCGI these days? Let alone like, php-fpm pools and the like?

The general gist I've come up with myself is 1000x processes for fork()/exec() pattern, 100,000 threads for pthreads, and 10,000,000 golang / async / whatever.

Maybe I'm off by a magnitude or two, but that's my general assumption. If you've got less than 1000x processes, CGI probably is fine. But I can't think of any reason to use CGI over FCGI, and FCGI's threads should get you to 100k or so.

I do feel like golang/async offers significantly more performance, but at the tradeoff of extra complexity. There's something to be said about the simplicity innate to collection of .php files in a directory, compared to a Django / Ruby on Rails / Maven Java Tomcat / Golang proxy / etc. etc (or whatever you need to get properly async and/or golang / cooperative multitasking code up)

-----

Then again, I'm not really a web developer.

Anyway, if anyone has reasons to go with fork()/exec() apache / one-process-per-connection CGI 90s style methodology, I'm all ears. Its just... not really that much simpler than FCGI / php-fpm in my experience.

Its not like Python / PHP / Ruby are very fast anyway. We're all doing these langauges because convenience is more important than performance. Still though, there's probably a few orders of magnitude between CGI and FCGI.

> Are there any arguments to not do this in production?

When the way the code is run is also the vehicle for returning output, you necessarily end up in a situation where you expose bad output to the user. This is an insanely common problem with PHP: warnings, errors, and stack traces are regularly returned in HTTP responses by default.

Consider PHP: the output of the code (echo, closing the PHP tag, etc) gets dumped to the HTTP response, unless you configure it otherwise. Error? To the HTTP response. This exposes internal state and may be a security issue (it often is) or reveals your source code.

By contrast, the only way to return data to a HTTP response in Python is to construct a response object and return it. All the built in methods for printing or generating output write to stdout/stderr or the built in python logging framework. You can get PHP-style errors, but only deliberately by having your framework catch exceptions and convert them to responses.

As much as this seems like a non-issue, it's actually really dangerous when files map to URLs, because you don't want all your files being URLs. Some files are supposed to be shared, or only loaded in certain circumstances. If you can access arbitrary chunks of code directly, and then read any error output from them (or even regular-but-unintended output), you could be leaking data. Without lots of explicit manual boilerplate to block this direct access, you're running a huge risk.

And this isn't just speculation. I've worked at a large PHP shop, and this was a frequent source of frustration and pain. Nobody is immune from this.

> As this page is rendered with CGI, we don't get a Last-Modified for it. That's a CGI disadvantage :p

I'm assuming you already know that's false, based on the emoticon, but for anyone else reading... CGI gives you the flexibility to set (or not set) the Last-Modified header as you see fit.

Additionally, CGI is a pretty decent way to give web access to a static site generator. Write or copy your Markdown in a textbox, hit submit, md file is saved, SSG is run, done.

The HTTP server/proxy is only going to handle TLS termination. It can also handle basic resource authentication in a pinch. But for all other auth tasks need to be done by the application. Just because a user can access a URL doesn't mean they have access to a particular row in a database or write access or something. The user data may not even live locally and be provided by an external provider which is also beyond the scope of the HTTP server.

I don't know why you would think it's the HTTP front end's job to do any of that. It's all clearly the domain of the back end application.