Serving 200M requests per day with a CGI-bin

#1

Post by feep » Fri, Jul 04, 2025, 1:16 PM UTC

Serving 200M requests per day with a CGI-bin
jacob.gold

Re: Serving 200M requests per day with a CGI-bin

#2

Post by Traubenfuchs » Fri, Jul 04, 2025, 1:55 PM UTC

Try an apache tomcat 11 next. You can just dump .jsp files or whole java servlet applications as .war file via ssh and it will just work!

One shared JVM for maximum performance!

It can also share db connection pools, caches, etc. among those applications!

Wow!

Re: Serving 200M requests per day with a CGI-bin

#3

Post by atemerev » Fri, Jul 04, 2025, 2:01 PM UTC

Traubenfuchs wrote: post #2

Try an apache tomcat 11 next. You can just dump .jsp files or whole java servlet applications as .war file via ssh and it will just work! One shared JVM for maximum performance! It can also share db connection pools, caches, etc. among those applications! Wow!

I miss this so much. Deployment should be just copying the file (over ssh or whatever). Why people overcomplicated it so much?

Re: Serving 200M requests per day with a CGI-bin

#4

Post by rexreed » Fri, Jul 04, 2025, 2:14 PM UTC

atemerev wrote: post #3

Traubenfuchs wrote: post #2

Try an apache tomcat 11 next. You can just dump .jsp files or whole java servlet applications as .war file via ssh and it will just work! One shared JVM for maximum performance! It can also share db connection pools, caches, etc. among those applications! Wow!

I miss this so much. Deployment should be just copying the file (over ssh or whatever). Why people overcomplicated it so much?

PHP can work the same way. Push / FTP / SFTP PHP file to directory, deployed.

Re: Serving 200M requests per day with a CGI-bin

#5

Post by sugarpimpdorsey » Fri, Jul 04, 2025, 2:18 PM UTC

Surprised with the choice of Apache. There are better choices for serving CGI nowadays. The only reason for still running Apache is you have legacy cruft that requires Apache (like .htaccess).

Re: Serving 200M requests per day with a CGI-bin

#6

Post by mrweasel » Fri, Jul 04, 2025, 2:23 PM UTC

atemerev wrote: post #3

Traubenfuchs wrote: post #2

Try an apache tomcat 11 next. You can just dump .jsp files or whole java servlet applications as .war file via ssh and it will just work! One shared JVM for maximum performance! It can also share db connection pools, caches, etc. among those applications! Wow!

I miss this so much. Deployment should be just copying the file (over ssh or whatever). Why people overcomplicated it so much?

> Why people overcomplicated it so much?

Because a lot of production software is half-baked. If you have to hand over an application to an operations team you need documentation, instrumentation, useful logging, error handling and a ton of other things. Instead software is now stuffed into containers that never receive security updates, because containers make things secure apparently. Then the developers can just dump whatever works into a container and hide the details.

To be fair most of that software is also way more complex today. There are a ton of dependencies and integrations and keeping track of them is a lot of work.

I did work with an old school C programmer that complained that a system we deployed was a ~2GB war file, running on Tomcat and requiring at least 8GB of memory and still crashed constantly. He had on multiple occasions offered to rewrite the how thing in C, which he figured would be <1MB and requiring at most 50MB of RAM to run. Sadly the customer never agreed, I would have loved to see if it had worked out as he predicted.

Re: Serving 200M requests per day with a CGI-bin

#7

Post by mrweasel » Fri, Jul 04, 2025, 2:24 PM UTC

sugarpimpdorsey wrote: post #5

Surprised with the choice of Apache. There are better choices for serving CGI nowadays. The only reason for still running Apache is you have legacy cruft that requires Apache (like .htaccess).

Apache is still a solid option. It does everything, works with everything and is easy to configure. Performance is perfectly fine for ~99% of everything.

Re: Serving 200M requests per day with a CGI-bin

#8

Post by immibis » Fri, Jul 04, 2025, 2:24 PM UTC

sugarpimpdorsey wrote: post #5

Surprised with the choice of Apache. There are better choices for serving CGI nowadays. The only reason for still running Apache is you have legacy cruft that requires Apache (like .htaccess).

I think the point of the experiment was to see how fast the old-school tech stack would go on modern hardware.

Re: Serving 200M requests per day with a CGI-bin

#9

Post by miroljub » Fri, Jul 04, 2025, 2:29 PM UTC

Traubenfuchs wrote: post #2

Try an apache tomcat 11 next. You can just dump .jsp files or whole java servlet applications as .war file via ssh and it will just work! One shared JVM for maximum performance! It can also share db connection pools, caches, etc. among those applications! Wow!

It depends on the application usage pattern. For heavily used applications, sure, it's an excellent choice.

But imagine having to host 50 small applications each serving a couple of hundreds requests per day. In that case, the memory overhead of Tomcat with 50 war files is much bigger than a simple Apache/Nginx server with a CGI script.

Re: Serving 200M requests per day with a CGI-bin

#10

Post by simonw » Fri, Jul 04, 2025, 2:30 PM UTC

I got my start in the CGI era, and it baked into me an extremely strong bias against running short-lived subprocesses for things.

We invented PHP and FastCGI mainly to get away from the performance hit of starting a new process just to handle a web request!

It was only a few years ago that I realized that modern hardware means that it really isn't prohibitively expensive to do that any more - this benchmark gets to 2,000/requests a second, and if you can even get to a few hundred requests a second it's easy enough to scale across multiple instances these days.

I have seen AWS Lambda described as the CGI model reborn and that's a pretty fair analogy.