Serving a half billion requests per day with Rust and CGI • Hacker News Board

Re: Serving a half billion requests per day with Rust and CGI

#81

Post by rajaravivarma_r » Wed, Jul 09, 2025, 4:19 PM UTC

jchw wrote: post #8

Honestly, I'm just trying to understand why people want to return to CGI. It's cool that you can fork+exec 5000 times per second, but if you don't have to, isn't that significantly better? Plus, with FastCGI, it's trivial to have separate privileges for the application server and the webserver. The CGI model may still work fine, but it is an outdated execution model that we left behind for more than one reason, not j…

I'm wondering the same, but honestly I have a soft corner for the old way of doing things as well, and I think it stems from it.

The performance numbers seem to show how bad it is in real world.

For testing I converted the CGI script into a FastAPI script and benchmarked it on my MacBookPro M3. I'm getting super impressive performance numbers,

Read ``` Statistics Avg Stdev Max Reqs/sec 2019.54 1021.75 10578.27 Latency 123.45ms 173.88ms 1.95s HTTP codes: 1xx - 0, 2xx - 30488, 3xx - 0, 4xx - 0, 5xx - 0 others - 0 Throughput: 30.29MB/s ``` Write (shown in the graph of the OP) ``` Statistics Avg Stdev Max Reqs/sec 931.72 340.79 3654.80 Latency 267.53ms 443.02ms 2.02s HTTP codes: 1xx - 0, 2xx - 0, 3xx - 13441, 4xx - 0, 5xx - 215 others - 572 Errors: timeout - 572 Throughput: 270.54KB/s ```

At this point, the contention might be the single SQL database. Throwing a beefy server like in the original post would increase the read performance numbers pretty significantly, but wouldn't do much on the write path.

I'm also thinking that at this age, one needs to go out of their way to do something with CGI. All macro, micro web frameworks comes with a HTTP server and there are plenty of options. I wouldn't do this for anything apart from fun.

FastAPI-guestbook.py https://gist.github.com/rajaravivarma-r/afc81344873791cb52f3...

Re: Serving a half billion requests per day with Rust and CGI

#82

Post by kragen » Wed, Jul 09, 2025, 4:22 PM UTC

jchw wrote: post #78

9rx wrote: post #75

Earlier quoted context omitted.

> No, it's not. It is not strictly limited to the CGI protocol, of course, but it is the marketing term for the concept of the application not acting as the server, of which CGI applications would be included. CGI, like all serverless applications, outsource the another process, such as Apache or nginx, to provide the server. Hence the literal name. > "Serverless" is a marketing term for a fully managed offering wher…

The term "serverless" is a generic PaaS marketing term to refer to managed services where you don't have to manage a server to use them, e.g. "Amazon Aurora Serverless". If you're managing CGI scripts on a traditional server, you're still managing a server. The point isn't really that the application is unaware of the server, it's that the server is entirely abstracted away from you. CGI vs serverless is apples vs or…

> If you're managing CGI scripts on a traditional server, you're still managing a server.

Usually somebody else is managing the server, or servers, so you don't have to think about it. That's been how it's worked for 30 years.

> Before PaaSes I don't think I've ever seen anyone once call CGI "serverless".

No, because "serverless" was a marketing term invented to sell PaaSes because they thought that it would sell better than something like "CloudCGI" (as in FastCGI or SpeedyCGI, which also don't use the CGI protocol). But CGI hosting fits cleanly within the roomy confines of the term.

Re: Serving a half billion requests per day with Rust and CGI

#83

Post by kragen » Wed, Jul 09, 2025, 4:29 PM UTC

jchw wrote: post #74

kragen wrote: post #71

Earlier quoted context omitted.

CGI and other "serverless" technologies have essentially the same benefits and drawbacks. Sometimes an AWS Lambda function has longer startup time than if you had a running process already waiting to service a web request, because it's spinning up (AFAIK) an entire VPS. So all the arguments for "serverless" are also arguments for CGI, and all the arguments against CGI are arguments against "serverless". That's the se…

> CGI and other "serverless" technologies have essentially the same benefits and drawbacks. Sometimes an AWS Lambda function has longer startup time than if you had a running process already waiting to service a web request, because it's spinning up (AFAIK) an entire VPS. So all the arguments for "serverless" are also arguments for CGI, and all the arguments against CGI are arguments against "serverless". Honestly th…

I appreciate the notes on the different nuances of "serverless".

I think you might not be very familiar with how people typically used CGI in the 01990s and 02000s, because you say "[serverless] is a return to a simpler CGI-like programming model, but it's pretty explicitly about the fact that you don't have to manage the server..." when that was the main reason to use CGI rather than something custom at the time; you could use a server that someone else managed. But you seem to think it was a difference rather than a similarity.

Why do you suppose we were running our CGI scripts under NCSA httpd before Apache came out? It wasn't because the HTTP protocol was super complicated to implement. I mean, CGI is a pretty thin layer over HTTP! But you can implement even HTTP/1.1 in an afternoon. It was because the guys in the computer center had a server (machine) and we didn't. Not only didn't we have to manage the server; they wouldn't let us!

As for Python, yeah, I'm pretty disenchanted with Python right now too, precisely because the whole Python ecosystem is just constantly breaking. And static files are kind of a problem for Django; it's optimized for serving them from a separate server.

Re: Serving a half billion requests per day with Rust and CGI

#84

Post by jchw » Wed, Jul 09, 2025, 4:30 PM UTC

9rx wrote: post #80

jchw wrote: post #78

Earlier quoted context omitted.

The term "serverless" is a generic PaaS marketing term to refer to managed services where you don't have to manage a server to use them, e.g. "Amazon Aurora Serverless". If you're managing CGI scripts on a traditional server, you're still managing a server. The point isn't really that the application is unaware of the server, it's that the server is entirely abstracted away from you. CGI vs serverless is apples vs or…

> It is being sold to businesses, that they don't have to manage a server. Do you mean a... computer ? Server is a software term. It is a process that listens for network requests. At least since CGI went out of fashion, embedding a server right in your application has been the style. Serverless sees a return to the application being less a server, pushing the networking bits somewhere else. Modern solutions may not…

> Do you mean a... computer? Server is a software term. It is a process that listens for network requests.

Between this and the guy arguing that UNIX doesn't have "folders" I can see that these kinds of threads bring out the most insane possible lines of rhetoric. Are you sincerely telling me right now you've never seen the term "server" used to refer to computers that run servers? Jesus Christ.

Pedantry isn't a contest, and I'm not trying to win it. I'm not sitting here saying that "Serverless is not a marketing term for CGI" to pull some epic "well, actually..." I'm saying it because God damnit, it's true. Serverless was a term invented specifically by providers of computers-that-aren't-yours to give people options to not need to manage the computers-that-aren't-yours. They actually use this term serverless for many things, again including databases, where you don't even write an application or a server in the first place; we're just using "serverless" as a synonym for "serverless function", which I am fine to do, but pointing that out is important for more than just pedantry reasons because it helps extinguish the idea that "serverless" was ever meant to have anything to do with application design. It isn't and doesn't. Serverless is not a marketing term for CGI. Not even in a relaxed way, it's just not. The selling point of Serverless functions is "you give us your request handler and we'll handle running it and scaling it up".

This has nothing to do with the rise of embedding a server into your application.

Re: Serving a half billion requests per day with Rust and CGI

#85

Post by 9rx » Wed, Jul 09, 2025, 4:35 PM UTC

jchw wrote: post #84

9rx wrote: post #80

Earlier quoted context omitted.

> It is being sold to businesses, that they don't have to manage a server. Do you mean a... computer ? Server is a software term. It is a process that listens for network requests. At least since CGI went out of fashion, embedding a server right in your application has been the style. Serverless sees a return to the application being less a server, pushing the networking bits somewhere else. Modern solutions may not…

> Do you mean a... computer? Server is a software term. It is a process that listens for network requests. Between this and the guy arguing that UNIX doesn't have "folders" I can see that these kinds of threads bring out the most insane possible lines of rhetoric. Are you sincerely telling me right now you've never seen the term "server" used to refer to computers that run servers? Jesus Christ. Pedantry isn't a cont…

> Serverless was a term invented specifically by providers of computers-that-aren't-yours to give people options to not need to manage the computers-that-aren't-yours.

No. "Cloud" was the term invented for that, inherited from networking diagrams where it was common to represent the bits you don't manage as cloud figures. Usage of "Serverless" emerged from AWS Lamba, which was designed to have an execution model much like CGI. "Severless" refers to your application being less a server. Lamba may not use CGI specifically, but the general idea is very much the same.

Re: Serving a half billion requests per day with Rust and CGI

#86

Post by kragen » Wed, Jul 09, 2025, 4:35 PM UTC

jchw wrote: post #84

9rx wrote: post #80

Earlier quoted context omitted.

> It is being sold to businesses, that they don't have to manage a server. Do you mean a... computer ? Server is a software term. It is a process that listens for network requests. At least since CGI went out of fashion, embedding a server right in your application has been the style. Serverless sees a return to the application being less a server, pushing the networking bits somewhere else. Modern solutions may not…

> Do you mean a... computer? Server is a software term. It is a process that listens for network requests. Between this and the guy arguing that UNIX doesn't have "folders" I can see that these kinds of threads bring out the most insane possible lines of rhetoric. Are you sincerely telling me right now you've never seen the term "server" used to refer to computers that run servers? Jesus Christ. Pedantry isn't a cont…

> Serverless is not a marketing term for CGI. Not even in a relaxed way, it's just not. The selling point of Serverless functions is "you give us your request handler and we'll handle running it and scaling it up".

That was the selling point of CGI hosting though. Except that the "scaling it up" part was pretty rare. There were server farms that ran CGI scripts (NCSA had a six-server cluster with round-robin DNS when they first published a paper describing how they did it, maybe 01994) but the majority of CGI scripts were almost certainly on single-server hosting platforms.

Re: Serving a half billion requests per day with Rust and CGI

#87

Post by KronisLV » Wed, Jul 09, 2025, 4:36 PM UTC

ptsneves wrote: post #39

neilv wrote: post #22

One reason to use CGI is legacy systems. A large, complex, and important system that I inherited was still using CGI (and it worked, because a rare "10x genuinely more productive" developer built it). Many years later, to reduce peak resource usage, and speed up a few things, I made an almost drop-in replacement library, to permit it to also run with SCGI (and back out easily to CGI if there was a problem in producti…

I found that ChatGPT revived vanilla javascript and jquery for me. Most of the chore part is done by chatgpt and the mental model of understanding what it wrote is very light and often single file. It is also easily embedded in static file generators. On the contrary Vue/React have a lot of context required to understand and mentally parse. On react the useCallback/useEffect/useMemo make me need to manually manage de…

> I found that ChatGPT revived vanilla javascript and jquery for me.

I used jQuery in a project recently where I just needed some interactivity for an internal dashboard/testing solution. I didn't have a bunch of time to setup a whole toolchain for Vue (and Pinia, Vue Router, PrimeVue, PrimeIcons, PrimeFlex and the automated component imports) because while I like using all of them and the developer experience is quite nice, the setup still takes a bit of time unless you have a nice up to date boilerplate project that's ready to go.

Not even having a build step was also really pleasant, didn't need to do complex multi-stage builds or worry that copying assets would somehow slow down a Maven build for the back end (relevant for those cases when you package your front end and back end together in the same container and use the back end to serve the front end assets, vs two separate containers where one is just a web server).

Only problem was that jQuery doesn't compose as nice, I missed the ability to nest a bunch of components. Might just have to look at Lit or something.

Re: Serving a half billion requests per day with Rust and CGI

#88

Post by kragen » Wed, Jul 09, 2025, 4:41 PM UTC

masklinn wrote: post #79

kragen wrote: post #64

Earlier quoted context omitted.

There's a fsync/close/rename dance that ext4fs recognizes as a safe, durable atomic file replacement, which is often sufficient for preventing data loss in cases like this.

FWIW POSIX requires that rename(2) be atomic, it's not just ext4, any POSIX FS should work that way. However this still requires a lockfile because while rename(2) is an atomic store it's not a full CAS, so you can have two processes reading the file concurrently, doing their internal update, writing to a temp file, then rename-ing to the target. There will be no torn version of the reference file, but the process fi…

I forget the particular failure mode that POSIX arguably permitted in this case. I think it had to do with fsyncing the directory? And the file ending up existing but empty after the reboot? Anyway, it broke userspace, so the ext4fs maintainers killed it. I think there were some LWN articles.

I might have been misremembering: https://lwn.net/Articles/322823/

"With ext4's delayed allocation, the metadata changes can be journalled without writing out the blocks. So in case of a crash, the metadata changes (that were journalled) get replayed, but the data changes don't."

That was without fsync, though. But apparently it's a topic of discussion more recently: https://lwn.net/Articles/789600/

I agree that it still requires a lockfile if write conflicts are not acceptable.

Re: Serving a half billion requests per day with Rust and CGI

#89

Post by SahAssar » Wed, Jul 09, 2025, 4:45 PM UTC

toast0 wrote: post #55

jchw wrote: post #36

Earlier quoted context omitted.

The part of the execution model that is dated is this: > having the web server execute stuff in a specific folder inside the document root just seems like a recipe for problems

Typically I've run cgi from a directory outside the document root. That's easy, and I think was the defaults? That said, fork+exec isn't the best for throughput. Especially if the httpd doesn't isolate forking into a separate, barebones, child process, fork+exec involves a lot of kernel work. FastCGI or some other method to avoid forking for each request is valuable regardless of runtime. If you have a runtime with h…

> FastCGI or some other method to avoid forking for each request is valuable regardless of runtime. If you have a runtime with high startup costs, even more so.

What's the point of using FastCGI compared to a plain http server then? If you are going to have a persistent server running why not just use the protocol you are already using the semantics of?

Re: Serving a half billion requests per day with Rust and CGI

#90

Post by jchw » Wed, Jul 09, 2025, 4:47 PM UTC

kragen wrote: post #82

jchw wrote: post #78

Earlier quoted context omitted.

The term "serverless" is a generic PaaS marketing term to refer to managed services where you don't have to manage a server to use them, e.g. "Amazon Aurora Serverless". If you're managing CGI scripts on a traditional server, you're still managing a server. The point isn't really that the application is unaware of the server, it's that the server is entirely abstracted away from you. CGI vs serverless is apples vs or…

> If you're managing CGI scripts on a traditional server, you're still managing a server. Usually somebody else is managing the server, or servers, so you don't have to think about it. That's been how it's worked for 30 years. > Before PaaSes I don't think I've ever seen anyone once call CGI "serverless". No, because "serverless" was a marketing term invented to sell PaaSes because they thought that it would sell bet…

Oh my god! This could go on forever.

Having a guy named Steve manage your servers is not "serverless" by my definition, because it's not about you personally having to manage the server, it's about anyone personally having to manage it. AWS Lambda is managed by Amazon as a singular giant computer spawning micro VMs. And sure yes, some human has to sit here and do operations, but the point is that they've truly abstracted the concept of a running server from both their side and yours. It's abstracted to the degree that even asking "what machine am I running on?" doesn't even have a meaningful answer and if you did have the answer you couldn't do anything with it.

Shared hosting with a cgi-bin is closer to this, but it falls short of fully abstracting the details. You're still running on a normal-ish server with shared resources and a web server configuration and all that jazz, it's just that you don't personally have to manage it... But someone really does personally have to manage it.

And anyway, there's no reason to think that serverless platforms are limited to things that don't actually run a server. On the contrary there are "serverless" platforms that run servers! Yes, truly, as far as I know containers running under cloud run are in fact normal HTTP servers. I'm actually not an expert on serverless despite having to be on this end of the argument, but I'll let Google speak for what it means for Cloud Run to be "serverless":

> Cloud Run is a managed compute platform that enables you to run stateless containers that are invocable via HTTP requests. Cloud Run is serverless: it abstracts away all infrastructure management, so you can focus on what matters most — building great applications.

These PaaSes popularized the term to mean this from the gitgo, just because you have passionately formed a belief that it ever meant something else doesn't change a thing.